Augmented reality feedback of inventory for an appliance

Abstract

A food storage system is disclosed. The system comprises an appliance having a cabinet defining an interior, the interior comprising at least one of a shelf, compartment, or drawer configured to store at least one food item, a vision system comprising at least one imager configured to record image data related to the at least one food item stored within the interior, and a controller in communication with the vision system. The controller can be operable to generate an augmented reality representation of the interior.

What is claimed is: 
     
         1 . An inventory management system for an appliance, the inventory management system comprising:
 a holder disposed in an interior of the appliance and configured to store a food item;   an imaging device configured to capture an image of the interior;   a display in communication with the imaging device configured to present image data corresponding to the image; and   a controller in communication with the imager and the display, the controller configured to:
 determine an identity of the food item; 
 determine a target location for the food item based on the identity of the food item and an environmental condition of the interior; and 
 communicate an instruction to present the image data with an indication of the target location at the display. 
   
     
     
         2 . The inventory management system according to  claim 1 , wherein the image data includes an augmented reality representation including a photographic representation of the interior with a digital representation of the food item overlaying a digital representation of a position of the food item in the interior. 
     
     
         3 . The inventory management system according to  claim 2 , wherein the controller is further configured to generate the augmented reality representation based on the identity of the food item. 
     
     
         4 . The inventory management system according to  claim 2 , wherein the indication is a color indicator and the photographic representation of the interior includes the color indicator. 
     
     
         5 . The inventory management system according to  claim 2 , wherein the digital representation of the food item is presented as rendered graphics of the at least one food item. 
     
     
         6 . The inventory management system according to  claim 1 , wherein the controller is further configured to:
 determine an expiration date of the food item based on a position of the food item in the interior and a heat distribution of the interior; and   communicate an instruction to present the image data with the expiration date at the display.   
     
     
         7 . The inventory management system according to  claim 1 , wherein the image data includes an overlay corresponding to a property of the food item. 
     
     
         8 . The inventory management system according to  claim 7 , wherein the property is a fill level and the overlay is a graphical image configured to indicate the fill level of the food item. 
     
     
         9 . The inventory management system according to  claim 7 , wherein the property is an expiration date and the overlay is a graphical image indicating the expiration date of the food item. 
     
     
         10 . The inventory management system according to  claim 7 , wherein the property is a favorite and the overlay is a graphical image indicating that the food item is a favorite. 
     
     
         11 : An inventory management system for an appliance, the inventory management system comprising:
 at least one holder disposed in an interior of the appliance and configured to store a plurality of food items;   an imaging device configured to capture an image of the interior;   a display in communication with the imaging device configured to present image data corresponding to the image; and   a controller in communication with the imager and the display, the controller configured to:
 determine an identity of each of one or more of the food items; 
 determine a target location for each of one or more of the food items based on the identity each of one or more of the food items and an environmental condition of the interior; 
 generate an augmented reality representation including a photographic representation of the interior with a digital representation of the food item overlaying a digital representation of a position of the food item in the interior, wherein the image data includes the augmented reality representation; and 
 communicate a first instruction to present the augmented reality representation with an indication of each target location at the display. 
   
     
     
         12 . The inventory management system according to  claim 11 , wherein presenting the augmented reality representation includes displaying food item overlays corresponding to the plurality of food items. 
     
     
         13 . The inventory management system according to  claim 11 , further comprising:
 a lighting system in communication with the controller and configured to project a pattern of light in the interior, wherein the controller is further configured to control the lighting system to indicate the target location via the pattern of a light.   
     
     
         14 . The inventory management system according to  claim 13 , wherein the controller is further configured to control the lighting system to display a color spectrum in the interior indicating the environmental condition. 
     
     
         15 . The inventory management system according to  claim 11 , wherein the image data includes an overlay corresponding to a property of the food item. 
     
     
         16 . The inventory management system according to  claim 15 , wherein the property is a fill level and the overlay is a graphical image configured to indicate the fill level of the food item. 
     
     
         17 . The inventory management system according to  claim 15 , wherein the property is an expiration date and the overlay is a graphical image indicating the expiration date of the food item. 
     
     
         18 . The inventory management system according to  claim 11 , wherein the controller is further configured to:
 receive recipe data corresponding to a recipe incorporating one or more of the plurality of food items;   communicate a second instruction to present a color indictor indicating positions of the one or more of the plurality of food items at the display.   
     
     
         19 . An inventory management system for an appliance, the inventory management system comprising:
 a holder disposed in an interior of the appliance and configured to store a food item;   an imaging device configured to capture an image of the interior;   a display in communication with the imaging device configured to display image data corresponding to the image; and   a controller in communication with the imager and the display, the controller configured to:
 determine a position of the food item based on the image; 
 determine a target location for the food item based on an environmental condition of the interior; 
 compare the position to the target location; 
 modify the image data to indicate the target location based on the comparison of the position to the target location; and 
 communicate an instruction to present the image data at the display. 
   
     
     
         20 . The inventory management system according to  claim 19 , wherein the image data includes an augmented reality representation including a photographic representation of the interior with a digital representation of the food item overlaying a digital representation of a position of the food item in the interior, wherein the photographic representation of the interior includes a color indicator indicating the comparison of the position to the target location, and wherein the environmental condition is a heat distribution of the interior.