Image-based spoilage sensing refrigerator

Abstract

Disclosed are various embodiments for determining whether food in a refrigerator has spoiled. In one embodiment, among others, a refrigerator comprises a compartment configured to contain a food item and an infrared camera configured to capture an infrared image of the compartment. The refrigerator also includes a network interface configured to communicate with a remote computing device.

Therefore, the following is claimed: 
     
         1 . A refrigerator, comprising:
 a compartment having an interior space configured to contain a food item;   an infrared camera mounted within the refrigerator and directed to capture an infrared image of the interior space of the compartment; and   a network interface configured to communicate with a remote computing device.   
     
     
         2 . The refrigerator of  claim 1 , further comprising:
 a processor;   a memory; and   an application stored in the memory that, when executed by the processor, causes the refrigerator to at least:
 capture the infrared image of the interior space of the compartment with the infrared camera; and 
 send the infrared image of the interior space of the compartment to the remote computing device via the network interface. 
   
     
     
         3 . The refrigerator of  claim 2 , wherein:
 the compartment further comprises a scale configured to determine a weight of the food item in the compartment;   the application further causes the refrigerator to determine that the food item is located in the compartment based at least in part on the weight of the food item; and   wherein the application causes the refrigerator to capture the infrared image of the interior space of the compartment occurs in response to a determination that the food item is located in the compartment.   
     
     
         4 . The refrigerator of  claim 2 , further comprising a visible spectrum camera configured to capture an image of the interior space of the compartment and wherein the application further causes the refrigerator to at least:
 determine that a door of the refrigerator is open;   capture a visible spectrum image of the interior space of the compartment in response to a determination that the door of the refrigerator is open; and   send the visible spectrum image of the interior space of the compartment to the remote computing device via the network interface.   
     
     
         5 . The refrigerator of  claim 4 , wherein:
 the compartment further comprises a scale configured to determine a weight of the food item in the compartment;   the application further causes the refrigerator to determine that the food item is located in the compartment based at least in part on the weight of the food item; and   wherein the application causes the refrigerator to capture the visible spectrum image of the interior space of the compartment occurs in response to a determination that the food item is located in the compartment.   
     
     
         6 . The refrigerator of  claim 4 , wherein the visible spectrum camera is located in the compartment. 
     
     
         7 . The refrigerator of  claim 2 , wherein the infrared camera is located in the compartment. 
     
     
         8 . The refrigerator of  claim 2 , wherein the application is further configured to periodically capture the infrared image of the interior space of the compartment at a predefined interval. 
     
     
         9 . The refrigerator of  claim 1 , wherein:
 the infrared image comprises a first infrared image of the interior space of the compartment;   the infrared camera comprises a first infrared camera configured to capture the first infrared image of the interior space of the compartment; and   the refrigerator further comprises a second infrared camera positioned to capture a second infrared image of the interior space of the compartment from a different angle than the first infrared image of the interior space of the compartment.   
     
     
         10 . A system, comprising:
 a computing device that comprises a processor and memory;   a refrigerator that comprises:
 a compartment having an interior space configured to contain a food item; 
 an infrared camera mounted within the refrigerator and directed toward the interior space of the compartment; and 
   an application stored in the memory that, when executed by the processor, causes the refrigerator to at least:
 capture an infrared image of the food item in the interior space of the compartment; 
 perform an image analysis of the infrared image to identify at least one temperature difference between a plurality of locations on the food item; and 
 determine that the food item is spoiled based at least in part on the at least one temperature difference between the plurality of locations on the food item. 
   
     
     
         11 . The system of  claim 10 , wherein the application, when executed by the processor, further causes the refrigerator to at least:
 determine a presence of the food item in the compartment based least in part on a sensor associated with the compartment.   
     
     
         12 . The system of  claim 10 , wherein determining that the food item is spoiled is further based at least in part on comparing the at least one temperature difference of the food item to an image model associated with a food type for the food item. 
     
     
         13 . The system of  claim 12 , wherein the image model is a three-dimensional model representative of spoilage of the food type for the food item. 
     
     
         14 . The system of  claim 10 , further comprising a visible spectrum camera, and wherein the application further causes the refrigerator to at least:
 determine that a door of the refrigerator is open; and   capture a visible spectrum image of the interior space of the compartment using the visible spectrum camera in response to the determination that the door of the refrigerator is open.   
     
     
         15 . The system of  claim 10 , wherein determining that the food item is spoiled based at least in part on a spoilage area of the food item meeting at least a spoilage area threshold for the food item. 
     
     
         16 . A computer-implemented method, comprising:
 determining, via a client device associated with a refrigerator, a presence of a food item in an interior space of a compartment of a refrigerator;   causing, via the client device, an infrared camera to capture an infrared image of the interior space of the compartment, the infrared camera being mounted within the refrigerator and directed toward the interior space of the compartment;   transmitting, via a network interface in communication with the client device, the infrared image of the interior space of the compartment to a remote computing device; and   receiving, via the client device, an indication that the food item is spoiled from the remote computing device.   
     
     
         17 . The method of  claim 16 , wherein the presence of the food item is determined based at least in part on a sensor reading captured in the compartment. 
     
     
         18 . The method of  claim 16 , wherein the presence of the food item is determined based at least in part on an update message generated in response to user input received from a display associated the refrigerator. 
     
     
         19 . The method of  claim 16 , further comprising rendering, via the client device, an alert on a display associated with the refrigerator, the alert indicating that the food item in the compartment is spoiled. 
     
     
         20 . The method of  claim 16 , wherein the refrigerator further comprises a visible spectrum camera, and the method further comprises:
 determining, via the client device, that a door of the refrigerator is open.   causing, via the client device, the visible spectrum camera to capture a visible spectrum image of the interior space of the compartment in response to a determination that the door of the refrigerator is open; and   sending, via the client device, the visible spectrum image of the interior space of the compartment to the remote computing device via the network interface.