Wearable food nutrition feedback system

Abstract

A see-through, head mounted display and sensing devices cooperating to provide feedback on food items detected in the device field of view. Feedback can include warnings based on personal wearer needs, general nutrition information, food consumption tracking and social interactions. The system includes one or more processing devices in communication with display and the sensors which identify food items proximate to the apparatus, determine feedback information relevant to a wearer of the apparatus; and render feedback information in the display.

1 .- 20 . (canceled) 
     
     
         21 . An augmented display apparatus, comprising:
 a see-through, head mounted display;   a plurality of sensors detecting visual information in a field of view of the apparatus;   one or more processing devices in communication with the see-through, head mounted display and the sensors, the one or more processing devices automatically:
 identify food items proximate to the apparatus; 
 match the food items identified with food item data stored in storage; 
 determine feedback information for one or more of the food items identified and relevant to a wearer of the apparatus before consumption of one or more of the food items, the feedback information includes providing a warning about the food items based on profile data of the wearer; and 
 render the feedback information as a virtual object, prior to the consumption, in the see-through, head mounted display. 
   
     
     
         22 . The apparatus of  claim 21 , wherein the one or more processing devices determine nutritional information for the food items, and the feedback information includes the warning based on the nutritional information. 
     
     
         23 . The apparatus of  claim 21 , wherein the one or more processing devices access image data to match sensor data to food types based on the image data and wherein the one or more processing devices nutritional information is based on recipe data of the food items. 
     
     
         24 . The apparatus of  claim 23 , wherein the one or more processing devices access the profile data of the wearer, and the one or more processing devices identify the food items based on wearer consumption history. 
     
     
         25 . The apparatus of  claim 21 , wherein the one or more processing devices access recipe data provided by a third party, and wherein the one or more processing devices provide the warning as feedback in the see-through, head mounted display based on recipe information associated with the food items. 
     
     
         26 . The apparatus of  claim 21 , wherein the one or more processing devices access the profile data of the wearer and track consumption of the food items, and wherein data of the consumption is recorded in the profile data of the wearer and incorporated into the feedback information for subsequent consumption. 
     
     
         27 . The apparatus of  claim 21 , wherein the consumption is identified by detecting a wearer gesture or sound of consumption. 
     
     
         28 . The apparatus of  claim 26 , wherein the tracked consumption of the food items is evaluated and incorporated into the feedback information as the warning to the wearer of the amount of consumption. 
     
     
         29 . The apparatus of  claim 21 , wherein the feedback information includes feedback on the food items by one or more friends of the wearer. 
     
     
         30 . A see-through, head mounted display device including a processor and computer readable storage, the storage including computer readable instructions when executed by the processor performing a method comprising:
 receiving input from a plurality of sensors mounted on the display device, the sensors mounted to detect image data in a field of view of the display device   identify food items proximate to the display device;   match the food items identified with food item data stored in storage;   determine feedback information for one or more of the food items identified and relevant to a wearer of the display device before consumption of one or more of the food items, the feedback information includes providing a warning about the food items based on profile data of the wearer; and   render the feedback information as a virtual object, prior to the consumption, in the display device.   
     
     
         31 . The see-through, head mounted display device of  claim 30 , wherein the processor further executes the instructions to determine nutritional information for the food items, and the feedback information includes the warning based on the nutritional information. 
     
     
         32 . The see-through, head mounted display device of  claim 30 , wherein the processor further executes the instructions to access the profile data of the wearer and track consumption of the food items, and wherein data of the consumption is recorded in the profile data of the wearer and incorporated into the feedback information for subsequent consumption. 
     
     
         33 . The see-through, head mounted display device of  claim 32 , wherein the tracked consumption of the food items is evaluated and incorporated into the feedback information as the warning to the wearer of the amount of consumption. 
     
     
         34 . The see-through, head mounted display device of  claim 30 , wherein the processor further executes the instructions to access the profile data of the wearer, and the one or more processing devices identify the food items based on wearer consumption history. 
     
     
         35 . A method for providing a warning for food items in a field of view of a see-through, head mounted display system, comprising:
 determining a current location, orientation and the field of view for the display system;   identifying food items proximate to the display system;   matching the food items identified with food item data stored in storage;   determining feedback information for one or more of the food items identified and relevant to a wearer of the display system before consumption of one or more of the food items, the feedback information includes providing the warning about the food items based on profile data of the wearer; and   rendering the feedback information as a virtual object, prior to the consumption, in the display device.   
     
     
         36 . The method of  claim 35 , further comprising determining nutritional information for the food items, and the feedback information includes the warning based on the nutritional information. 
     
     
         37 . The method of  claim 35 , further comprising accessing the profile data of the wearer and track consumption of the food items, and wherein data of the consumption is recorded in the profile data of the wearer and incorporated into the feedback information for subsequent consumption. 
     
     
         38 . The see method of  claim 36 , wherein the tracked consumption of the food items is evaluated and incorporated into the feedback information as the warning to the wearer of the amount of consumption. 
     
     
         39 . The method of  claim 35 , further comprising accessing the profile data of the wearer, and the one or more processing devices identify the food items based on wearer consumption history. 
     
     
         40 . The method of  claim 35 , wherein the feedback information includes feedback on the food items by one or more friends of the wearer.