System and method for automated personalized and community-specific eating and activity planning, linked to tracking system with automated multimodal item identification and size estimation system

Abstract

The system and method for automated personalized and community-specific eating and activity planning, linked to tracking with automated multimodal item identification and size estimation, enables and integrates health and other user datastreams, enables rewards and links to healthy eating and activity partners based on that data—both external and internal. The system and method also provide personalized wellness recommendations for eating, physical activity, sleep, stress reduction and other elements of daily living, tailored to each user based on the preferences, prior history, location and budget information provided by the users. Data inputs for elements such as food tracking are made simpler and more accurate through multimodal recognition combining database subsetting based on geolocation user check-ins based on global positioning system recommendations (such as checking into a restaurant and subsetting to a menu), voice recordings interpreted to text by existing voice recognition algorithms, descriptive text entered by users tracking the food or other item of interest, or other humans through services such as mechanical turk, together with any of a number of available image visual recognition tools using pixel level color and texture (pixel comparison) analysis plus instance based and classification and regression tree algorithms. The system and method also enables action, such as single click ordering of the healthy meals or shopping list on one's plan from local restaurants and grocery stores, and receipt of mobile vouchers and coupons with a unique validation system for use at retailers. Once foods are bought, scanning of unique barcodes and multimodal item recognition from FitNet can also be used for tracking and management of a user's pantry and food at home.

1 . A system for nutritional management, comprising:
 one or more computing devices;   a nutritional management unit that is capable of being connected to and interacting with each of the one or more computing devices over a link; and   the nutritional management unit further comprising a nutritional planning unit that uses multimodal recognition to determine one of a food, an exercise and other items from images and voice recordings and subsetting to menus upon checking into a specific food venue, and a recommendation unit that recommends, based on user data, one of a meal and an activity to the user to balance the caloric and nutritional intake, and caloric output, physical activity methods and sleep duration of the user.   
     
     
         2 . The system of  claim 1 , wherein the nutritional planning unit further comprises a user interface that allows the user to track entries of nutritional planning. 
     
     
         3 . The system of  claim 2 , wherein the entries are one of a meal, an exercise, a sleep time, a mood and a custom item. 
     
     
         4 . The system of  claim 1 , wherein the multimodal recognition is one of a geolocation at a time proximate to an entry of the food item, a visual recognition of the food item and a barcode of the package of the food item. 
     
     
         5 . The system of  claim 1 , wherein each computing device is one of a smartphone mobile device, a laptop computer, a tablet computer, a body scale, an accelerometer or GPS-based physical activity tracking device, and a sleep tracking device. 
     
     
         6 . The system of  claim 1 , wherein the user data further comprises one or more of stored foods, stored activities, favorite and least favorite foods and activities, restrictions, budget information, transportation preferences, home location, age, gender and wherein the recommendation unit recommends one of a personalized meal and a personalized activity to the user based on the user data. 
     
     
         7 . The system of  claim 2 , wherein user interface generates a lifemap that displays one or more variables over a period of time for the user based on the user data and wherein the one or more variables are one of calories, breakdown of carbohydrate, fat, protein or other nutrients, exercise caloric output, duration, intensity and type, sleep duration and quality, mood score, body weight or fat percentage, and other symptoms, performance outcomes or disease outcomes. 
     
     
         8 . The system of  claim 1 , wherein the nutritional planning unit generates a personalized plan and guide for the user based, in part, on the recommendations. 
     
     
         9 . The system of  claim 1 , wherein the nutritional planning unit generates a printable voucher or coupon that is redeemable for one of a discounted purchase and a free purchase through any point of sale system by a user of the system. 
     
     
         10 . The system of  claim 1 , wherein the nutritional planning unit generates a mobile voucher or coupon that is redeemable for one of a discounted purchase and a free purchase through a point of sale system by a user of the system. 
     
     
         11 . The system of  claim 1 , wherein the nutritional planning unit predicts a risk of future diseases and causes of symptoms. 
     
     
         12 . A method for nutritional management, physical activity management, sleep management, weight management and performance management using one or more computing devices and a nutritional management unit that is capable of being connected to and interacting with each of the one or more computing devices over a link, the method comprising:
 recognizing, by a nutritional planning unit of the nutritional management unit, one of a food, an exercise and other items using multimodal recognition and subsetting to menus upon checking into a specific food venue; and   recommending, by a recommendation unit of the nutritional management unit, one of a meal and an activity to the user to balance the caloric and nutritional intake, and caloric output, physical activity methods and sleep duration of the user.   
     
     
         13 . The method of  claim 12  further comprising generating, by the nutritional planning unit, a user interface that allows the user to track entries of nutritional planning. 
     
     
         14 . The method of  claim 13 , wherein the entries are one of a meal, an exercise, a sleep time, a mood and a custom item. 
     
     
         15 . The method of  claim 12 , wherein the multimodal recognition is one of a geolocation at a time proximate to an entry of the food item, a visual recognition of the food item and a barcode of the package of the food item. 
     
     
         16 . The method of  claim 12 , wherein the user data further comprises one or more of stored foods, stored activities, favorite and least favorite foods and activities, restrictions, budget information, transportation preferences, home location, age, gender and wherein recommending one of a meal and an activity to the user further comprises recommending one of a personalized meal and a personalized activity to the user based on the user data. 
     
     
         17 . The method of  claim 12  further comprising generating a lifemap that displays one or more nutritional variables over a period of time for the user based on the user data and wherein the one or more nutritional variables are one of calories, exercise, sleep and mood. 
     
     
         18 . The method of  claim 12  further comprising generating, by the nutritional planning unit, a personalized plan guide for the user based, in part, on the recommendations. 
     
     
         19 . The method of  claim 12  further comprising generating one of a voucher and a mobile coupon that is redeemable by a user of the system. 
     
     
         20 . The method of  claim 12  further comprising predicting, by the nutritional planning unit, a risk of future diseases and causes of symptoms based on the user data