Energy Expense Determination From Spatiotemporal Data

Abstract

A mechanism for interpreting spatiotemporal data from augmented and virtual reality devices into fitness related metrics and recommendations. Spatiotemporal data can be compared to the physical data of the end user to interpret energy expense such as estimated calorie burn. A virtual reality or augmented reality device can report positional data over time as the user engages with a corresponding environment. The positional data is compared to a biometric profile of the end user to understand how their physical body is exerting itself. The spatiotemporal data is a series of coordinate data expressed over time. The biometric data is the physical profile of the end user. This data is used to understand and quantify physical movement while using a virtual reality or augmented reality device. The energy expense determination mechanism can be used to provide a virtual coach for activities and workouts, games involving physical movement, and other applications.

What is claimed is: 
     
         1 . A method for determining energy expense, the method comprising:
 receiving spatiotemporal data, the spatiotemporal data being based on tracking position of a plurality of sensors configured to be placed on a user's one or more body parts,   the spatiotemporal data indicating spatial coordinates of a plurality of points associated with a plurality of the user's one or more body parts as obtained from the tracking of the position of the plurality of sensors in three dimensional space, while the user is performing a workout by moving the user's one or more body parts on which the sensors are placed;   determining the energy expense for the user during the workout based on the spatiotemporal data, captured over a predetermined period of time, and a predetermined resting energy expense, the energy expense being adjustable based on user biometric data and nature of activity the user is engaged in;   adjusting the energy expense based on the user biometric data and the nature of the activity; and   outputting the energy expense by the user during the workout.   
     
     
         2 . The method of  claim 1 , wherein the spatiotemporal data is received as data derived from image data of the user performing the workout, and the biometric data includes at least a body mass index (BMI) of the user. 
     
     
         3 . The method of  claim 1 , wherein the energy expense includes calories burned. 
     
     
         4 . The method of  claim 3 , wherein the energy expense and calories burned are displayed as an overlay to activity elements displayed in a virtual reality (VR) or augmented reality (AR) visual space. 
     
     
         5 . The method of  claim 4 , wherein the activity elements are augmented elements displayed within space captured by a camera. 
     
     
         6 . The method of  claim 1 , wherein the user biometric data comprises age, height, weight, and sex of the user. 
     
     
         7 . The method of  claim 1 , wherein determining the energy expense includes:
 selecting one of a plurality of calculation methods; and   calculating the energy expense based on the selected calculation method.   
     
     
         8 . The method of  claim 1 , wherein determining the energy expense includes comparing the energy expense to a range of expected energy expenses to confirm validity of the energy expense. 
     
     
         9 . The method of  claim 1 , wherein determining the energy expense comprises:
 transmitting the received spatiotemporal data to a remote server; and   receiving a calculated energy expense from the remote server.   
     
     
         10 . The method of  claim 1 , wherein an estimated heart rate for the user is determined while performing the workout based on the spatiotemporal data and the user biometric data that does not change during the workout,
 the energy expense for the user during the workout is determined based on the spatiotemporal data and the estimated heart rate, the estimated heart rate and the energy expense being determined without measuring a heart rate of the user, and   the estimated heart rate for the user is output with the energy expense.   
     
     
         11 . A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform operations comprising:
 receiving spatiotemporal data,
 wherein the spatiotemporal data indicates spatial coordinates of a plurality of points associated with a plurality of body parts of a user, and 
 the spatiotemporal data is updated periodically; 
   determining an energy expense for the user during a workout based on the spatiotemporal data, captured over a predetermined period of time, and a predetermined resting energy expense, wherein the energy expense is adjustable based on user biometric data and nature of activity the user is engaged in;   adjusting the energy expense based on the user biometric data and the nature of the activity; and   outputting the energy expense by the user during the workout.   
     
     
         12 . The non-transitory computer readable storage medium of  claim 11 , wherein the spatiotemporal data is received as data derived from image data of the user performing the workout. 
     
     
         13 . The non-transitory computer readable storage medium of  claim 11 , wherein the energy expense includes calories burned. 
     
     
         14 . The non-transitory computer readable storage medium of  claim 13 , wherein the energy expense and the calories burned are displayed as an overlay to activity elements, wherein the activity elements are augmented elements displayed within space captured by a camera. 
     
     
         15 . A system comprising:
 a server including memory and one or more processors performing operations to:
 receive spatiotemporal data from a client application stored and executing on a client device,
 the client application is configured to render activity elements associated with a workout of a user in a virtual reality (VR) visual space or an augmented reality (AR) visual space, and 
 the spatiotemporal data indicates spatial coordinates of a plurality of points associated with a body of the user while the user is performing the workout; 
 
 determine an energy expense for the user during the workout based on the spatiotemporal data; and 
 transmit the energy expense for the user during the workout to the client device for display by the client application, the client application is configured to render energy expense information with the activity elements within the virtual reality (VR) visual space or the augmented reality (AR) visual space. 
   
     
     
         16 . The system of  claim 15 , wherein the spatiotemporal data is data derived from image data of the user performing the workout. 
     
     
         17 . The system of  claim 15 , wherein the energy expense includes calories burned. 
     
     
         18 . The system of  claim 17 , wherein the energy expense and the calories burned are displayed as an overlay to the activity elements displayed in the virtual reality (VR) visual space or the augmented reality (AR) visual space. 
     
     
         19 . The system of  claim 15 , wherein the activity elements are augmented elements displayed within space captured by a camera. 
     
     
         20 . The system of  claim 15 , wherein determining the energy expense for the user during the workout is further based user biometric data including age, height, weight, and sex of the user.