Augmented Video Processing

Abstract

Embodiments described herein may allow for dynamic image processing based on biometric data. An example device may include: an interface configured to receive video data that is generated by an image capture device; an interface configured to receive biometric data of a user of the image capture device from one or more sensors generated synchronously with the video data; and an image processing system configured to apply image processing to the video data to generate edited video data. The image processing may be based, at least in part, on the biometric data.

We claim: 
     
         1 . A computing device, comprising:
 an interface configured to receive video data that is generated by an image capture device;   an interface configured to receive, from one or more sensors, biometric data of a user of the device generated during capture of the received video data, wherein the biometric data corresponds in time with the video data; and   a processing system operable to:
 determine that a measurable characteristic of a first portion of the biometric data differs by a threshold amount from a corresponding measurable characteristic of a second portion of the biometric data; 
 automatically apply image processing to the video data to generate edited video data, wherein the image processing comprises, in response to a determination that the measurable characteristic of the first portion differs by the threshold amount from the second portion, applying a first color parameter adjustment to the first portion of video data, but not to the second portion of the video data; and 
 generate, and store to data storage for subsequent playback, a video file comprising the edited video data. 
   
     
     
         2 . The computing device of  claim 1 , wherein applying the first color parameter adjustment to the first portion of video data, but not to the second portion of the video data, comprises:
 adjusting one or more of the following color parameters for the first portion of the video data: (a) a saturation parameter, (b) a hue parameter color calibration, (c) an intensity parameter, (d) a white balance parameter, or (e) a color-space conversion parameter.   
     
     
         3 . The computing device of  claim 1 , wherein the biometric data comprise heartrate data indicating a higher heartrate at a time corresponding to the first portion of the video data than at a time corresponding to the second portion of the video data, and wherein the application of the first color parameter adjustment to the first portion of video data, but not to the second portion of the video data, comprises:
 adjusting a first color parameter for the first portion of video data so as to visually emphasize the first portion of the video data as compared to the second portion of the video data.   
     
     
         4 . The computing device of  claim 3 , wherein the adjustment to the first color parameter for the first portion of video data comprises an adjustment to hue, saturation, or both towards a particular color. 
     
     
         5 . The computing device of  claim 1 , further comprising:
 an interface configured to receive time-synchronization data that indicates a timing relationship between the biometric data and the video data, wherein the image processing of the video data is further based on the time-synchronization data.   
     
     
         6 . The computing device of  claim 1 , wherein the computing device is implemented as part of or takes the form of a head-mountable device (HMD). 
     
     
         7 . The computing device of  claim 1 , wherein the computing device is implemented as or takes the form of a cloud server. 
     
     
         8 . The computing device of  claim 1 , wherein the biometric data comprises one or more of the following types of data: (a) heart rate data, (b) respiration rate data, (c) body temperature data, (d) level of perspiration data, (e) muscle movement data, (f) eye movement data, or (h) speech data. 
     
     
         9 . The computing device of  claim 5 , wherein the one or more sensors comprise one or more of the following types of sensors: (a) a heart rate monitor, (b) a respiration rate sensor, (c) a thermometer, (e) a perspiration sensor, (f) a microphone, (g) a decibel meter, (h) a bone conducting transducer, (i) an electromyograph, (j) a strain gauge, or (k) an eye-detection sensor. 
     
     
         10 . A computing device, comprising:
 an interface configured to receive video data that is generated by an image capture device;   an interface configured to receive, from one or more sensors, biometric data of a user of the device generated during capture of the received video data, wherein the biometric data corresponds in time with the video data; and   a processing system operable to:
 determine that a measurable characteristic of a first portion of the biometric data differs by a threshold amount from a corresponding measurable characteristic of a second portion of the biometric data; 
 automatically apply image processing to the video data to generate edited video data, wherein the image processing comprises, in response to a determination that the measurable characteristic of the first portion differs by the threshold amount from the second portion, applying at least one visual effect to the first portion of video data, but not to the second portion of the video data; and 
 generate, and store to data storage for subsequent playback, a video file comprising the edited video data. 
   
     
     
         11 . The computing device of  claim 10 , wherein the at least one visual effect comprises one or more of the following visual effects: (a) a fade-in effect, (b) a fade-out effect, (c) noise reduction, (d) detail enhancement or sharpening, (e) edge enhancement, or (f) motion compensation. 
     
     
         12 . The computing device of  claim 10 , further comprising one or more of the following types of sensors: (a) an accelerometer, (b) an inertial measurement unit, (c) a proximity sensor; (d) a microphone; (e) a gyroscope, (f) a magnetometer, (g) an optical sensor, (h) an ultrasonic sensor, (i) an odometer, and (j) a pedometer. 
     
     
         13 . The computing device of  claim 10 , wherein the image processing system is further configured to:
 apply image processing based, at least in part, on additional data comprising one or more of the following types of data: (a) location, (b) ambient light intensity, (c) time of day, (d) a user's mode of travel, (e) a type of activity a user is participating in, (f) height of a user, (g) weight of a user, and (h) age of a user.   
     
     
         14 . The computing device of  claim 10 , wherein the one or more sensors comprise one or more contextual sensors from the following contextual sensors: (a) a location-tracking sensor, (b) light intensity sensor, (c) a clock, or (d) a sensor configured to receive an input from a user. 
     
     
         15 . The computing device of  claim 10 , wherein the image processing comprises at least one of: (a) fast motion effects, (b) slow motion effects, (c) blur effects, (d) color calibration, (e) light intensity calibration, (f) fade-in or fade-out effects, (g) frame cutting, (h) frame rate conversion, (i) white point conversion, (j) color space conversion, (k) noise reduction, (l) detail enhancement, (m) edge enhancement, or (n) motion compensation. 
     
     
         16 . The computing device of  claim 10 , wherein the image processing system is configured to decrease a speed of the video data to generate edited video data based, at least in part, on receiving biometric data indicative of a higher than normal respiration rate. 
     
     
         17 . The computing device of  claim 10 , wherein the image processing system is configured to adjust a color saturation of the video data to generate edited video data based, at least in part, on receiving biometric data indicative of speed of travel. 
     
     
         18 . The computing device of  claim 10 , wherein the image processing comprises one or more audio effects including: (a) audio overlay, (b) noise removal, (c) noise amplification, (d) noise suppression, (e) audio volume adjustment, (f) pitch shift, and (g) time stretching.