Neural network system for gesture, wear, activity, or carry detection on a wearable or mobile device

Abstract

A neural network system includes an eyewear device. The eyewear device has a movement tracker, such as an accelerometer, gyroscope, or an inertial measurement unit for measuring acceleration and rotation. The neural network system tracks, via the movement tracker, movement of the eyewear device from at least one finger contact inputted from a user on an input surface. The neural network system identifies a finger gesture by detecting at least one detected touch event based on variation of the tracked movement of the eyewear device over a time period. The neural network system adjusts the image presented on the image display of the eyewear device based on the identified finger gesture. The neural network system can also detect whether the user is wearing the eyewear device and identify an activity of the user wearing the eyewear device based on the variation of the tracked movement over the time period.

What is claimed is: 
     
         1 . A method for generating a neural network model for a wearable device, the method comprising:
 generating a trained model using training data acquired from the wearable device;   extracting a calibrated set of weights from the trained model;   generating a trained inferencing code model based on the trained model;   compiling the trained inferencing code model into a trained machine inferencing code model;   quantizing the calibrated set of weights; and   loading the trained inferencing machine code model and the quantized calibrated set of weights onto the wearable device.   
     
     
         2 . The method of  claim 1 , wherein the trained model is a trained scripting language model and wherein generating the trained model comprises:
 generating the trained scripting language model in an interpreted programming language based on the acquired training data.   
     
     
         3 . The method of  claim 2 , wherein the inferencing comprises:
 generating C language code based on the trained scripting language model.   
     
     
         4 . The method of  claim 2 , wherein the acquired training data includes at least one of:
 tracked movement of the wearable device over time intervals occurring while the wearable device is being carried and while the wearable device is not being carried; or   tracked movement of the wearable device over time intervals occurring while the wearable device is being worn and while the wearable device is not being worn; or.   
     
     
         5 . The method of  claim 2 , wherein the acquired training data includes tracked movement of the wearable device over time intervals in response to known and unclassified gestures or activities occurring while the wearable device is being worn and while the wearable device is not being worn. 
     
     
         6 . The method of  claim 2 , wherein the wearable device is an eyewear device having a left side and a right side and wherein the generating the trained model comprises:
 creating labels for each element of trained data, the labels comprising left corresponding the left side of the eyewear device and right corresponding the right side of the eyewear device.   
     
     
         7 . The method of  claim 1 , wherein the calibrated set of weights are extracted in a floating point format and wherein the quantizing comprises:
 converting the calibrated set of weights from the floating point format to a fixed number of bits format.   
     
     
         8 . The method of  claim 2 , wherein the wearable device comprises statically allocated memory and wherein the loading comprises:
 loading the trained inferencing machine code model and the quantized calibrated set of weights into the statically allocated memory of the wearable device.   
     
     
         9 . The method of  claim 2 , wherein the wearable device comprise flash memory and random access memory (RAM) and wherein the loading comprises:
 loading the trained inferencing machine code model and the quantized calibrated set of weights into at least one of the flash memory or RAM of the wearable device.   
     
     
         10 . A wearable device comprising a neural network model, the neural network model generated by:
 generating a trained model using training data acquired from the wearable device;   extracting a calibrated set of weights from the trained model;   generating a trained inferencing code model based on the trained model;   compiling the trained inferencing code model into a trained machine inferencing code model; and   quantizing the calibrated set of weights;   wherein the neural network model comprises the trained inferencing machine code model and the quantized calibrated set of weights.   
     
     
         11 . The wearable device of  claim 10 , wherein the trained model is a trained scripting language model and wherein generating the trained model comprises:
 generating the trained scripting language model in an interpreted programming language based on the acquired training data.   
     
     
         12 . The wearable device of  claim 11 , wherein the inferencing comprises:
 generating C language code based on the trained scripting language model.   
     
     
         13 . The wearable device of  claim 11 , wherein the acquired training data includes at least one of:
 tracked movement of the wearable device over time intervals occurring while the wearable device is being carried and while the wearable device is not being carried; or   tracked movement of the wearable device over time intervals occurring while the wearable device is being worn and while the wearable device is not being worn.   
     
     
         14 . The wearable device of  claim 11 , wherein the acquired training data includes tracked movement of the wearable device over time intervals in response to known and unclassified gestures or activities occurring while the wearable device is being worn and while the wearable device is not being worn. 
     
     
         15 . The wearable device of  claim 11 , wherein the wearable device is an eyewear device having a left side and a right side and wherein the generating the trained model comprises:
 creating labels for each element of trained data, the labels comprising left corresponding the left side of the eyewear device and right corresponding the right side of the eyewear device.   
     
     
         16 . The wearable device of  claim 10 , wherein the calibrated set of weights are extracted in a floating point format and wherein the quantizing comprises:
 converting the calibrated set of weights from the floating point format to a fixed number of bits format.   
     
     
         17 . The wearable device of  claim 11 , further comprising statically allocated memory, wherein the trained inferencing machine code model and the quantized calibrated set of weights are loaded into the statically allocated memory of the wearable device. 
     
     
         18 . The wearable device of  claim 11 , further comprising flash memory and random access memory (RAM), wherein the trained inferencing machine code model and the quantized calibrated set of weights are loaded into at least one of the flash memory or RAM of the wearable device. 
     
     
         19 . A non-transitory computer readable media storing instructions for generating a neural network model for use with a wearable device, the instructions, when executed by a processor perform functions comprising functions to:
 generate a trained model using training data acquired from the wearable device;   extract a calibrated set of weights from the trained model;   generate a trained inferencing code model based on the trained model;   compile the trained inferencing code model into a trained machine inferencing code model; and   quantize the calibrated set of weights;   wherein the neural network model comprises the trained inferencing machine code model and the quantized calibrated set of weights.   
     
     
         20 . The non-transitory computer readable media of  claim 19 , wherein the trained model is a trained scripting language model and wherein to generate the trained model the instructions when executed by the processor perform a function to:
 generate the trained scripting language model in an interpreted programming language based on the acquired training data.