Systems and Methods for Training and/or Using Representation Learning Neural Networks for Electromyographic Data

Abstract

Systems and methods that use data augmentation during the training of representation learning networks on neuromuscular data (e.g., EMG) with reconstruction cost. The method may include training network(s) on neuromuscular data received from channel(s) of neuromuscular sensor(s). The training may include randomly generating a first augment and a different, second augment for each channel. The training may include augmenting the data of each channel by applying the first and second augments to the data to generate first and second augmented data. The training may include processing at least the first augmented data through at least a first representation learning neural network to determine a first latent representation of one or more neuromuscular activation state variables. The training may include determining a reconstruction cost using the first latent representation and the second augmented data. The training may include updating the first representation learning neural network based on the reconstruction cost.

1 . A computer-implemented method, comprising:
 receiving, via one or more channels of one or more neuromuscular sensors, neuromuscular data for a preset window from each channel; and   training at least one or more networks using the first augmented data and the second augmented data, the one or more networks including one or more representation learning neural networks, wherein training the one or more representation learning neural networks includes:
 randomly generating a first augment and a second augment different from the first augment for each channel; 
 augmenting the neuromuscular data of each channel by applying the first augment and the second augment to the neuromuscular data of respective channel to generate a first augmented data and a second augmented data for the respective channel; 
 processing at least the first augmented data for each channel through at least a first representation learning neural network to determine a first latent representation of one or more neuromuscular activation state variables for the respective channel; 
   determining a reconstruction cost using the first latent representation and the second augmented data for each channel; and   updating the first representation learning neural network based on the reconstruction cost.   
     
     
         2 . The method according to  claim 1 , wherein each augment corresponds to a temporal shift, each temporal shift having an integer value. 
     
     
         3 . The method according to  claim 2 , wherein:
 the augmenting the neuromuscular data for each channel includes moving the neuromuscular data forward or backward temporally within the preset window based on the integer value of the respective temporal shift.   
     
     
         4 . The method according to  claim 3 , wherein the one or more networks includes a second representation neural learning network, the method further comprising:
 processing the second augmented data for each channel through a second representation learning neural network to determine a second latent representation of one or more neuromuscular activation state variables for the preset window of time;   wherein the reconstruction cost is determined using the first latent representation and the second latent representation; and   updating the first representation learning neural network and/or the second representation learning neural network based on the reconstruction cost.   
     
     
         5 . The method according to  claim 4 , wherein the one or more networks includes a projector network, the method further comprising:
 processing the first latent representation of one or more neuromuscular activation state variables for the preset window of time through a projector network to generate a first projected latent representation;   wherein the reconstruction cost is determined between the first projected latent representation and the second latent representation; and   updating the first representation learning neural network and/or the second representation learning neural network based on the reconstruction cost.   
     
     
         6 . The method according to  claim 1 , wherein the one or more representation learning neural networks includes one or more of autoencoder neural networks, one or more of transformer neural networks and/or one or more of linear transformation networks. 
     
     
         7 . The method according to  claim 1 , wherein each augment includes a temporal value, a magnitude value, a randomly generated temporal order, and/or a randomly generated binary mask. 
     
     
         8 . A system, comprising:
 one or more processors;   one or more neuromuscular sensors coupled to the processor, each neuromuscular sensor including one or more channels; and   a memory having stored thereon computer-executable instructions which are executable by the one or more processors to cause the computing system to perform at least the following:   receiving, via one or more channels of one or more neuromuscular sensors, neuromuscular data for a preset window from each channel; and   training at least one or more networks using the first augmented data and the second augmented data, the one or more networks including one or more representation learning neural networks, wherein the training the one or more representation learning neural networks includes:
 randomly generating a first augment and a second augment different from the first augment for each channel; 
 augmenting the neuromuscular data of each channel by applying the first augment and the second augment to the neuromuscular data of respective channel to generate a first augmented data and a second augmented data for the respective channel; 
 processing at least the first augmented data for each channel through at least a first representation learning neural network to determine a first latent representation of one or more neuromuscular activation state variables for the respective channel; 
   determining a reconstruction cost using the first latent representation and the second augmented data for each channel; and   updating the first representation learning neural network based on the reconstruction cost.   
     
     
         9 . The system according to  claim 8 , wherein each augment corresponds to a temporal shift, each temporal shift having an integer value. 
     
     
         10 . The system according to  claim 9 , wherein:
 the augmenting the neuromuscular data for each channel includes moving the neuromuscular data forward or backward temporally within the preset window based on the integer value of the respective temporal shift.   
     
     
         11 . The system according to  claim 10 , wherein:
 the one or more networks includes a first representation learning neural network; and the one or more processors are further configured to cause the computing system to perform at least the following:   processing the second augmented data for each channel through a second representation learning neural network to determine a second latent representation of one or more neuromuscular activation state variables for the preset window of time;   wherein the reconstruction cost is determined using the first latent representation and the second latent representation; and   updating the first representation learning neural network and/or the second representation learning neural network based on the reconstruction cost.   
     
     
         12 . The system according to  claim 11 , wherein:
 the one or more networks includes a projector network; and the one or more processors are further configured to cause the computing system to perform at least the following:   processing the first latent representation of one or more neuromuscular activation state variables for the preset window of time through a projector network to generate a first projected latent representation;   wherein the reconstruction cost is determined between the first projected latent representation and the second latent representation; and   updating the first representation learning neural network and/or the second representation learning neural network based on the reconstruction cost.   
     
     
         13 . The system according to  claim 1 , wherein the one or more representation learning neural networks includes one or more of autoencoder neural networks, one or more of transformer neural networks and/or one or more of linear transformation networks. 
     
     
         14 . The system according to  claim 1 , wherein each augment includes a temporal value, a magnitude value, a randomly generated temporal order, and/or a randomly generated binary mask. 
     
     
         15 . A non-transitory computer-readable storage medium comprising one or more computer-executable instructions, that when executed by one or more processors of a computing device, cause the computing device to perform at least the following:
 receiving, via one or more channels of one or more neuromuscular sensors, neuromuscular data for a preset window from each channel; and   training at least one or more networks using the first augmented data and the second augmented data, the one or more networks including one or more representation learning neural networks, wherein the training the one or more representation learning neural networks includes:
 randomly generating a first augment and a second augment different from the first augment for each channel; 
 augmenting the neuromuscular data of each channel by applying the first augment and the second augment to the neuromuscular data of respective channel to generate a first augmented data and a second augmented data for the respective channel; 
 processing at least the first augmented data for each channel through at least a first representation learning neural network to determine a first latent representation of one or more neuromuscular activation state variables for the respective channel; 
   determining a reconstruction cost using the first latent representation and the second augmented data for each channel; and
 updating the first representation learning neural network based on the reconstruction cost. 
   
     
     
         16 . The medium according to  claim 15 , wherein each augment corresponds to a temporal shift, each temporal shift having an integer value. 
     
     
         17 . The medium according to  claim 16 , wherein:
 the augmenting the neuromuscular data for each channel includes moving the neuromuscular data forward or backward temporally within the preset window based on the integer value of the respective temporal shift.   
     
     
         18 . The medium according to  claim 17 , wherein:
 the one or more networks includes a first representation learning neural network; and the one or more processors are further configured to cause the computing system to perform at least the following:   processing the second augmented data for each channel through a second representation learning neural network to determine a second latent representation of one or more neuromuscular activation state variables for the preset window of time;   wherein the reconstruction cost is determined using the first latent representation and the second latent representation; and   updating the first representation learning neural network and/or the second representation learning neural network based on the reconstruction cost.   
     
     
         19 . The medium according to  claim 18 , wherein:
 the one or more networks includes a projector network; and the one or more processors are further configured to cause the computing system to perform at least the following:   processing the first latent representation of one or more neuromuscular activation state variables for the preset window of time through a projector network to generate a first projected latent representation;   wherein the reconstruction cost is determined between the first projected latent representation and the second latent representation; and   updating the first representation learning neural network and/or the second representation learning neural network based on the reconstruction cost.   
     
     
         20 . The medium according to  claim 15 , wherein the one or more representation learning neural networks includes one or more of autoencoder neural networks, one or more of transformer neural networks and/or one or more of linear transformation networks.