Motion data driven performance evaluation and training

Abstract

Method and apparatus for evaluating user movement in a fulfillment center. A plurality of sub-tasks for performing a fulfillment operation are determined. Embodiments retrieve a training set of motion data that includes a plurality of training motion data samples, each specifying motion data over a respective window of time, and train a first data model to classify instances of motion data into portions, based on the plurality of sub-tasks. At least one data model is trained to assess performance of the fulfillment operation, using the training set of motion data. An instance of motion data describing motion performed during the fulfillment operation is received, the instance of motion data is divided into a plurality of portions, using the first data model, and a measure of quality is generated, by analyzing the plurality of portions of the instance of motion data using the trained at least one data model.

What is claimed is: 
     
       1. A method, comprising:
 determining a plurality of sub-tasks for carrying out a fulfillment operation; 
 retrieving a training set of motion data that includes a plurality of training motion data samples, each specifying motion data over a respective window of time; 
 training a first data model to classify instances of motion data into temporal chunks, based on the plurality of sub-tasks; 
 training at least one data model to assess performance of the fulfillment operation, using the training set of motion data; 
 determining an instance of motion data describing motion performed in carrying out the fulfillment operation, comprising:
 capturing, via one or more camera devices, images of a user performing the fulfillment operation; 
 tracking, based on the captured images, movement of one or more markers on a motion capture device worn by the user; and 
 generating the instance of motion data based on the movement of the one or more markers; 
 
 dividing the instance of motion data into a plurality of temporal chunks, using the first data model; 
 generating a measure of quality for the instance of motion data, by analyzing the plurality of temporal chunks of the instance of motion data using the trained at least one data model, wherein generating the measure of quality comprises generating a quality score for at least one first sub-task of the plurality of sub-tasks; and 
 upon determining that the quality score for the first sub-task satisfies a predetermined condition, rendering, to the user via an immersive reality device, during performance of the fulfillment operation, a plurality of frames depicting motion of the performance of the first sub-task based on one of the plurality of temporal chunks of the instance of motion data corresponding to the first sub-task of the plurality of sub-tasks. 
 
     
     
       2. The method of  claim 1 , wherein the training set of motion data comprises a plurality of positive motion data samples and a plurality of negative motion data samples. 
     
     
       3. The method of  claim 1 , wherein training the at least one data model to assess performance of the fulfillment operation, using the training set of motion data, comprises training a plurality of data models to assess the performance of the fulfillment operation, wherein each of the plurality of data models corresponds to a respective one of the plurality of sub-tasks. 
     
     
       4. The method of  claim 1 , wherein the quality score for the at least one first sub-task of the plurality of sub-tasks comprises a plurality of scores, each corresponding to a different performance metric for the first sub-task. 
     
     
       5. The method of  claim 1 , wherein training a first data model to classify instances of motion data into temporal chunks, based on the plurality of sub-tasks, further comprises:
 receiving, for each of the plurality of training motion data samples, a respective time window within the training motion data samples that corresponds to each of the plurality of sub-tasks, 
 wherein the first data model is trained to classify instances of motion data into temporal chunks, based on the time windows for each of the plurality of training motion data samples and each of the plurality of sub-tasks. 
 
     
     
       6. The method of  claim 1 , wherein the fulfillment operation comprises at least one of a pick operation and a stow operation. 
     
     
       7. The method of  claim 6 , wherein the plurality of sub-tasks further comprise at least logging into a workstation, selecting a workflow, waiting on a pod to arrive, scanning a container, picking a product out of the container, identifying a target bin for the product, placing the product in the target bin, and scanning the target bin. 
     
     
       8. The method of  claim 1 , wherein the predetermined condition comprises the quality score being less than a predefined threshold score associated with the first sub-task. 
     
     
       9. A method, comprising:
 receiving an instance of motion data describing motion performed by an operator in carrying out a fulfillment operation, the motion data having been captured by a motion capture system tracking, via one or more first motion capture devices, movement of a second motion capture device worn by the operator during performance of the fulfillment operation; 
 determining, for at least one portion of the instance of motion data, which sub-task of a plurality of sub-tasks of the fulfilment operation corresponds to the at least one portion of the instance of motion data, based on a first data model; 
 analyzing the instance of motion data using at least one data model trained with historically collected motion data; 
 determining a measure of quality for the instance of motion data, based on the analysis, wherein determining the measure of quality comprises determining a quality score for the sub-task corresponding to the at least one portion of the instance of motion data; and 
 upon determining that the quality score for the sub-task satisfies a predetermined condition, rendering, to the operator via an immersive reality device, during the performance of the fulfillment operation, a plurality of frames depicting motion of the performance of the sub-task based on the at least one portion of the instance of motion data. 
 
     
     
       10. The method of  claim 9 , further comprising:
 retrieving a training set of motion data that includes a plurality of training motion data samples, each specifying motion data over a respective window of time; and 
 training the first data model to classify instances of motion data into portions, based on the plurality of sub-tasks. 
 
     
     
       11. The method of  claim 10 , further comprising:
 training a plurality of data models to assess performance of the fulfillment operation, using the training set of motion data. 
 
     
     
       12. The method of  claim 11 , wherein analyzing the instance of motion data using the at least one data model trained with historically collected motion data further comprises:
 assessing performance of each portion of the instance of motion data, using a respective one of the plurality of data models; and 
 generating an overall quality assessment of the performance of the fulfillment operation described by the instance of motion data, based on the performance assessment of the portions of the instance of motion data. 
 
     
     
       13. The method of  claim 9 , wherein:
 the one or more first motion capture devices comprise camera devices; and 
 tracking the movement of the second motion capture device comprises capturing, via the camera devices, images of one or more markers on the second motion capture device during performance of the fulfillment operation. 
 
     
     
       14. The method of  claim 9 , wherein tracking the movement of the second motion capture device comprises:
 detecting, via the first motion capture device, one or more signals emitted by the second motion capture device during performance of the fulfillment operation; and 
 determining, for each of the one or more signals, a respective set of location coordinates corresponding to a different position of the second motion capture device over a period of time during the performance of the fulfillment operation.