Real-time persistent object tracking for intelligent video analytics systems

Abstract

Apparatuses, systems, and techniques for real-time persistent object tracking for intelligent video analytics systems. A state of a first object included in an environment may be tracked based on a first set of images depicting the environment. The first set of images may be generated during a first time period. It may be determined that the first object is not detected in the environment depicted in a second set of images. The second set of images may be generated during a second time period that is subsequent to the first time period. One or more predicted future states of the first object may be obtained in view of the state of the first object in the environment depicted in the first set of images. A second object may be detected in the environment depicted in a third set of images generated during a third time period that is subsequent to the second time period. A determination may be made as to whether a current state of the second object corresponds to at least one of the one or more predicted future states of the first object. In response to a determination that a current state of the second object corresponds to at least one of the predicted future states of the first object, an identifier associated with the second object is updated to correspond to an identifier associated with the first object.

What is claimed is: 
     
       1. A method comprising:
 tracking, based on a first set of images depicting an environment, a state of a first object included in the environment, wherein the first set of images is generated during a first time period; 
 determining that the first object is not detected in the environment depicted in a second set of images generated during a second time period that is subsequent to the first time period; 
 obtaining one or more predicted future states of the first object in view of the state of the first object in the environment depicted in the first set of images; 
 detecting a second object included in the environment depicted in a third set of images generated during a third time period that is subsequent to the second time period, wherein a number of images of the third set of images exceeds a threshold number of images; 
 determining whether a current state of the second object corresponds to at least one of the one or more predicted future states of the first object; and 
 responsive to determining that a current state of the second object corresponds to at least one of the one or more predicted future states of the first object, updating an identifier associated with the second object to correspond to an identifier associated with the first object. 
 
     
     
       2. The method of  claim 1 , wherein obtaining the one or more predicted future states of the first object comprises:
 obtaining state data associated with the first object based on the state of the first object in the environment depicted in each of the first set of images; 
 calculating a path that the first object is expected to follow in the environment during a future time period based on the obtained state data; and 
 determining the one or more predicted future states of the first object based on the calculated path. 
 
     
     
       3. The method of  claim 2 , wherein obtaining the state data associated with the first object comprises:
 providing an indication of at least one of a prior state or a current state of the first object in the environment depicted in each of the first set of images as an input to one or more state prediction functions; and 
 determining the state data associated with the first object based on an output of the one or more state prediction functions. 
 
     
     
       4. The method of  claim 2 , wherein obtaining the state data associated with the first object comprises:
 providing an indication of at least one of a prior state or a current state of the first object in the environment depicted in each of the first set of images as an input to a machine learning model; 
 obtaining one or more outputs of the machine learning model; 
 extracting, from the one or more outputs, one or more sets of object state data and, for each set of object state data, an indication of a level of confidence that each set of object state data corresponds to the first object; and 
 identifying a set of object state data associated with a level of confidence that satisfies a confidence criterion. 
 
     
     
       5. The method of  claim 4 , wherein the machine learning model comprises a recurrent neural network. 
     
     
       6. The method of  claim 1 , further comprising:
 updating coordinates indicating a path taken by the first object based on the state of the first object included in the environment depicted in the first set of images and the current state of the second object included in the environment depicted in the third set of images. 
 
     
     
       7. The method of  claim 1 , wherein tracking the state of the first object comprises:
 obtaining the first set of images and a first set of bounding boxes associated with the first set of images, wherein the first set of bounding boxes indicate one or more regions of the first set of images that include a detected presence of the first object; and 
 determining the state of the first object included in the environment depicted in the first set of images based on the first set of bounding boxes. 
 
     
     
       8. The method of  claim 1 , wherein determining that the first object is not included in the environment depicted in the second set of images comprises:
 obtaining the second set of images; and 
 determining that no bounding boxes associated with the second set of images correspond to a region of a respective image of the second set of images that includes a detected presence of the first object in the environment depicted in the respective image. 
 
     
     
       9. The method of  claim 1 , further comprising:
 tracking, based on a fourth set of images depicting the environment, the state of the first object included in the environment, wherein the fourth set of images is generated during a fourth time period that is subsequent to the third time period. 
 
     
     
       10. A system comprising:
 a memory device; and 
 a processing device coupled to the memory device, wherein the processing device is to:
 track, based on a first set of images depicting an environment, a state of a first object included in the environment, wherein the first set of images is generated during a first time period; 
 determine that the first object is not detected in the environment depicted in a second set of images generated during a second time period that is subsequent to the first time period; 
 obtain one or more predicted future states of the first object in view of the state of the first object in the environment depicted in the first set of images; 
 detect a second object included in the environment depicted in a third set of images generated during a third time period that is subsequent to the second time period wherein a number of images of the third set of images exceeds a threshold number of images; 
 determine whether a current state of the second object corresponds to at least one of the one or more predicted future state of the first object; and 
 responsive to determining that a current state of the second object corresponds to at least one of the one or more predicted future states of the first object, update an identifier associated with the second object to correspond to an identifier associated with the first object. 
 
 
     
     
       11. The system of  claim 10 , wherein to obtain the one or more predicted future states of the first object comprises, the processing device is to:
 obtain state data associated with the first object based on the state of the first object in the environment depicted in each of the first set of images; 
 calculate a path that the first object is expected to follow in the environment during a future time period based on the obtained state data; and 
 determine the one or more predicted future states of the first object based on the calculated path. 
 
     
     
       12. The system of  claim 11 , wherein to obtain the state data associated with the first object, the processing device is to:
 provide an indication of at least one of a prior state or a current state of the first object in the environment depicted in each of the first set of images as an input to one or more state prediction functions; and 
 determine the state data associated with the first object based on an output of the one or more state prediction functions. 
 
     
     
       13. The system of  claim 11 , wherein to obtain the state data associated with the first object, the processing device is to:
 provide an indication of at least one of a prior state or a current state of the first object in the environment depicted in each of the first set of images as an input to a machine learning model; 
 obtain one or more outputs of the machine learning model; 
 extract, from the one or more outputs, one or more sets of object state data and, for each set of object state data, an indication of a level of confidence that each set of object state data corresponds to the first object; and 
 identify a set of object state data associated with a level of confidence that satisfies a confidence criterion. 
 
     
     
       14. The system of  claim 13 , wherein the machine learning model comprises a recurrent neural network. 
     
     
       15. A non-transitory computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to:
 track, based on a first set of images depicting an environment, a state of a first object included in the environment, wherein the first set of images is generated during a first time period; 
 determine that the first object is not detected in the environment depicted in a second set of images generated during a second time period that is subsequent to the first time period; 
 obtain one or more predicted future states of the first object in view of the state of the first object in the environment depicted in the first set of images; 
 detect a second object included in the environment depicted in a third set of images generated during a third time period that is subsequent to the second time period, wherein a number of images of the third set of images exceeds a threshold number of images; 
 determine whether a current state of the second object corresponds to at least one of the one or more predicted future state of the first object; and 
 responsive to determining that a current state of the second object corresponds to at least one of the one or more predicted future states of the first object, update an identifier associated with the second object to correspond to an identifier associated with the first object. 
 
     
     
       16. The non-transitory computer readable storage medium of  claim 15 , to obtain the one or more predicted future states of the first object comprises, the processing device is to:
 obtain state data associated with the first object based on the state of the first object in the environment depicted in each of the first set of images; 
 calculate a path that the first object is expected to follow in the environment during a future time period based on the obtained state data; and 
 determine the one or more predicted future states of the first object based on the calculated path. 
 
     
     
       17. The non-transitory computer readable storage medium of  claim 16 , wherein to obtain the state data associated with the first object, the processing device is to:
 provide an indication of at least one of a prior state or a current state of the first object in the environment depicted in each of the first set of images as an input to one or more state prediction functions; and 
 determine the state data associated with the first object based on an output of the one or more state prediction functions. 
 
     
     
       18. The non-transitory computer readable storage medium of  claim 16 , wherein to obtain the state data associated with the first object, the processing device is to:
 provide an indication of at least one of a prior state or a current state of the first object in the environment depicted in each of the first set of images as an input to a machine learning model; 
 obtain one or more outputs of the machine learning model; 
 extract, from the one or more outputs, one or more sets of object state data and, for each set of object state data, an indication of a level of confidence that each set of object state data corresponds to the first object; and 
 identify a set of object state data associated with a level of confidence that satisfies a confidence criterion.