Video-based systems and methods for generating compliance-annotated motion trails in a video sequence for assessing rule compliance for moving objects

Abstract

The present disclosure uses automated, computer-driven analysis of video feeds to extract raw motion metadata, identify moving objects and assess whether or not the moving objects are compliant with certain predefined rules. The motion can be visually plotted on a background in the form of motion trails, and a chronological plot of the motion may be provided as a menu to find anomalies through visual screening. The motion metadata may be filtered through annotations. A highlighted portion of the video feed relating to a specific topic of interest may be replayed.

What is claimed is: 
     
         1 . A method for generating compliance-annotated motion trails for a scene, the method comprising:
 receiving a video sequence for the scene, the video sequence comprising a series of image frames capturing the scene;   identifying moving objects within the video sequence;   identifying time-stamped frame-to-frame vector sequences representing movement of the identified moving objects within the scene, wherein each identified moving object has its own vector sequence;   classifying the identified moving objects into categories, wherein the categories include at least human beings and non-human objects;   applying classification-based compliance rules to the respective vector sequences for each of the classified identified moving objects to determine non-compliant portions of the vector sequences and compliant portions of the vector sequences;   presenting an image of the scene with at least one visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects overlaid on the image.   
     
     
         2 . The method of  claim 1 , wherein identifying time-stamped frame-to-frame vector sequences representing movement of the identified moving objects within the scene comprises, for each identified moving object, assigning a unique identifier to that moving object and associating a time-series of movement vectors for that moving object with the unique identifier for that moving object. 
     
     
         3 . The method of  claim 1 , further comprising storing a subset of image frames from the video sequence for the scene, wherein the subset has a subset image frame frequency that is less than a sequence image frame frequency for the video sequence. 
     
     
         4 . The method of  claim 1 , wherein the category of non-human objects includes subcategories comprising two or more of the following:
 fixed machine with moving parts;   vehicle; and   transported object;   
       and wherein classifying the identified moving objects into categories comprises classifying the non-human objects into the subcategories. 
     
     
         5 . The method of  claim 1 , wherein the classification-based compliance rules include one or more of:
 boundary-based rules;   speed-based rules;   proximity-based rules;   combination-based rules; and   sequence-based rules.   
     
     
         6 . The method of  claim 1 , wherein presenting an image of the scene with at least one visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects overlaid on the image comprises presenting visible motion trails for the respective vector sequences for respective particular ones of a plurality of the classified identified moving objects overlaid on the image. 
     
     
         7 . The method of  claim 1 , wherein presenting an image of the scene with at least one visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects overlaid on the image is in response to a selection. 
     
     
         8 . The method of  claim 1 , wherein the visible motion trails are overlaid on the image for one of:
 all of the classified identified moving objects; and   a subset of the classified identified moving objects.   
     
     
         9 . The method of  claim 1 , wherein the at least one visible motion trail represents one of the non-compliant portion of the vector sequence, the compliant portion of the vector sequence, the complete vector sequence and an arbitrary portion of the vector sequence. 
     
     
         10 . The method of  claim 1 , wherein the visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects that is classified as an idle object comprises a static indication of the idle object. 
     
     
         11 . The method of  claim 1 , wherein the at least one visible motion trail is colour-coded according to at least one of:
 its classification;   its speed; and   the classification-based compliance rules.   
     
     
         12 . The method of  claim 1 , further comprising, responsive to a selection of a particular one of the classified identified moving objects, replaying a subset of the video sequence showing that particular one of the classified identified moving objects. 
     
     
         13 . The method of  claim 12 , further comprising indicating, in association with the subset of the video sequence showing that particular one of the classified identified moving objects, at least one of compliant and non-compliant behavior of that particular one of the classified identified moving objects. 
     
     
         14 . The method of  claim 12 , wherein:
 the selection specifies one of a time-based portion of the vector sequence for the particular one of the classified identified moving objects, the compliant portion of the vector sequence for the particular one of the classified identified moving objects and the non-compliant portion of the vector sequence for the particular one of the classified identified moving objects.   
     
     
         15 . The method of  claim 1 , wherein a portion of the scene is excluded from classification. 
     
     
         16 . A data processing system comprising memory and at least one processor, wherein the memory contains instructions which, when executed by the at least one processor, cause the at least one processor to carry out a method for generating compliance-annotated motion trails in a video sequence from a fixed scene, the method comprising: 
     
     
         17 . The data processing system of  claim 16 , wherein identifying time-stamped frame-to-frame vector sequences representing movement of the identified moving objects within the scene comprises, for each identified moving object, assigning a unique identifier to that moving object and associating a time-series of movement vectors for that moving object with the unique identifier for that moving object. 
     
     
         18 . The data processing system of  claim 16 , wherein the method further comprises storing a subset of image frames from the video sequence for the scene, wherein the subset has a subset image frame frequency that is less than a sequence image frame frequency for the video sequence. 
     
     
         19 . The data processing system of  claim 16 , wherein the category of non-human objects includes subcategories comprising two or more of the following:
 fixed machine with moving parts;   vehicle; and   transported object;   
       and wherein classifying the identified moving objects into categories comprises classifying the non-human objects into the subcategories. 
     
     
         20 . The data processing system of  claim 16 , wherein the classification-based compliance rules include one or more of:
 boundary-based rules;   speed-based rules;   proximity-based rules;   combination-based rules; and   sequence-based rules.   
     
     
         21 . The data processing system of  claim 16 , wherein presenting an image of the scene with at least one visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects overlaid on the image comprises presenting visible motion trails for the respective vector sequences for respective particular ones of a plurality of the classified identified moving objects overlaid on the image. 
     
     
         22 . The data processing system of  claim 16 , wherein presenting an image of the scene with at least one visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects overlaid on the image is in response to a selection. 
     
     
         23 . The data processing system of  claim 16 , wherein the visible motion trails are overlaid on the image for one of:
 all of the classified identified moving objects; and   a subset of the classified identified moving objects.   
     
     
         24 . The data processing system of  claim 16 , wherein the at least one visible motion trail represents one of the non-compliant portion of the vector sequence, the compliant portion of the vector sequence, the complete vector sequence and an arbitrary portion of the vector sequence. 
     
     
         25 . The data processing system of  claim 16 , wherein the visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects that is classified as an idle object comprises a static indication of the idle object. 
     
     
         26 . The data processing system of  claim 16 , wherein the at least one visible motion trail is colour-coded according to at least one of:
 its classification;   its speed; and   the classification-based compliance rules.   
     
     
         27 . The data processing system of  claim 16 , wherein the method further comprises, responsive to a selection of a particular one of the classified identified moving objects, replaying a subset of the video sequence showing that particular one of the classified identified moving objects. 
     
     
         28 . The data processing system of  claim 27 , wherein the method further comprises indicating, in association with the subset of the video sequence showing that particular one of the classified identified moving objects, at least one of compliant and non-compliant behavior of that particular one of the classified identified moving objects. 
     
     
         29 . The data processing system of  claim 27 , wherein:
 the selection specifies one of a time-based portion of the vector sequence for the particular one of the classified identified moving objects, the compliant portion of the vector sequence for the particular one of the classified identified moving objects and the non-compliant portion of the vector sequence for the particular one of the classified identified moving objects.   
     
     
         30 . The data processing system of  claim 16 , wherein a portion of the scene is excluded from classification. 
     
     
         31 . A computer program product comprising a tangible computer-readable medium embodying instructions which, when executed by a data processing system, cause the data processing system to carry out a method for generating compliance-annotated motion trails in a video sequence from a fixed scene, the method comprising:
 receiving a video sequence for the scene, the video sequence comprising a series of image frames capturing the scene;   identifying moving objects within the video sequence;   identifying time-stamped frame-to-frame vector sequences representing movement of the identified moving objects within the scene, wherein each identified moving object has its own vector sequence;   classifying the identified moving objects into categories, wherein the categories include at least human beings and non-human objects;   applying classification-based compliance rules to the respective vector sequences for each of the classified identified moving objects to determine non-compliant portions of the vector sequences and compliant portions of the vector sequences;   presenting an image of the scene with at least one visible motion trail corresponding to a respective vector sequence for a respective particular one of the classified identified moving objects overlaid on the image.