Method and system for automated labeling at scale of motion-detected events in video surveillance

Abstract

A method and system are provided for automated labeling at scale of motion-detected events in video surveillance imagery that may be private to and viewable only by the owner of that imagery. Further provided are processing, organization, and archival aspects related to such automated labeling at scale.

1 . A computer-implemented method for smart labeling at scale of motion detected events in video surveillance, comprising the steps of:
 at a server, labeling at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and storing said label zone by:
 receiving said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity; 
 upon authorizing and using said upload token, presenting said source image to said end user entity for labeling; and 
 receiving at least one label name of said at least one zone of said source image, correlating said at least one label name with said at least one zone for said source image, and storing said correlated labeled zone; and 
   at a server in real time, processing label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, said processing by:
 receiving and decrypting an encrypted token that uniquely identifies said end user entity and said source camera; 
 upon authorizing said decrypted token, receiving at least a first incoming image frame, and at least a second incoming image frame from said source camera; 
 comparing one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame; 
 for each identified significant motion, determining a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays; 
 for each motion zone, determining a matching candidate labeled zone from said stored correlated labeled zones; and 
 responsive to each determined match, generating and indexing a label hit for the matching motion zone with candidate labeled zone; and 
   wherein at least one of the steps is performed by a processor.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein said correlating said at least one label name with said at least one zone for said image source comprises translating said at least one labeled zone into percentage offsets from the edges of said source image. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein said comparing uses any of: pixel resolution in analysis of specific bounding boxes, image filtering parameters, and degree of computational effort in searching for contiguous zones. 
     
     
         4 . The computer-implemented method of  claim 1 , further comprising generating tasks for each generated label hit. 
     
     
         5 . The computer-implemented method of  claim 4 , wherein a task of said generated tasks is sending a notification to the end user entity. 
     
     
         6 . The computer-implemented method of  claim 1 , wherein label hits are correlated each with any one of source images sourced from a particular camera of a particular user and wherein the user is able to perform any of: view, review, and rate the accuracy of said label hit. 
     
     
         7 . An apparatus for smart labeling at scale of motion detected events in video surveillance, comprising:
 a server adapted to label at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and to store said label zone by comprising:
 a receiver adapted to receive said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity; 
 an authorizing processor adapted to authorize said token; 
 a presenter adapted to present, using said token, said source image to said end user entity for labeling; 
 a receiver adapted to receive at least one label name of said at least one zone of said source image, to correlate said at least one label name with said at least one zone for said image source, and store said correlated labeled zone; and 
   a server adapted to process, in real time, label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, by comprising:
 a decryptor adapted to receive and decrypt an encrypted token that uniquely identifies said end user entity and said source camera; 
 a receiver adapted to receive, upon authorization of said decrypted token, at least a first incoming image frame and at least a second incoming image frame from said source camera; 
 a comparing component adapted to compare one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame; 
 a determiner adapted to determine, for each identified significant motion, a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays; 
 for each motion zone:
 a matching component adapted to determine a matching a candidate labeled zone from said stored correlated labeled zones; and 
 
 a generator adapted to generate and index a label hit for the matching motion zone with candidate labeled zone, responsive to each determined match. 
   
     
     
         8 . The apparatus of  claim 7 , wherein said correlate said at least one label name with said at least one zone for said image source comprises to translate said at least one labeled zone into percentage offsets from the edges of said source image. 
     
     
         9 . The apparatus of  claim 7 , wherein said comparing uses any of: pixel resolution in analysis of specific bounding boxes, image filtering parameters, and degree of computational effort in searching for contiguous zones. 
     
     
         10 . The apparatus of  claim 7 , further comprising a generating component adapted to generate tasks for each generated label hit. 
     
     
         11 . The apparatus of  claim 10 , wherein a task of said generated tasks is sending a notification to the end user entity. 
     
     
         12 . The apparatus of  claim 7 , wherein label hits are correlated each with any one of source images sourced from a particular camera of a particular user and wherein the user is able to perform any of: view, review, and rate the accuracy of said label hit. 
     
     
         13 . A non-transitory computer readable storage medium having stored thereon a computer program for smart labeling at scale of motion detected events in video surveillance, said computer program comprising a program code which, when executed by a processor, performs the steps of:
 at a server, labeling at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and storing said label zone by:
 receiving said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity; 
 upon authorizing and using said upload token, presenting said source image to said end user entity for labeling; and 
 receiving at least one label name of said at least one zone of said source image, correlating said at least one label name with said at least one zone for said source image, and storing said correlated labeled zone; and 
   at a server in real time, processing label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, said processing by:
 receiving and decrypting an encrypted token that uniquely identifies said end user entity and said source camera; 
 upon authorizing said decrypted token, receiving at least a first incoming image frame, and at least a second incoming image frame from said source camera; 
 comparing one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame; 
 for each identified significant motion, determining a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays; 
 for each motion zone, determining a matching candidate labeled zone from said stored correlated labeled zones; and 
 responsive to each determined match, generating and indexing a label hit for the matching motion zone with candidate labeled zone. 
   
     
     
         14 . The non-transitory computer readable storage medium of  claim 13 , wherein said program code further comprises correlating said at least one label name with said at least one zone for said image source comprising translating said at least one labeled zone into percentage offsets from the edges of said source image. 
     
     
         15 . The non-transitory computer readable storage medium of  claim 13 , wherein said comparing uses any of: pixel resolution in analysis of specific bounding boxes, image filtering parameters, and degree of computational effort in searching for contiguous zones. 
     
     
         16 . The non-transitory computer readable storage medium of  claim 13 , wherein said program code further comprises generating tasks for each generated label hit. 
     
     
         17 . The non-transitory computer readable storage medium of  claim 16 , wherein a task of said generated tasks is sending a notification to the end user entity. 
     
     
         18 . The non-transitory computer readable storage medium of  claim 13 , wherein label hits are correlated each with any one of source images sourced from a particular camera of a particular user and wherein the user is able to perform any of: view, review, and rate the accuracy of said label hit. 
     
     
         19 . A computer-implemented method for smart labeling at scale of motion detected events in video surveillance, comprising the steps of:
 at a server, labeling at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and storing said label zone by:
 receiving said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity; 
 upon authorizing and using said upload token, presenting said source image to said end user entity for labeling; and 
 receiving at least one label name of said at least one zone of said source image, correlating said at least one label name with said at least one zone for said source image, and storing said correlated labeled zone; 
   wherein at least one of the steps is performed by a processor.   
     
     
         20 . A computer-implemented method for smart labeling at scale of motion detected events in video surveillance, comprising the steps of:
 at a server in real time, processing label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, said processing by:
 receiving and decrypting an encrypted token that uniquely identifies said end user entity and said source camera; 
 upon authorizing said decrypted token, receiving at least a first incoming image frame, and at least a second incoming image frame from said source camera; 
 comparing one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame; 
 for each identified significant motion, determining a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays; 
 for each motion zone, determining a matching candidate labeled zone from said stored correlated labeled zones; and 
 responsive to each determined match, generating and indexing a label hit for the matching motion zone with candidate labeled zone; and 
   wherein at least one of the steps is performed by a processor.