Bandwidth efficient video surveillance system

Abstract

A system and method are disclosed for providing a real-time wireless video surveillance system. The video surveillance system leverages edge computing to enable wireless video surveillance distributing video processing between edges of the network and the cloud to reduce the amount of video that is uploaded to the cloud for analysis.

We claim: 
     
         1 . A method of selectively uploading video of a live event, the method comprising:
 receiving analytic data from each of a plurality of computing devices associated with a plurality of video cameras capturing videos of the live event, the analytic data generated from processing of the captured videos to indicate utility detected in the videos, where utility of a video is a measure of a number of tracked items detected in the video relevant to a search criterion for the video;   determining a computing device from which to request video based on the analytic data indicating that the computing device has a highest average utility, among the plurality of computing devices, over a period of time; and   issuing a request for video from the computing device determined to have the highest average utility over the period of time to upload.   
     
     
         2 . The method of  claim 1 , wherein at least the computing device determined to have the highest utility comprises a wireless transmitter for wirelessly transmitting the video, the method further comprising receiving from the wireless transmitter all of the video during the period of time when a product of wireless transmit capacity of the wireless transmitter and a length of the period of time is greater than a size of the video frames to be uploaded in the period of time, and receiving from the wireless transmitter part of the video being transmitted during the period of time, in a decreasing order of utility, when the product of the wireless transmit capacity of the wireless transmitter and a length of the period of time is less than a size of the video frames to be uploaded during the period of time. 
     
     
         3 . The method of  claim 1 , wherein issuing a request for video from the computing device further comprises requesting a video frame of the video from a period of time which has a higher utility than video from other periods of time. 
     
     
         4 . The method of  claim 1 , wherein the computing device comprises a first computing device, the method further comprising the steps of:
 identifying one or more secondary computing devices capturing at least some video of the live event which overlaps with the video captured by the first computing device;   re-identifying at least one item from the video from the first computing device in corresponding frames of video from the one or more secondary computing devices to determine a secondary computing device from the one or more secondary computing devices from which to request video based on the analytic data indicating that the secondary computing device has items which are not tracked by the first computing device over the period of time;   prioritizing video frames from the first computing device and the secondary computing device by utility; and   issuing a request for video from the first computing device and the secondary computing device in accordance with the prioritizing.   
     
     
         5 . The method of  claim 4 , further comprising the steps of determining a number of tracked items in video outputs of the first computing device and the one or more secondary computing devices, reducing a count of the tracked items in the one or more secondary computing devices for each tracked item identified in corresponding video frames from the first computing device, and prioritizing transmission of video from the first computing device and the one or more secondary computing devices based on the count of the tracked items in each frame to be transmitted from the first computing device and the one or more secondary computing devices. 
     
     
         6 . The method of  claim 1 , wherein receiving analytic data further comprises receiving analytic data which includes one or more frame indices representing one or more periods of time having a different number of tracked items than one or more immediately preceding periods of time, and a frame utility of tracked items in the one or more periods of time associated with the one or more frame indices. 
     
     
         7 . The method of  claim 6 , wherein receiving analytic data further comprises receiving analytic data which includes a count of tracked items in the one or more periods of time associated with the one or more frame indices. 
     
     
         8 . The method of  claim 1 , further comprising receiving the video from the computing device at least in part via a wireless network whereby transmission of a first video frame from the computing device has been prioritized over a second video frame where the first video frame has a higher utility. 
     
     
         9 . The method of  claim 8 , further comprising prioritizing an order in which the plurality of video frames are uploaded based on bandwidth availability of the wireless network and the analytic data of the video frames to be uploaded during the period of time, the analytic data identifying as the computing device a computing device having a highest average utility of tracked items to upload during the period of time. 
     
     
         10 . The method of  claim 1 , further comprising determining utility for the plurality of computing devices as a measure of one of: i) numbers of tracked items in periods of time, ii) whether a tracked item is detected or not detected in the periods of time, and iii) a probability measure of whether a tracked item is detected in the periods of time. 
     
     
         11 . A computer readable storage medium having instructions stored thereon that, when executed by a processor, cause the processor to perform a method of selectively uploading video of a live event, the method comprising:
 receiving analytic data from each of a plurality of computing devices associated with a plurality of video cameras capturing videos of the live event, the analytic data generated from processing of the captured videos to indicate utility detected in the videos, where utility of a video is a measure of a number of tracked items detected in the video relevant to a search criterion for the video;   determining a computing device from which to request video based on the analytic data indicating that the computing device has a highest average utility, among the plurality of computing devices, over a period of time; and   issuing a request for video from the computing device determined to have the highest average utility over the period of time to upload.   
     
     
         12 . The computer readable medium of  claim 11 , wherein at least the computing device determined to have the highest utility comprises a wireless transmitter for wirelessly transmitting the analytic data, the instructions further controlling the processor to perform steps including receiving from the wireless transmitter all of the video during the period of time when a product of wireless transmit capacity of the wireless transmitter and a length of the period of time is greater than a size of the video frames to be uploaded in the period of time, and receiving from the wireless transmitter part of the video being transmitted during the period of time, in a decreasing order of utility, when the product of the wireless transmit capacity of the wireless transmitter and a length of the period of time is less than a size of the video frames to be uploaded during the period of time. 
     
     
         13 . The computer readable medium of  claim 11 , wherein the instructions for controlling the processor to issue a request for video from the computing device further comprises instructions for controlling the processor to perform the step of requesting a video frame of the video from a period of time which has a higher utility than video from other periods of time. 
     
     
         14 . The computer readable medium of  claim 11 , wherein the computing device comprises a first computing device, the instructions further comprising instructions for controlling the processor to perform the steps of:
 identifying one or more secondary computing devices capturing at least some video of the live event which overlaps with the video captured by the first computing device;   re-identifying at least one item from the video from the first computing device in corresponding frames of video from the one or more secondary computing devices to determine a secondary computing device from the one or more secondary computing devices from which to request video based on the analytic data indicating that the secondary computing device has items which are not tracked by the first computing device over the period of time;   prioritizing video frames from the first computing device and the secondary computing device by utility; and   issuing a request for video from the first computing device and the secondary computing device in accordance with the prioritizing.   
     
     
         15 . The computer readable medium of  claim 14 , wherein the instructions for controlling the processor further comprise instructions for performing the steps of determining a number of tracked items in video outputs of the first computing device and the one or more secondary computing devices, reducing a count of the tracked items in the one or more secondary computing devices for each tracked item identified in corresponding video frames from the first computing device, and prioritizing transmission of video from the first computing device and the one or more secondary computing devices based on the count of the tracked items in each frame to be transmitted from the first computing device and the one or more secondary computing devices. 
     
     
         16 . The computer readable medium of  claim 11 , wherein the instructions for controlling the processor to receive analytic data further comprises instructions for controlling the processor to perform the step of receiving analytic data which includes one or more frame indices representing one or more periods of time having a different number of tracked items than one or more immediately preceding periods of time, and a frame utility of tracked items in the one or more periods of time associated with the one or more frame indices. 
     
     
         17 . The method of  claim 16 , wherein the instructions for controlling the processor to receive analytic data further comprises instructions for controlling the processor to receive analytic data including a count of tracked items in the one or more periods of time associated with the one or more frame indices. 
     
     
         18 . The computer readable medium of  claim 11 , wherein the instructions for controlling the processor further comprise instructions for performing the step of receiving the video from the computing device at least in part via a wireless network whereby transmission of a first video frame from the computing device has been prioritized over a second video frame where the first video frame has a higher utility. 
     
     
         19 . The computer readable medium of  claim 18 , wherein the instructions for controlling the processor further comprise instructions for performing the step of prioritizing an order in which the plurality of video frames are uploaded based on bandwidth availability of the wireless network and the analytic data of the video frames to be uploaded during the period of time, the analytic data identifying as the computing device a computing device having a highest average utility of tracked items to upload during the period of time. 
     
     
         20 . The computer readable medium of  claim 11 , wherein the instructions for controlling the processor further comprise instructions for determining utility for the plurality of computing devices as a measure of one of: i) numbers of tracked items in periods of time, ii) whether a tracked item is detected or not detected in the periods of time, and iii) a probability measure of whether a tracked item is detected in the periods of time.