Video streaming anomaly detection

Abstract

Monitoring of a vehicle is provided. A plurality of video feeds captured from cameras of the vehicle are received, over a network from a vehicle, each of the plurality of video feeds including a plurality frames, each of the frames of each of the video feeds being assigned a sequence number that increases for each successive frame. The sequence numbers are analyzed to identify missing frames, delayed frames, or stale frames. The plurality of video feeds is displayed, to one or more monitors, the sequence numbers corresponding to the displayed frames, and for each of the plurality of video feeds, indications of whether any missed frames, delayed frames, or stale frames were identified.

What is claimed is: 
     
       1. A system for monitoring a vehicle, comprising:
 a monitoring system having one or more monitors, the monitoring system configured to communicate with a vehicle over a network and programmed to
 receive a plurality of video feeds captured from cameras of the vehicle, each of the plurality of video feeds including a plurality of frames, each of the frames of each of the video feeds being assigned a sequence number that increases for each successive frame; 
 analyze the sequence numbers to identify missing frames, delayed frames, or stale frames; 
 display, to the one or more monitors, the plurality of video feeds, the sequence numbers corresponding to the displayed frames, and for each of the plurality of video feeds, indications of whether any missed frames, delayed frames, or stale frames were identified, wherein a stale frame is a frame with a lower frame number than another previously received frame for the same video feed; 
 receive remote driving commands from an operator of the monitoring system based on the indications of whether any missed frames, delayed frames, or stale frames were identified; and 
 send the remote driving commands from the monitoring system to the vehicle. 
 
 
     
     
       2. The system of  claim 1 , wherein the monitoring system is further programmed to:
 identify a corresponding frame rate for each of the plurality of video feeds; 
 determine time deltas between successive frames for each of the plurality of video feeds; and 
 indicate in the one or more monitors which, if any, of the plurality of video feeds have time deltas at least a predefined threshold amount of time in excess of a time difference between frames as defined by the corresponding frame rate. 
 
     
     
       3. The system of  claim 1 , wherein the monitoring system is further programmed to:
 periodically measure connection latency between the monitoring system and the vehicle; and 
 display a graph of the connection latency between the monitoring system and the vehicle to the one or more monitors. 
 
     
     
       4. The system of  claim 3 , wherein the monitoring system is further programmed to, responsive to determining connection latency exceeds a predefined threshold, send a command to the vehicle to perform operations including one or more of to lower a resolution of one or more of the plurality of video feeds or to perform compression on one or more of the plurality of video feeds. 
     
     
       5. The system of  claim 1 , wherein the monitoring system is further programmed to identify a quantity of streams and frame rates for each of the plurality of video feeds from metadata transmitted with the plurality of video feeds. 
     
     
       6. The system of  claim 1 , wherein the monitoring system is further programmed to send a synchronize command to the vehicle to cause the vehicle to reset the sequence numbers for each of the plurality of video feeds to an initial value. 
     
     
       7. A method for monitoring a vehicle, comprising:
 receiving, to a monitoring system over a network from a vehicle, a plurality of video feeds captured from cameras of the vehicle, each of the plurality of video feeds including a plurality of frames, each of the frames of each of the video feeds being assigned a sequence number that increases for each successive frame; 
 analyzing, by the monitoring system, the sequence numbers to identify missing frames, delayed frames, or stale frames; 
 displaying, to one or more monitors of the monitoring system, the plurality of video feeds, the sequence numbers corresponding to the displayed frames, and for each of the plurality of video feeds, indications of whether any missed frames, delayed frames, or stale frames were identified, wherein a stale frame is a frame with a lower frame number than another previously received frame for the same video feed; 
 receiving remote driving commands from an operator of the monitoring system based on the indications of whether any missed frames, delayed frames, or stale frames were identified; and 
 sending the remote driving commands from the monitoring system to the vehicle. 
 
     
     
       8. The method of  claim 7 , further comprising:
 identifying a corresponding frame rate for each of the plurality of video feeds; 
 determining time deltas between successive frames for each of the plurality of video feeds; and 
 indicating in the one or more monitors which, if any, of the plurality of video feeds have time deltas at least a predefined threshold amount of time in excess of a time difference between frames as defined by the corresponding frame rate. 
 
     
     
       9. The method of  claim 7 , further comprising:
 periodically measuring connection latency between the monitoring system and the vehicle; and 
 displaying a graph of the connection latency between the monitoring system and the vehicle to the one or more monitors. 
 
     
     
       10. The method of  claim 9 , further comprising, responsive to determining connection latency exceeds a predefined threshold, sending a command to the vehicle to perform operations including one or more to lower a resolution of one or more of the plurality of video feeds or to perform compression on one or more of the plurality of video feeds. 
     
     
       11. The method of  claim 7 , further comprising identifying a quantity of streams and frame rates for each of the plurality of video feeds from metadata transmitted with the plurality of video feeds. 
     
     
       12. The method of  claim 7 , further comprising sending a synchronize command to the vehicle to cause the vehicle to reset the sequence numbers for each of the plurality of video feeds to an initial value. 
     
     
       13. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a monitoring system, cause the monitoring system to perform operations including to:
 receive, over a network from a vehicle, a plurality of video feeds captured from cameras of the vehicle, each of the plurality of video feeds including a plurality of frames, each of the frames of each of the video feeds being assigned a sequence number that increases for each successive frame; 
 analyze the sequence numbers to identify missing frames, delayed frames, or stale frames; 
 display, to one or more monitors for use by an operator of the monitoring system, the plurality of video feeds, the sequence numbers corresponding to the displayed frames, and for each of the plurality of video feeds, indications of whether any missed frames, delayed frames, or stale frames were identified, wherein a stale frame is a frame with a lower frame number than another previously received frame for the same video feed; 
 receive remote driving commands from an operator of the monitoring system based on the indications of whether any missed frames, delayed frames, or stale frames were identified; and 
 send the remote driving commands from the monitoring system to the vehicle. 
 
     
     
       14. The medium of  claim 13 , further comprising instructions that, when executed by the one or more processors of the monitoring system, cause the monitoring system to perform operations including to:
 identify a corresponding frame rate for each of the plurality of video feeds; 
 determine time deltas between successive frames for each of the plurality of video feeds; and 
 indicate in the one or more monitors which, if any, of the plurality of video feeds have time deltas at least a predefined threshold amount of time in excess of a time difference between frames as defined by the corresponding frame rate. 
 
     
     
       15. The medium of  claim 13 , further comprising instructions that, when executed by the one or more processors of the monitoring system, cause the monitoring system to perform operations including to:
 periodically measure connection latency between the monitoring system and the vehicle; and 
 display a graph of the connection latency between the monitoring system and the vehicle to the one or more monitors. 
 
     
     
       16. The medium of  claim 15 , further comprising instructions that, when executed by the one or more processors of the monitoring system, cause the monitoring system to perform operations including to responsive to determining connection latency exceeds a predefined threshold, send a command to the vehicle to perform operations including one or more of to lower a resolution of one or more of the plurality of video feeds or to perform compression on one or more of the plurality of video feeds.