Method for video processing using a buffer

Abstract

A method for processing video comprises storing, at a video data buffer, an input video frame data received from a source, causing the stored video frame data to be output from the video data buffer at an output video frame rate, and varying the output video frame rate based on a comparison of an amount of video frame data stored at the video data buffer to a threshold amount of frame data. The threshold amount of frame data may be based on a target total latency between capture of the input video frame data at the source and display of the stored video frame data output from the video data buffer.

What is claimed is: 
     
       1. A method for processing video, comprising:
 storing, at a video data buffer, an input video frame data received from a source; 
 causing the stored video frame data to be output from the video data buffer at an output video frame rate; and 
 varying the output video frame rate based on a comparison of an amount of video frame data stored at the video data buffer to a threshold amount of frame data, 
 wherein the threshold amount of frame data is based on a target total latency between capture of the input video frame data at the source and display of the stored video frame data output from the video data buffer. 
 
     
     
       2. The method of  claim 1 , further comprising setting the output video frame rate to be lower than a nominal frame rate on a condition that the amount of video frame data stored at the video data buffer is lower than the threshold amount of frame data. 
     
     
       3. The method of  claim 2 , further comprising setting the output video frame rate to be higher than the nominal frame rate on a condition that the amount of video frame data stored at the video data buffer is higher than the threshold amount of frame data. 
     
     
       4. The method of  claim 3 , wherein the nominal frame rate is set based on a frame rate of the source. 
     
     
       5. The method of  claim 4 , wherein:
 the source comprises an image capture device, and 
 the nominal frame rate is a frame rate at which the image capture device captures image frames, the captured image frames making up the input video frame data. 
 
     
     
       6. The method of  claim 1 , further comprising setting the output video frame rate to be lower than a nominal frame rate by a predetermined amount on a condition that the amount of video frame data stored at the video data buffer is lower than the threshold amount of frame data. 
     
     
       7. The method of  claim 1 , further comprising setting the output video frame rate to be lower than a nominal frame rate by an amount that is proportional to a difference between the amount of video frame data stored at the video data buffer and the threshold amount of frame data on a condition that the amount of video frame data stored at the video data buffer is lower than the threshold amount of frame data. 
     
     
       8. The method of  claim 1 , further comprising setting the output video frame rate to be higher than a nominal frame rate by a predetermined amount on a condition that the amount of video frame data stored at the video data buffer is higher than the threshold amount of frame data. 
     
     
       9. The method of  claim 1 , further comprising setting the output video frame rate to be higher than a nominal frame rate by an amount that is proportional to a difference between the amount of video frame data stored at the video data buffer and the threshold amount of frame data on a condition that the amount of video frame data stored at the video data buffer is higher than the threshold amount of frame data. 
     
     
       10. The method of  claim 1 , wherein the threshold amount of frame data comprises a first threshold amount of frame data, the method further comprising:
 decreasing the output video frame rate on a condition that the amount of video frame data stored at the video data buffer is lower than the first threshold amount of frame data; and 
 increasing the output video frame rate on a condition that the amount of video frame data stored at the video data buffer is higher than a second threshold amount of frame data. 
 
     
     
       11. The method of  claim 10 , further comprising setting the first and second threshold amounts of frame data based on the target total latency. 
     
     
       12. The method of  claim 10 , wherein:
 the first threshold amount of frame data corresponds to a first amount of time equal to the target total latency minus a transmission latency minus an offset, the transmission latency being a latency between capture of the input video frame data at the source and receipt of the input video frame data at the video data buffer, and 
 the second threshold amount of frame data corresponds to a second amount of time equal to the target total latency minus the transmission latency plus the offset. 
 
     
     
       13. The method of  claim 10 , further comprising setting the output video frame rate to equal the nominal frame rate on a condition that the amount of video frame data stored at the video data buffer is in a range from the first threshold amount of frame data to the second threshold amount of frame data. 
     
     
       14. The method of  claim 1 , further comprising displaying, on a display device, the stored video frame data output from the video data buffer. 
     
     
       15. The method of  claim 14 , further comprising:
 receiving, via one or more input devices, inputs from a user; and 
 controlling one or more instruments in response to the inputs. 
 
     
     
       16. The method of  claim 15 , wherein the source is an image capture device and the image capture device is one of the instruments. 
     
     
       17. The method of  claim 16 , further comprising setting the threshold amount of frame data such that an average latency between capture of frames of the input video frame data at the image capture device and display of the frames at the display device is equal to the target total latency. 
     
     
       18. The method of  claim 1 , wherein the threshold amount of data is a first threshold amount of data, the method further comprising:
 decreasing the output video frame rate if the amount of video frame data stored in the video data buffer is less than the first threshold amount of frame data; 
 increasing the output video frame rate if the amount of video frame data stored in the video data buffer is greater than a second threshold amount of frame data; and 
 setting the first threshold amount of frame data and the second threshold amount of frame data such that a midpoint between the first threshold amount of frame data and the second threshold amount of frame data is equal to an amount of data equivalent to the target total latency minus a transfer period, the transfer period being a latency between capturing the input video frame data at the source storing the input video frame data at the video data buffer. 
 
     
     
       19. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a video processing device, cause the video processing device to perform operations comprising:
 storing, at a video data buffer, input video frame data received from a source; 
 causing the stored video frame data to be output from the video data buffer at an output video frame rate; and 
 varying the output video frame rate based on a comparison of an amount of video frame data stored at the video data buffer to a threshold amount of frame data, 
 wherein the threshold amount of frame data is based on a target total latency between capture of the input video frame data at the source and display of the stored video frame data output from the video data buffer.