Video encoding system and method for providing content adaptive rate control

Abstract

A video encoding system ( 200 ) provides content adaptive rate control that includes a visual analyzer ( 208 ) utilizing at least one visual analysis tool for processing a video frame to provide visual information describing the video frame. An encoder ( 204 ) generates encoding status information relating to the video frame being processed. A rate controller ( 206 ) is responsive to the encoding status information generated by the encoder ( 204 ) and the visual information generated by the visual analyzer ( 208 ) to generate a rate control adjustment signal. The encoder ( 204 ) is responsive to the rate control adjustment signal for encoding the video frame.

1 . A video encoding system providing content adaptive rate control comprising: 
 a visual analyzer, utilizing at least one visual analysis tool, for processing a video frame to provide visual information describing the video frame;    an encoder for generating encoding status information relating to the video frame; and    a rate controller, responsive to the encoding status information, and further responsive to the visual information being generated by said visual analyzer, for generating a rate control adjustment information,    said encoder being responsive to the rate control adjustment information for encoding the video frame.    
     
     
         2 . The video encoding system according to  claim 1  wherein the video frame is processed in real time by said visual analyzer, said rate controller, and said encoder.  
     
     
         3 . The video encoding system according to  claim 1  further comprising a storage device for storing the visual information generated by said visual analyzer, 
 wherein the visual information being stored is used by said rate controller and said encoder for enabling the encoding of the video frame being processed.    
     
     
         4 . The video encoding system according to  claim 3 , wherein the visual information includes a visual analysis metric computed by said at least one visual analysis tool for the video frame currently being processed, and wherein 
 said storage device stores the visual analysis metric for the video frame currently being processed and for video frames previously processed.    
     
     
         5 . The video encoding system according to  claim 4 , wherein said at least one visual analysis tool is a color layout descriptor tool, and wherein 
 said color layout descriptor tool generates a color layout descriptor metric providing an approximation of the visual content changes throughout a video sequence.    
     
     
         6 . The video encoding system according to  claim 1 , wherein said at least one visual analysis metric is used for computing a distance metric between the frame currently being processed and a frame previously processed; and wherein 
 said rate controller further computes a frame drop decision function using the distance metric and compares the frame drop decision function computed with a frame drop decision parameter to determine when the frame is intended to be dropped.    
     
     
         7 . The video encoding system according to  claim 6 , wherein said rate controller compares the distance metric computed for the frame when the frame is not intended to be dropped with a frame coding parameter; and wherein 
 the rate controller generates the rate control adjustment information and in response thereto said encoder    encodes the frame as an intra-frame when the distance metric is greater than the frame coding parameter; and    further encodes the frame as an inter-frame when the distance metric is less than the frame coding parameter.    
     
     
         8 . A video encoding system providing content adaptive rate control comprising: 
 a visual analyzer utilizing visual analysis tools for processing a video frame to compute visual analysis metrics describing the video frame;    a rate controller, responsive to the visual analysis metrics being computed, for computing a distance metric and a frame drop decision function, the frame drop decision function being used for determining when the video frame is intended to be dropped and when the video frame is not intended to be dropped;    said visual analyzer utilizing a visual analysis tool for processing a video frame that is not intended to be dropped to compute a visual analysis metric describing the video frame not intended to be dropped,    said rate controller further computing a visual information quantization metric using the visual analysis metric computed for the frame that is not intended to be dropped; and    an encoder for encoding the frame that is not intended to be dropped using the visual information quantization metric.    
     
     
         9 . The video encoding system according to  claim 8  wherein said visual analysis tools include at least a color location descriptor tool.  
     
     
         10 . The video encoding system according to  claim 9  wherein said visual analysis tools further include at least a motion activity descriptor tool.  
     
     
         11 . The video encoding system according to  claim 8  wherein said visual analysis tool used for process a video frame that is not intended to be dropped is a texture descriptor tool.  
     
     
         12 . The video encoding system according to  claim 8  wherein said rate controller determines when the encoding of the video frame currently being processed is complete and when the encoding of the video frame currently being processed is not complete; wherein 
 said rate controller updates the visual information quantization metric for the video frame currently being processed when the encoding of the frame currently being processed is not complete.    
     
     
         13 . The video encoding system according to  claim 12  wherein said rate controller: further determines when the encoding of the frame currently being processed is complete to enable the processing a next video frame of the sequence of video frames.  
     
     
         14 . A method for video encoding using content adaptive rate control comprising: 
 inputting a frame from a sequence of video frames;    processing the frame using a visual analysis tool to compute a visual analysis metric describing the frame;    computing a distance metric between the frame currently being processed and a frame previously processed;    computing a frame drop decision function using the distance metric; and    comparing the frame drop decision function computed with a frame drop decision parameter to determine when the frame is intended to be dropped.    
     
     
         15 . The method for video encoding according to  claim 14 , wherein said step of inputting and said step of processing are performed by a visual analyzer, said step of computing and said step of determining are performed by a rate controller and said step of encoding is performed by an encoder.  
     
     
         16 . The method for video encoding according to  claim 14 , wherein the frame drop decision parameter is stored in a storage device.  
     
     
         17 . The method for video encoding according to  claim 14 , wherein the visual analysis tool is a color layout descriptor tool.  
     
     
         18 . The method for video encoding according to  claim 14 , further comprising: 
 comparing the distance metric computed for the frame when the frame is not intended to be dropped with a frame coding parameter; and    encoding the frame as an intra-frame when the distance metric is greater than the frame coding parameter; and    encoding the frame as an inter-frame when the distance metric is less than the frame coding parameter?   
     
     
         19 . The method for video encoding according to  claim 18 , wherein 
 the frame is encoded as the intra-frame by encoding a difference between the frame currently being processed and the frame previously being processed, and    the frame is encoded as the inter-frame by encoding the frame currently being processed.    
     
     
         20 . The method for video encoding according to  claim 18 , wherein the step of comparing is performed in a rate controller, and the step of encoding is performed in an encoder.  
     
     
         21 . The method for video encoding according to  claim 14 , further comprising 
 processing the frame using a second visual analysis tool to compute a second visual analysis metric describing the frame;    computing the distance metric for the frame currently being processed using the visual analysis metric derived for the frame currently being processed and a frame previously processed and the second visual analysis metric for the frame currently being processed;    computing the frame drop decision function using the distance metric; and    comparing the frame drop decision function computed with a frame drop decision parameter to determine when the frame is intended to be dropped.    
     
     
         22 . The method for video encoding according to  claim 21 , wherein the second visual analysis tool is a motion activity descriptor tool.  
     
     
         23 . A method for video encoding using content adaptive rate control comprising: 
 inputting a frame from a sequence of video frames;    processing the frame using visual analysis tools compute visual analysis metrics describing the frame, the visual analysis metrics being used to compute a distance metric and a frame drop decision function, the frame drop decision function being used for determining when the frame is intended to be dropped and when the frame is not intended to be dropped;    computing a visual information quantization metric using the visual analysis metrics computed for a frame that is not intended to be dropped; and    encoding the frame using the visual quantization metrics.    
     
     
         24 . The method for video encoding according to  claim 23 , further comprising: 
 determining when the encoding of the frame currently being processed is complete and when the encoding of the frame currently being processed is not complete; and    updating the visual information quantization metric for the frame currently being processed when the encoding of the frame currently being processed is not complete.    
     
     
         25 . The method for video encoding according to  claim 24 , further comprising: 
 determining when the encoding of the frame currently being processed is complete;    inputting a next frame for processing.    
     
     
         26 . The method for video encoding according to  claim 20 , wherein the visual analysis tools comprise at least one of a color layout description tool, a motion activity descriptor tool and a texture descriptor tool.  
     
     
         27 . The method for video encoding according to  claim 20 , wherein said step of inputting and said step of processing are performed by a visual analyzer, said step of computing is performed by a rate controller and said step of encoding is performed by an encoder.