Frame interpolation and apparatus using frame interpolation

Abstract

A first motion vector is estimated by using a first frame and a second frame that follows the first frame. A support frame is generated from at least either the first or the second frame by using the first motion vector. The support frame is divided into a plurality of small blocks. Motion vector candidates are estimated by using the first and second frames, in relation to each of the small blocks. A small block on the first frame, a small block on the second frame and the small blocks on the support frame are examined. The small block on the first frame and the small block on the second frame correspond to each of the motion vector candidates. A second motion vector is selected from the motion vector candidates, and points the small block on the first frame and the small block on the second frame which have the highest correlation with each of the small blocks on the support frame. An interpolated frame is generated from at least either the first or the second frame by using the second motion vector.

1 : A frame interpolation apparatus comprising: 
 a motion vector estimating unit configured to estimate a first motion vector by using a first frame and a second frame that follows the first frame;    a support frame generating unit configured to generate a support frame from at least either the first or the second frame by using the first motion vector;    a support frame dividing unit configured to divide the support frame into a plurality of small blocks;    a motion vector candidate estimating unit configured to estimate motion vector candidates by using the first and second frames, in relation to each of the small blocks;    an examining unit configured to examine a first small block on the first frame; a second small block on the second frame and the small blocks on the support frame, the first small block on the first frame and the second small block on the second frame corresponding to each of motion vector candidates;    a selecting unit configured to select a second motion vector from the motion vector candidates, the second motion vector pointing the first small block on the first frame and the second small block on the second frame which have the highest correlation with each of the small blocks on the support frame; and    an interpolated frame generating unit configured to generate an interpolated frame from at least either the first or the second frame by using the second motion vector,    wherein the support frame generating unit includes: 
 a scale-converting unit configured to scale-convert a length of the first motion vector based on time at which the frame is interpolated; and  
 a generating unit configured to generate the support frame by motion compensation using the scale-converted first motion vector.  
   
     
     
         2 : The frame interpolation apparatus according to  claim 1 , wherein the motion vector candidate estimating unit employs forward motion estimation, backward motion estimation, or bi-directional motion estimation.  
     
     
         3 : The frame interpolation apparatus according to  claim 1  further comprising: 
 a pixel detecting unit configured to detect pixels with pixel value from the support frame,    wherein the examining unit computes correlation by using only the detected pixels.    
     
     
         4 : The frame interpolation apparatus according to  claim 1 , wherein the motion vector candidate estimating unit includes: 
 a dividing unit configured to divide a third frame with no image data into a plurality of small blocks of square grid;    a correlation computing unit configured to compute correlation between a first small block in the first frame and a second small block in the second frame, the first and second small blocks being located at symmetrical positions as a block pair with one of the small blocks of the third frame as a symmetrical center, correlation being computed for all block pairs in the first and second frames; and    a vector selecting unit configured to select vectors connecting small blocks in the block pairs as the motion-vector candidates in descending order of correlation.    
     
     
         5 : The frame interpolation apparatus according to  claim 1 , wherein the examining unit includes: 
 a first obtaining unit configured to obtain the first small block from the first frame and the second small block from the second frame, the first and second small blocks determined by each motion-vector candidate;    a second obtaining unit configured to obtain a third small block from the support frame at a location corresponding to the motion vector candidates;    a first correlation computing unit configured to compute a first correlation between the first and third small blocks;    a second correlation computing unit configured to compute a second correlation between the second and third small blocks;    a detecting unit configured to detect the second motion vector from the motion-vector candidates, having maximum correlation that is sum of the first and second correlations.    
     
     
         6 : A frame interpolation apparatus comprising: 
 a support frame generating unit configured to generate a support frame from by using a first frame and a second frame that follows the first frame;    a support frame dividing unit configured to divide the support frame into a plurality of small blocks;    a first estimating unit configured to estimate at least one motion-vector candidate group by recursive searching for motion-vector candidates in each small block of the support frame;    an examining unit configured to examine the motion-vector candidate group by using the support frame, to obtain a first motion-vector group and a first unmatched-filter group; and    an interpolated frame generating unit configured to generate an interpolated frame by motion compensation using the first motion-vector group, the first unmatched-filter group, and the first and the second frames.    
     
     
         7 : The frame interpolation apparatus according to  claim 6 , wherein the support frame generating unit includes: 
 a dividing unit configured to divide the first frame into a plurality of first small blocks;    a second estimating unit configured to estimate a second motion-vector group and a second unmatched-filter group by recursive searching to each first small block;    a scale-converting unit configured to scale-convert a length of the second motion-vector group based on time at which the frame is interpolated, to obtain a third motion-vector group; and    a generating unit configured to generate the support frame by motion compensation using the third motion-vector group, the second unmatched-filer group, and the first and second frames.    
     
     
         8 : The frame interpolation apparatus according to  claim 7 , wherein the first estimating unit employs forward motion estimation, backward motion estimation, or bi-directional motion estimation.  
     
     
         9 : The frame interpolation apparatus according to  claim 6 , wherein the first estimating unit includes: 
 a correlation computing unit configured to compute correlation between a first small block in the first frame and a second small block in the second frame, the first and second small blocks being located at symmetrical positions as a block pair with one of small blocks in the third frame as a symmetrical center, correlation being computed for all block pairs in the first and second frames; and    a second estimating unit configured to estimate the motion vector candidates from vectors connecting small blocks in the block pairs in descending order of correlation.    
     
     
         10 : A frame interpolation apparatus comprising: 
 a decoding unit configured to decode a motion-compensated predictive signal to separate the signal into a video bitstream and motion-vector information;    an extracting unit configured to extract a first and a second frame and a corresponding first motion vector from the video bitstream, thus generating a support frame from the first and second frames by using the first motion vector;    an estimating unit configured to estimate motion-vector candidates from the first and second frames in relation to each block in the support frame;    an examining unit configured to examine an image block in the first frame and an image block in the second frame, and an image block in the support frame, to select one of the motion vector candidates as a second motion vector having the highest correlation as a second motion vector; and    a generating unit configured to generate an interpolated frame from image blocks, in the first and second frames, determined by the second motion vector.    
     
     
         11 : The frame interpolation apparatus according to  claim 10 , the extracting unit includes a obtaining unit configured to obtain a predictive error corresponding to decoded first and second frames and estimating the first motion vector from the decoded first and second frames if the predictive error is larger than a reference value.  
     
     
         12 : A computer-readable medium storing executable program for interpolating frame, the program comprising instructions for: 
 estimating a first motion vector by using a first frame and a second frame that follows the first frame;    generating a support frame from at least either the first or the second frame by using the first motion vector;    dividing the support frame into a plurality of small blocks;    estimating motion vector candidates by using the first and second frames, in relation to each of the small blocks;    examining a first small block on the first frame, a second small block on the second frame and the small blocks on the support frame, the first small block on the first frame and the second small block on the second frame corresponding to each of motion vector candidates;    selecting a second motion vector from the motion vector candidates, the second motion vector pointing the small block on the first frame and the small block on the second frame which have the highest correlation with each of the small blocks on the support frame; and    generating an interpolated frame from at least either the first or the second frame by using the second motion vector,    wherein the instruction for generating the support frame includes instructions for:    scale-converting a length of the first motion vector based on time at which the frame is interpolated; and    generating the support frame by motion compensation using the scale-converted first motion vector.    
     
     
         13 : The computer-readable medium storing executable program for interpolating frame according to  claim 12 , wherein the instruction for estimating motion-vector candidates employs forward motion estimation, backward motion estimation, or bi-directional motion estimation.  
     
     
         14 : The computer-readable medium storing executable program for interpolating frame according to  claim 12 , the program further comprising instructions for: 
 detecting pixels with pixel value from the support frame,    wherein the examining of the image block computes correlation by using only the detected pixels.    
     
     
         15 : The computer-readable medium storing executable program for interpolating frame according to  claim 12 , wherein the instruction for estimating of motion vector candidates includes instructions for: 
 dividing a third frame with no image data into a plurality of small blocks of square grid;    computing correlation between a first small block in the first frame and a second small block in the second frame, the first and second small blocks being located at symmetrical positions as a block pair with one of the small blocks of the third frame as a symmetrical center, correlation being computed for all block pairs in the first and second frames; and    selecting vectors connecting small blocks in the block pairs as the motion-vector candidates in descending order of correlation.    
     
     
         16 : The computer-readable medium storing executable program for interpolating frame according to  claim 12 , wherein the instruction for examining the image block includes instructions for: 
 obtaining the first small block from the first frame and the second small block from the second frame, the first and second small blocks determined by each motion-vector candidate;    obtaining a third small block from the support frame at a location corresponding to the motion vector candidates;    computing a first correlation between the first and third small blocks;    computing a second correlation between the second and third small blocks;    detecting the second motion vector from the motion-vector candidates, having maximum correlation that is sum of the first and second correlations.    
     
     
         17 : A computer-readable frame medium storing executable program for interpolating frame, the program comprising instructions for: 
 generating a support frame from by using a first frame and a second frame that follows the first frame;    dividing the support frame into a plurality of small blocks;    estimating at least one motion-vector candidate group by recursive searching for motion-vector candidates in each small block of the support frame;    examining the motion-vector candidate group by using the support frame, to obtain a first motion-vector group and a first unmatched-filter group; and    generating an interpolated frame by motion compensation using the first motion-vector group, the first unmatched-filter group, and the first and the second frames.    
     
     
         18 : The computer-readable medium storing executable program for interpolating frame according to  claim 17 , wherein the instruction for generating the support frame includes instructions for: 
 dividing the first frame into a plurality of first small blocks;    estimating a second motion-vector group and a second unmatched-filter group by recursive searching to each first small block;    scale-converting a length of the second motion-vector group based on time at which the frame is interpolated, to obtain a third motion-vector group; and    generating the support frame by motion compensation using the third motion-vector group, the second unmatched-filer group, and the first and second frames.    
     
     
         19 : The computer-readable medium storing executable program for interpolating frame according to  claim 18 , wherein the instruction for estimating at least one motion-vector candidates employs forward motion estimation, backward motion estimation, or bi-directional motion estimation.  
     
     
         20 : The computer-readable medium storing executable program for interpolating frame according to  claim 17 , wherein the instruction for estimating at least one motion-vector candidates includes instructions for: 
 computing correlation between a first small block in the first frame and a second small block in the second frame, the first and second small blocks being located at symmetrical positions as a block pair with one of small blocks in the third frame as a symmetrical center, correlation being computed for all block pairs in the first and second frames; and    estimating the motion vector candidates from vectors connecting small blocks in the block pairs in descending order of correlation.    
     
     
         21 : A computer-readable medium storing executable program for interpolating frame, the program comprising instructions for: 
 decoding a motion-compensated predictive signal to separate the signal into a video bitstream and motion-vector information;    extracting a first and a second frame and a corresponding first motion vector from the video bitstream, thus generating a support frame from the first and second frames by using the first motion vector;    estimating motion-vector candidates from the first and second frames in relation to each block in the support frame;    examining an image block in the first frame and an image block in the second frame, and an image block in the support frame, to select one of the motion vector candidates as a second motion vector having the highest correlation as a second motion vector; and    generating an interpolated frame from image blocks, in the first and second frames, determined by the second motion vector.    
     
     
         22 : The computer-readable medium storing executable program for interpolating frame according to  claim 21 , wherein the instruction for extracting first and second frames includes instruction for obtaining a predictive error corresponding to decoded first and second frames and estimating the first motion vector from the decoded first and second frames if the predictive error is larger than a reference value.