Image processing apparatus, image processing method, and program

Abstract

It is a subject to enable a moving image whose motion is smooth to be generated from a moving image of photographed frames. A motion vector calculating unit calculates a motion vector between the frame images of the moving image. An interpolation frame image generating unit generates an interpolation frame image corresponding to a time position between the frame images on the basis of the motion vector. A motion vector reliability calculating unit, a synthesizing ratio calculating unit, and an image synthesizing unit add a motion blur to the frame image by using the frame image and the interpolation frame image.

What is claimed is: 
     
       1. An image processing apparatus comprising:
 one or more processors; and 
 a memory storing instructions which, when the instructions are executed by the processor, cause the image processing apparatus to function as: 
 a calculating unit configured to calculate a motion vector between frame images of a moving image; 
 an interpolating unit configured to generate a plurality of interpolation frame images corresponding to a time position between frame images which are adjacent with respect to time on the basis of the motion vector; and 
 a generating unit configured to generate a frame image with motion blur acquired by adding a motion blur by synthesizing the frame image and the plurality of interpolation frame images, wherein 
 the generating unit determines a synthesizing ratio of synthesizing the frame image with one of the plurality of interpolation frame images, based on, a time span indicating difference between the time position of the frame image and the time position of one of the plurality of interpolation frame images, and 
 the generating unit acquires a reliability of the motion vector and time spans between the frame images and the interpolation frame image which are precedent and subsequent with respect to time, and determines the synthesizing ratio on the basis of the reliability of the motion vector and the time spans. 
 
     
     
       2. An apparatus according to  claim 1 , wherein the generating unit generates the frame image with blur of a same frame rate as a frame rate at the time when the moving image is photographed. 
     
     
       3. An apparatus according to  claim 1 , wherein when the reliability of the motion vector is smaller than a first threshold value, the generating unit sets the synthesizing ratio to the interpolation frame image at a time position where the time spans from both of the frame image which is precedent with respect to the time and the frame mage which is subsequent with respect to the ti e re larger than a predetermined span to a value smaller than the synthesizing ratio to the interpolation frame image at a time position where the time spans are equal to or smaller than the predetermined span. 
     
     
       4. An apparatus according to  claim 1 , wherein when the reliability of the motion vector is smaller than a first threshold value, the generating unit sets the synthesizing ratio to the interpolation frame image at a time position where the time span from the frame image which is precedent with respect to the time is larger than a predetermined span to a value smaller than the synthesizing ratio to the interpolation frame image at a time position where the time span is equal to or smaller than the predetermined span. 
     
     
       5. An apparatus according to  claim 1 , wherein when the reliability of the motion vector is equal to or larger than a first threshold value, the generating unit sets the synthesizing ratios to all images to a same predetermined ratio. 
     
     
       6. An apparatus according to  claim 3 , wherein the generating unit calculates the synthesizing ratio in such a manner that a difference between the synthesizing ratio of the present frame and the synthesizing ratio of the frame which is precedent to the present frame with respect to the time is equal to or less than a predetermined value. 
     
     
       7. An apparatus according to  claim 1 , wherein the generating unit calculates the reliability of the motion vector on the basis of at least one of a plurality of correlation values among the frame images and variations of a plurality of motion vectors in a neighboring region of a target pixel. 
     
     
       8. An apparatus according to  claim 1 , wherein the generating unit adjusts the reliability of the motion vector in the whole frame image on the basis of a minimum value among the reliabilities of a plurality of motion vectors in the whole frame image. 
     
     
       9. An apparatus according to  claim 1 , wherein the generating unit excludes the motion vector in which the reliability of the motion vector satisfies a predetermined condition among a plurality of motion vectors. 
     
     
       10. An apparatus according to  claim 3 , wherein when the reliability of the mot on sector is smaller than a second threshold value smaller than the first threshold value, the generating unit generates the frame image with blur by executing a spatial filtering processing to the frame image on the basis of the motion vector. 
     
     
       11. An apparatus according to  claim 1 , wherein the frame image is an image photographed in an exposure time shorter than a time interval between the frame images. 
     
     
       12. An image processing method of an image processing apparatus, comprising the steps of:
 calculating a motion vector between frame images of a moving image; 
 generating a plurality of interpolation frame images corresponding to a time position between frame images which are adjacent with respect to time on the basis of the moving image; and 
 generating a frame image with motion blur acquired by adding a motion blur, by synthesizing the frame image and the plurality of interpolation frame images, wherein 
 the generating step determines a synthesizing ratio of synthesizing the frame image with one of the plurality of interpolation frame images, based on a time span indicating difference between the time position of the frame image and the time position of one of the plurality of interpolation frame images, and 
 the generating step acquires a reliability of the motion vector and time spans between the frame images and the interpolation frame image which are precedent and subsequent with respect to time, and determines the synthesizing ratio on the basis of the reliability of the motion vector and the time spans. 
 
     
     
       13. A non-transitory computer-readable recording medium storing a readable controlling program for operating a computer as:
 a calculating unit configured to calculate a motion vector between frame images of a moving image; 
 an interpolating unit configured to generate a plurality of interpolation frame images corresponding to a time position between frame images which are adjacent with respect to time on the basis of the motion vector; and 
 a generating unit configured to generate a frame image with motion blur acquired by adding a motion blur, by synthesizing the frame image and the plurality of interpolation frame images, wherein 
 the generating unit determines a synthesizing ratio of synthesizing the frame image with one of the plurality of interpolation frame images, based on a time span indicating difference between the time position of the frame image and the time position of one of the plurality of interpolation frame images, and 
 the generating unit acquires a reliability of the motion vector and time spans between the frame images and the interpolation frame image which are precedent and subsequent with respect to time, and determines the synthesizing ratio on the basis of the reliability of the motion vector and the time spans.