Image processing apparatus, image processing method, and program

Abstract

An apparatus and a method capable of effectively eliminating flicker are provided. Flicker which occurs in a captured image captured under a lighting environment such as a fluorescent light having a luminance change is effectively eliminated or reduced. A short exposure image and a long exposure image, which are images captured with at least two different exposure periods, are input, a short exposure image row profile having integrated values of a row-by-row signal quantity of the short exposure image and a reference profile having integrated values of a row-by-row signal quantity of the long exposure image are generated, a flicker correction waveform is generated by a division process on the short exposure image row profile by the reference profile or the like, and a corrected image is generated whose flicker components have been eliminated by a process of multiplying pixel values of each of the rows of the short exposure image by a row-by-row coefficient defined by the flicker correction waveform.

1 . An image processing apparatus comprising:
 a signal processing unit configured to receive a short exposure image and along exposure image, which are images captured with at least two different exposure periods, and perform a flicker correction process of eliminating a flicker component contained in the short exposure image, wherein   the signal processing unit
 generates 
 a short exposure image row profile having integrated values of a row-by-row signal quantity of the short exposure image, and 
 a reference profile having integrated values of a row-by-row signal quantity of the long exposure image, 
 generates a flicker correction waveform by data processing including a division process on the short exposure image row profile by the reference profile, and 
 generates a corrected image whose flicker component has been eliminated by a process of multiplying pixel values of each of rows of the short exposure image by a row-by-row coefficient corresponding to the flicker correction waveform. 
   
     
     
         2 . The image processing apparatus according to  claim 1 , wherein
 the long exposure image is an image captured using a setting of the exposure period which is approximately equal to an integral multiple of a luminance change cycle of an image capture environment.   
     
     
         3 . The image processing apparatus according to  claim 1 ,
 wherein   in a generation process of the flicker correction waveform,   the signal processing unit generates the flicker correction waveform by performing a sinusoidal approximation process on flicker component data obtained by the division process on the short exposure image row profile by the reference profile, or on phase-inverted data with respect to the flicker component data.   
     
     
         4 . The image processing apparatus according to  claim 1 , wherein
 in a generation process of the flicker correction waveform,   the signal processing unit generates the flicker correction waveform by performing a spline interpolation process on flicker component data obtained by the division process on the short exposure image row profile by the reference profile, or on phase-inverted data with respect to the flicker component data.   
     
     
         5 . The image processing apparatus according to  claim 1 , wherein
 in a generation process of the flicker correction waveform,   the signal processing unit generates the flicker correction waveform by performing a linear interpolation process on flicker component data obtained by the division process on the short exposure image row profile by the reference profile, or on phase-inverted data with respect to the flicker component data.   
     
     
         6 . The image processing apparatus according to  claim 1 , wherein
 in a generation process of the flicker correction waveform,   the signal processing unit performs discrete Fourier transform on flicker component data obtained by the division process on the short exposure image row profile by the reference profile, or on phase-inverted data with respect to the flicker component data, and reclassifies second- and higher-order components as a lower order.   
     
     
         7 . The image processing apparatus according to  claim 1 , wherein
 in a generation process of the corrected image whose flicker component has been eliminated,   the signal processing unit performs the process of multiplying pixel values of each of rows of the short exposure image by a row-by-row coefficient corresponding to the flicker correction waveform after phase matching for aligning a peak location of a row profile of an image to be corrected with a peak location of the flicker correction waveform.   
     
     
         8 . The image processing apparatus according to  claim 1 , wherein
 the signal processing unit determines whether each image contains a moving subject or not, and performs a generation process of the flicker correction waveform by applying an image determined to contain no moving subjects.   
     
     
         9 . The image processing apparatus according to  claim 1 , wherein
 the signal processing unit determines whether or not a short exposure image to be corrected contains a moving subject, and performs image correction applying the flicker correction waveform to an image determined to contain no moving subjects.   
     
     
         10 . The image processing apparatus according to  claim 1 , wherein
 the signal processing unit determines whether or not a short exposure image to be corrected contains a moving subject, and with respect to an image determined to contain a moving subject, modifies the flicker correction waveform taking into account a phase variation in a row profile due to an effect of the moving subject, and performs image correction applying a modified flicker correction waveform.   
     
     
         11 . The image processing apparatus according to  claim 1 , wherein
 the signal processing unit determines whether or not a short exposure image to be corrected contains a moving subject, and with respect to an image determined to contain a moving subject, clips the flicker correction waveform so as to align in phase with the image containing a moving subject based on an inter-frame phase mismatch calculated based on a still image, and performs image correction applying a clipped flicker correction waveform.   
     
     
         12 . An image processing method performed in an image processing apparatus, comprising:
 performing a signal processing step in which a signal processing unit receives a short exposure image and a long exposure image, which are images captured with at least two different exposure periods, and performs a flicker correction process of eliminating a flicker component contained in the short exposure image, wherein   the signal processing step includes performing
 a generation process of a short exposure image row profile having integrated values of a row-by-row signal quantity of the short exposure image, and a reference profile having integrated values of a row-by-row signal quantity of the long exposure image, 
 a generation process of a flicker correction waveform by data processing including a division process on the short exposure image row profile by the reference profile, and 
 a generation process of a corrected image whose flicker component has been eliminated by a process of multiplying pixel values of each of rows of the short exposure image by a row-by-row coefficient corresponding to the flicker correction waveform. 
   
     
     
         13 . A program for implementing a flicker correction process of eliminating a flicker component contained in an image in an image processing apparatus, wherein
 the program   causes a signal processing unit to receive a short exposure image and a long exposure image, which are images captured with at least two different exposure periods, and   instructs the signal processing unit to perform
 a generation process of a short exposure image row profile having integrated values of a row-by-row signal quantity of the short exposure image, and a reference profile having integrated values of a row-by-row signal quantity of the long exposure image, 
 a generation process of a flicker correction waveform by data processing including a division process on the short exposure image row profile by the reference profile, and 
 a generation process of a corrected image whose flicker component has been eliminated by a process of multiplying pixel values of each of rows of the short exposure image by a row-by-row coefficient corresponding to the flicker correction waveform.