Method, apparatus, and program for processing an image

Abstract

The present invention relates to a method, apparatus and program for processing an image, capable of producing a sharp image. A digital camera 1 includes an imaging device 4 formed of a single-plane sensor that provides one color signal for each pixel. A plurality of images are captured at a high shutter speed by the imaging device 4 and stored in a frame memory 22 of a signal processor 7 . The signal processor 7 detects a positional relationship among the plurality of images stored in the frame memory 22 and estimates a correct image with three color signals at each pixel, based on the detected positional relationship. The present invention is applicable, for example, to a digital camera.

The invention claimed is: 
     
       1. An image processing method for estimating an output image from a plurality of input images, the method comprising:
 detecting a positional relationship among the plurality of input images captured by an image capture device, each pixel of the image having n pixel values; 
 estimating the output image by identifying observed pixel components, from the plurality of input images, for each pixel position of the output image; 
 producing the output image based on the identified observed pixel components, the output image having (n+1) or more pixel values for each pixel of the output image; 
 determining a corrected position of each pixel based on the detected positional relationship; and 
 identifying a pixel value of a pixel of the input images whose corrected position is located in the vicinity of each pixel position of the output image based on the corrected position and the position of the corresponding pixel of the output image. 
 
     
     
       2. An image processing method according to  claim 1 , wherein estimating further comprises:
 determining a corrected position of each pixel based on the detected positional relationship; and 
 identifying a pixel value of a pixel of the input images whose corrected position is located in the vicinity of each pixel position of the output image based on an error depending on a factor of gaining up. 
 
     
     
       3. An image processing method according to  claim 1 , wherein estimating further comprises:
 determining a corrected position of each pixel based on the detected positional relationship; and 
 estimating the output image by determining a weighted sum of pixel values of the input images whose corrected position is located in the vicinity of each pixel position of the output image. 
 
     
     
       4. An image processing method according to  claim 3 , wherein the correct pixel value of a pixel of interest is estimated by determining a weighted sum of pixel values of the input images whose corrected position is located in the vicinity of each pixel position of the output image, wherein weights of the weighted sum being determined by using a function having a lowpass-filter characteristic with respect to a distance between the position of the pixel of interest in the output image and the corrected position. 
     
     
       5. An image processing method according to  claim 4 , wherein the correct pixel value of the pixel of interest is estimated by selectively performing:
 a first calculation process of determining the weighted sum of pixel values of the input images whose corrected position is located in the vicinity of the position of the pixel of interest; and 
 a second calculation process of determining the weighted sum of pixel values of the input images whose corrected position is located in the vicinity of the position of a pixel of the output image located adjacent to the pixel of interest. 
 
     
     
       6. An image processing method according to  claim 5 , wherein the second calculation process is selected when the sum of weights in the first calculation process is equal or nearly equal to 0. 
     
     
       7. An image processing method according to  claim 4 , wherein the correct pixel value of the pixel of
 interest is estimated by selectively performing:
 a first calculation process of determining the weighted sum of pixel values of the input images whose corrected position is located in a first vicinity of the position of the pixel of interest; and 
 a second calculation process of determining the weighted sum of pixel values of the input images whose corrected position is located in a second vicinity of the position of the pixel of interest, the second vicinity being greater than the first vicinity. 
 
 
     
     
       8. An image processing method according to  claim 7 , wherein the second calculation process is selected when the sum of weights in the first calculation process is equal or nearly equal to 0. 
     
     
       9. An image processing method according to  claim 4 , wherein the correct pixel value of the pixel of interest is estimated by selectively performing:
 a first calculation process of determining the sum of pixel values weighted by a first weighting factor of the input images whose corrected position is located in the vicinity of the position of the pixel of interest; and 
 a second calculation process of determining the sum of pixel values weighted by a second weighting factor of the input images whose corrected position is located in the vicinity of the position of the pixel of interest. 
 
     
     
       10. An image processing method according to  claim 9 , wherein the second calculation process is selected when the sum of weights in the first calculation process is equal or nearly equal to 0. 
     
     
       11. An image processing method according to  claim 1 , wherein the positional relationship among the plurality of input images is detected with reference to an input image captured at the exact or nearly exact center of a time during which the plurality of input images were captured. 
     
     
       12. An image processing apparatus for estimating an output image from a plurality of input images, comprising:
 means for capturing the plurality of input images having n pixel values for each pixel; 
 means for detecting a positional relationship among the plurality of input images; 
 means for estimating the output image by identifying observed pixel components, from the plurality of input images, for each pixel position of the output image; 
 means for producing the output image based on the identified observed pixel components, the output image having (n+1) or more pixel values for each pixel of the output image; 
 means for determining a corrected position of each pixel based on the detected positional relationship; and 
 means for identifying a pixel value of a pixel of the input images hose corrected position is located in the vicinity of each pixel position of the output image based on the corrected position and the position of the corresponding pixel of the output image. 
 
     
     
       13. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a process of estimating an output image from a plurality of input images, the process comprising the steps of:
 detecting a positional relationship among the plurality of input images captured by image capture means, each pixel of the image having n pixel values; and 
 estimating the output image by identifying observed pixel components, from the plurality of input images, for each pixel position of the output image; 
 producing the output image based on the identified observed pixel components, the output image having (n+1) or more pixel values for each pixel of the output image; 
 determining a corrected position of each pixel based on the detected positional relationship; and 
 Identifying a pixel value of a pixel of the input images whose corrected position is located in the vicinity of each pixel position of the output image based on the corrected position and the position of the corresponding pixel of the output image.