Image processing device, image processing method, and program

Abstract

Image processing methods and apparatus are described. The image processing method comprises receiving input of a visible-ray image and an infrared-ray image obtained by photographing a same subject, estimating, based on the visible-ray image, the infrared-ray image and motion information, a blur estimate associated with the visible-ray image, and generating, based on the estimated blur estimate, a corrected visible-ray image.

1 - 19 . (canceled) 
     
     
         20 . An image processing device, comprising:
 image processing circuitry configured to:   receive input of a first image and a second image obtained by photographing a same subject, wherein the first image is a visible-ray image and the second image is an image captured by detecting at least far-infrared light from the object;   estimate, based on the first image, the second image, and motion information, a blur estimate associated with the first image; and   generate, based on the blur estimate, a corrected visible-ray image.   
     
     
         21 . The image processing device of  claim 20 , wherein the second image is an image captured by detecting far-infrared ray light from the object. 
     
     
         22 . The image processing device of  claim 20 , further comprising estimating, based on the first image and the second image, an image-based blur estimate. 
     
     
         23 . The image processing device of  claim 22 , wherein estimating the image-based blur estimate comprises:
 applying each of a plurality of filters having different blurring characteristics to the second image to produce a plurality of blurred second images;   comparing the first image to the plurality of blurred second images; and   selecting a filter from among the plurality of filters that produced the blurred second image having blurring most similar to the first image.   
     
     
         24 . The image processing device of  claim 23 , wherein the different blurring characteristics correspond to different point-spread functions. 
     
     
         25 . The image processing device of  claim 24 , wherein comparing the first image to the plurality of blurred second images comprises:
 calculating correlation values between the first image and each of the plurality of blurred second images, and   wherein selecting a filter from among the plurality of filters comprises selecting the filter that produced the blurred second image having a highest correlation value from among the calculated correlation values.   
     
     
         26 . The image processing device of  claim 20 , wherein the image processing circuitry is further configured to:
 estimate, based on the motion information, a motion-based blur estimate.   
     
     
         27 . The image processing device of  claim 22 , wherein estimating the motion-based blur estimate comprises:
 determining, based on the motion information, a direction and magnitude of blur in the first image.   
     
     
         28 . An image processing method that is performed in an image processing device, the image processing method comprising:
 receiving input of a first image and a second image obtained by photographing a same subject, wherein the first image is a visible-ray image and the second image is an image captured by detecting at least far-infrared light from the object;   estimating, based on the first image, the second image, and motion information, a blur estimate associated with the first image; and   generating, based on the blur estimate, a corrected visible-ray image.   
     
     
         29 . The image processing method of  claim 28 , wherein the second image is an image captured by detecting far-infrared ray light from the object. 
     
     
         30 . The image processing method of  claim 28 , further comprising estimating, based on the first image and the second image, an image-based blur estimate. 
     
     
         31 . The image processing method of  claim 30 , wherein estimating the image-based blur estimate comprises:
 applying each of a plurality of filters having different blurring characteristics to the second image to produce a plurality of blurred second images;   comparing the first image to the plurality of blurred second images; and   selecting a filter from among the plurality of filters that produced the blurred second image having blurring most similar to the first image.   
     
     
         32 . The image processing method of  claim 31 , wherein the different blurring characteristics correspond to different point-spread functions. 
     
     
         33 . The image processing method of  claim 32 , wherein comparing the first image to the plurality of blurred second images comprises:
 calculating correlation values between the first image and each of the plurality of blurred second images, and   wherein selecting a filter from among the plurality of filters comprises selecting the filter that produced the blurred second image having a highest correlation value from among the calculated correlation values.   
     
     
         34 . The image processing method of  claim 33 , further comprising:
 estimating, based on the motion information, a motion-based blur estimate.   
     
     
         35 . The image processing method of  claim 34 , wherein estimating the motion-based blur estimate comprises:
 determining, based on the motion information, a direction and magnitude of blur in the first image.   
     
     
         36 . A non-transitory computer readable medium encoded with a plurality of instructions that, when executed by image processing circuitry of an image processing device, perform an image processing method, the image processing method comprising:
 receiving input of a first image and a second image obtained by photographing a same subject, wherein the first image is a visible-ray image and the second image is an image captured by detecting at least far-infrared light from the object;   estimating, based on the first image, the second image, and motion information, a blur estimate associated with the first image; and   generating, based on the blur estimate, a corrected visible-ray image.   
     
     
         37 . The non-transitory computer readable medium of  claim 36 , wherein the second image is an image captured by detecting far-infrared ray light from the object. 
     
     
         38 . The non-transitory computer readable medium of  claim 36 , further comprising estimating, based on the first image and the second image, an image-based blur estimate. 
     
     
         39 . The non-transitory computer readable medium of  claim 38 , wherein estimating the image-based blur estimate comprises:
 applying each of a plurality of filters having different blurring characteristics to the second image to produce a plurality of blurred second images;   comparing the first image to the plurality of blurred second images; and   selecting a filter from among the plurality of filters that produced the blurred second image having blurring most similar to the first image.   
     
     
         40 . The non-transitory computer readable medium of  claim 39 , wherein the different blurring characteristics correspond to different point-spread functions. 
     
     
         41 . The non-transitory computer readable medium of  claim 40 , wherein comparing the first image to the plurality of blurred second images comprises:
 calculating correlation values between the first image and each of the plurality of blurred second images, and   wherein selecting a filter from among the plurality of filters comprises selecting the filter that produced the blurred second image having a highest correlation value from among the calculated correlation values.   
     
     
         42 . The non-transitory computer readable medium of  claim 41 , further comprising:
 estimating, based on the motion information, a motion-based blur estimate.   
     
     
         43 . The non-transitory computer readable medium of  claim 42 , wherein estimating the motion-based blur estimate comprises:
 determining, based on the motion information, a direction and magnitude of blur in the first image.