Methods and systems for synthesizing pickup images

Abstract

In an image pickup apparatus, a level compensator compensates the level of an image xL (i, j) sensed by an image sensor by exposure for a long time on the basis of the exposure of the image to produce a compensated image xL′ (i, j), while another level compensator compensates the level of an image xS (i, j) sensed by the image sensor by exposure for a short time on the basis of the exposure of the image to produce a compensated image xS′ (i, j). The image pickup apparatus also includes an image synthesizer to synthesize these compensates images xL′ (i, j) and xS′ (i, j) to produce a single synthetic image x (i, j). A dynamic range compressor also included in the image pickup apparatus compresses the synthetic image x (i, j) to produce a compressed image y (i, j) for delivery to outside.

1. An image pickup apparatus adapted to produce a single image from a plurality of images acquired by sensing an object under different exposure conditions, comprising:
 means for sensing the object under different exposure conditions to acquire the plurality of images; 
 means for calculating a pixel value for each of the plurality of images; 
 means for multiplying the pixel value by a factor set based on the respective exposure condition for the image to calculate a positive value compensation amount; 
 means for synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; 
 means for compressing the dynamic range of the synthesized image to an extent depending upon the performance of an output destination of the image pickup apparatus to produce a compressed image; 
 first means for compensating the level of the plurality of images on the basis of the respective exposure conditions under which the plurality of images have been sensed by subtracting a positive compensation value from each of the plurality of images, to produce a plurality of compensated images; 
 means for calculating the positive compensation value by multiplying a level average of each of the plurality of images by a respective coefficient for the image; and 
 means for selecting the coefficient for each of the plurality of images based on the respective exposure condition of that image. 
 
     
     
       2. The image pickup apparatus of  claim 1 , further comprising second means for compensating the compressed image by subtracting a positive value from each pixel level of the compressed image to produce a compensated compressed image. 
     
     
       3. The image pickup apparatus of  claim 2 , the second means for compensating further comprising:
 means for low-pass filtering a signal of the compressed image; 
 means for normalizing the filtered signal; and 
 means for multiplying the normalized signal by a predetermined factor to calculate the positive value. 
 
     
     
       4. The image pickup apparatus of  claim 3 , the second means for compensating further comprising:
 means for time-smoothing the positive value; and 
 means for subtracting the time-smoothed positive value from each pixel level of the compressed image. 
 
     
     
       5. The image pickup apparatus of  claim 1 , wherein:
 each of the plurality of images is an image signal in which a frequency-modulated color signal is superposed on a brightness signal; 
 the image pickup apparatus further comprises means for separating the image signal into the brightness and color signals; 
 the separated brightness signal and the separated color signal are synthesized by the means for synthesizing to produce a synthetic brightness signal and a synthetic color signal, respectively; 
 the synthetic brightness and color signals are compressed by the means for compressing to produce a compressed brightness signal and a compressed color signal, respectively; and 
 the compressed brightness and color signals are compensated by the means for second compensating means to produce a compensated compressed brightness signal and a compensated compressed color signal, respectively. 
 
     
     
       6. The image pickup apparatus of  claim 5 , wherein the second means for compensating comprises:
 means for calculating a compensation amount based on the compressed brightness signal; 
 means for compensating the level of the compressed brightness signal with the compensation amount to produce a compensated compressed brightness signal; and 
 means for producing the compensated compressed color signal based on the compensated compressed brightness signal. 
 
     
     
       7. The image pickup apparatus of  claim 5 , further comprising means for mixing the compensated compressed brightness signal and the compensated compressed color signal. 
     
     
       8. An image pickup apparatus adapted to synthesize a plurality of images acquired by sensing an object under different exposure conditions to produce a single image, the apparatus comprising:
 means for sensing the object under different exposure conditions to acquire the plurality of images; 
 means for calculating a pixel value for each of the plurality of images; 
 means for multiplying the pixel value by a factor set based on the respective exposure condition for the image to calculate a positive value compensation amount for the image; 
 first means for compensating the levels of the plurality of images on the basis of the respective exposure conditions under which the plurality of images have been sensed by subtracting a positive compensation value from each of the plurality of images; 
 means for calculating the positive compensation value by multiplying a level average of each of the plurality of images by a respective coefficient for the image; and 
 means for selecting the coefficient for each of the plurality of images based on the exposure condition of that image; 
 means for synthesizing the plurality of compensated images to produce a single synthesized image having a wide dynamic range; 
 means for compressing the dynamic range of the synthesized image to an extent depending upon the performance of an output destination of the image pick up apparatus to produce a compressed image; and 
 second means for compensating the level of the compressed image to provide a compensated compressed image. 
 
     
     
       9. The apparatus as set forth in  claim 8 , wherein the second compensating means comprises means for subtracting a predetermined positive value from each pixel level of the compressed image to produce the compensated compressed image. 
     
     
       10. The apparatus as set forth in  claim 9 , wherein the second compensating means further comprises:
 means for filtering a signal of the compressed image; 
 means for normalizing the filtered signal; and 
 means for multiplying the normalized signal by a predetermined factor to calculate a positive value. 
 
     
     
       11. The apparatus as set forth in  claim 10 , wherein the second compensating means further comprises;
 means for time-smoothing the positive value; and 
 means for subtracting the time-smoothed positive value from each pixel level of the compressed image. 
 
     
     
       12. The apparatus as set forth in  claim 8 , wherein:
 the means for sensing the object is adapted to output an image signal in which a frequency-modulated color signal is superposed on a brightness signal, 
 the apparatus further comprises means for separating the image signal into the brightness and color signals; 
 the means for synthesizing synthesizes the separated brightness and color signals to produce a synthesized brightness signal and a synthesized color signal, respectively; 
 the means for compressing compresses the synthesized brightness and color signals to produce a compressed brightness signal and a compressed color signal, respectively; and 
 the second means for compensating compensates the compressed brightness and color signals to produce a compensated compressed brightness signal and a compensated compressed color signal, respectively. 
 
     
     
       13. The apparatus as set forth in  claim 12 , wherein the level second compensating means further comprises:
 means for calculating a compensation amount based on the separated compressed brightness signal 
 third means for compensating the level of the compressed brightness signal with the compensation amount to produce a compensated compressed brightness signal, and 
 means for producing the compensated compressed color signal based on the compensated compressed brightness signal. 
 
     
     
       14. The apparatus as set forth in  claim 12 , further comprising means for mixing the compensated compressed brightness and color signals. 
     
     
       15. An image processing method of synthesizing a plurality of input images acquired by sensing an object under different exposure conditions to produce a single image, the method comprising the steps of:
 receiving a plurality of images acquired by sensing an object under different exposure conditions; 
 providing a plurality of compensated images by subtracting a positive compensation value from each of the plurality of images; 
 wherein the positive compensation value is calculated by multiplying a level average of each of the plurality of images by a coefficient based on the respective exposure condition for the image; 
 wherein the coefficient is selected for each of the plurality of images based on the exposure condition of that image; 
 calculating a pixel value for each of the plurality of images; 
 multiplying the pixel value by a factor set based on the respective exposure condition for the image to calculate the positive compensation value; 
 synthesizing the plurality of input images to produce a single synthetic image having a wide dynamic range; 
 compressing the dynamic range of the synthetic image to an extent depending upon the performance of an output destination to produce a compressed image; and 
 compensating the level of the compressed image to provide a compensated compressed image. 
 
     
     
       16. The method as set forth in  claim 15 , wherein compensating the level of the compressed image further comprises the step of subtracting a predetermined positive value from each pixel level of the compressed image to produce the compensated compressed image. 
     
     
       17. The method as set forth in  claim 16 , wherein calculating the positive compensation value further comprises the steps of:
 low-pass filtering the signal of the compressed image; 
 normalizing the filtered signal; and 
 multiplying the normalized signal by a predetermined factor to calculate the positive value. 
 
     
     
       18. The method as set forth in  claim 17 , further comprising the steps of:
 time-smoothing the positive value obtained; and 
 wherein providing a plurality of compensated images includes subtracting the time-smoothed positive value from each pixel level of the compressed image. 
 
     
     
       19. The method as set forth in  claim 15 , wherein:
 each of the plurality of input images is an image signal in which a frequency-modulated color signal is superposed on a brightness signal, and 
 the method further comprises:
 separating the image signal into the brightness and color signals; and 
 synthesizing the separated brightness signal and the separated color signal to produce a synthesized brightness signal and a synthesized color signal, respectively; 
 compressing the synthesized brightness and color signals to produce a compressed brightness signal and a compressed color signal, respectively; and 
 compensating the compressed brightness and color signals to produce a compensated compressed brightness signal and a compensated compressed color signal, respectively. 
 
 
     
     
       20. The method as set forth in  claim 19 , wherein compensating the level of the compressed image includes:
 calculating a compensation amount based on the separated compressed brightness signal; 
 compensating the level of the compressed brightness signal to produce a compensated compressed brightness signal; and 
 producing the compensated compressed color signal based on the compensated compressed brightness signal. 
 
     
     
       21. The method as set forth in  claim 19 , further comprising the step of mixing the compensated compressed brightness and color signals. 
     
     
       22. An image processing apparatus adapted to synthesize a plurality of input images acquired by sensing an object under different exposure conditions to produce a single image, the apparatus comprising:
 means for receiving a plurality of images acquired by sensing an object under different exposure conditions; 
 first means for compensating the levels of the plurality of input images on the basis of the respective exposure conditions under which the plurality of images have been sensed; 
 means for providing a plurality of compensated images by subtracting a positive compensation from each of the plurality of images; 
 means for calculating said positive compensation value by multiplying a level average of each of the plurality of images by a coefficient based on the corresponding exposure condition for the image; 
 means for selecting said coefficient for each of the plurality of images based on the respective exposure condition of that image; 
 means for calculating a pixel value for each of the plurality of images; 
 means for multiplying the pixel value by a factor set based on the respective exposure condition for the image to calculate the positive compensation value; 
 means for synthesizing the plurality of images to produce a single synthesized image having a wide dynamic range; 
 means for compressing the dynamic range of the synthesized image to an extent depending upon the performance of an output destination of the image processing apparatus to produce a compressed image; and 
 second means for compensating the level of the compressed image to provide a compensated compressed image. 
 
     
     
       23. The apparatus as set forth in  claim 22 , wherein the second compensating means comprises means for subtracting a positive value from each pixel level of the compressed image to produce the compensated compressed image. 
     
     
       24. The apparatus as set forth in  claim 23 , wherein the second compensating means further comprises:
 means for filtering a signal of the compressed image; 
 means for normalizing the filtered signal; and 
 means for multiplying the normalized signal by a predetermined factor to calculate the positive value. 
 
     
     
       25. The apparatus as set forth in  claim 24 , wherein the second compensating means further comprises:
 means for time-smoothing the positive value; and 
 means for subtracting the time-smoothed positive value from each pixel level of the compressed image. 
 
     
     
       26. The apparatus as set forth in  claim 22 , further comprising means for separating the image signal into the brightness and color signals;
 wherein the signal synthesizing means synthesizes the separated brightness signal and the separated color signal to produce a synthesized brightness signal and a synthesized color signal, respectively; 
 the means for compressing compresses the synthesized brightness and color signals to produce a compressed brightness signal and a compressed color signal, respectively; and 
 the second compensating means compensates the compressed brightness and color signals to produce a compensated compressed brightness signal and a compensated compressed color signal, respectively. 
 
     
     
       27. The apparatus as set forth in  claim 26 , wherein the second compensating means comprises:
 means for calculating a compensation amount based on the compressed brightness signal separated by the signal separating means; 
 means for compensating the level of the compressed brightness signal with the compensation amount to produce a compensated compressed brightness signal; and 
 means for producing the compensated compressed color signal based on the compensated compressed brightness signal. 
 
     
     
       28. The apparatus as set forth in  claim 26 , further comprising means for mixing the compensated compressed brightness and color signals.