Super resolution methods for electro-optical systems

Abstract

There is disclosed in an optical system having a predetermined Numerical aperture which provides a corresponding level of spatial resolution and having detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed, and noisy blurred background data of the same scene, an improved method for increasing the spatial resolution of the imaging data produced by the diffraction limited optical system. The improvement comprises filtering the noisy blurred background data of the same scene to obtain noise suppressed data; applying estimations of point spread functions associated with the noise suppressed data and optical system to estimates of the noise suppressed data to obtain a reconstructed background image (I r (x)); and low pass filtering the noisy blurred scene data containing the object to be reconstructed (D 1 ) and using the reconstructed background image (I r (x)) to eliminate the background data from the image data to obtain a reconstructed image of an object with increased spatial resolution.

What is claimed is:  
     
       1. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved method for increasing the spatial resolution of the imaging data produced by the diffraction limited optical system, the improvement therewith comprising: 
       filtering the noisy blurred background data (D 2 ) of the same scene to obtain noise suppressed data;  
       applying estimations of point spread functions associated with the noise suppressed data and optical system and to estimates of the noise suppressed data to obtain a reconstructed background image (I r (x)); and  
       low pass filtering the noisy blurred scene data containing the object to be reconstructed (D 1 ) and using the reconstructed background image (I r (x)) to eliminate the background data from said image data to obtain a reconstructed image of the object with increased spatial resolution.  
     
     
       2. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved method for increasing the spatial resolution of the imaging data produced by the diffraction limited optical system, the improvement therewith comprising: 
       filtering the noisy blurred background data (D 2 ) of the same scene to obtain noise suppressed data;  
       applying estimations of point spread functions associated with the noise suppressed data and optical system in the noise suppressed data to obtain a reconstructed background image (I r (x)); and  
       low pass filtering the noisy blurred scene data containing the object to be reconstructed (D 1 ) and using the reconstructed background image (I r (x)) to eliminate the background data from said image data to obtain a reconstructed image of the object with increased spatial resolution;  
       wherein the step of filtering to remove noise from D 2  comprises using the modified method of sieves to remove said noise, using the equations:  
       
         
             I ( x )=Σ y   h   0 ( x−y ) D   2 ( y )  
         
       
       
         
             I ( x )=Σ y   h   3 ( x−y ) D   2 ( y )  
         
       
       
         
             I ( x )=Σ y   h   4 ( x−y ) D   2 ( y )  
         
       
       where h 3  and h 4  are two and three pixel wide point spread functions for removing noise by averaging adjacent pixels together, and where h 0  represents the optical system point spread function.  
     
     
       3. The method according to claim  1 , wherein the reconstructed background I r (x) is obtained using the Richardson-Lucy reconstruction technique. 
     
     
       4. The method according to claim  2 , further comprising reconstructing new point spread functions (h 3T , h 4T ) to account for the combined effect of the method of sieves two and three pixel wide point spread functions, and the optical system point spread functions. 
     
     
       5. The method according to claim  4 , wherein the step of reconstructing the background I r (x) further comprises: 
       a) estimating the true background scene data;  
       b) blurring the estimate of the true background scene data;  
       c) dividing on a pixel by pixel basis the noisy scene data (D 3 , D 4 ) by the blurred estimate of the true background scene data to create a new array (T 3 , T 4 );  
       d) correlating said new array with the complete method of sieves and optical system point spread functions and multiplying the result on a pixel by pixel basis with the current estimate of the true background scene data to provide a new estimate of said true background scene Z(x); and  
       e) taking the new estimate of the true background scene to be the reconstructed background scene (I r (x)).  
     
     
       6. The method according to claim  5 , further comprising the step of determining whether a threshold number of iterations has been met and repeating substeps b-e of claim  5  until said threshold is achieved. 
     
     
       7. The method according to claim  5 , wherein the noise suppressed values D 3  and D 4  are used as the first estimate of the true background scene. 
     
     
       8. The method according to claim  5 , wherein the step of correlating the new array to obtain a new estimate of the true background scene further comprises using said new point spread functions, h 3T  and h 4T , where 
       
         
             Z   3 ( x )=Σ y   T   3 ( y ) h   3T ( y−x )  
         
       
         Z   4 ( x )=Σ y   T   4 ( y ) h   4T ( y−x ) 
       
         
             Z ( x )= I   n ( x )[ Z   3 ( x )+ Z   4 ( x )]/2.  
         
       
     
     
       9. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved method for increasing the spatial resolution of the imaging data produced by the diffraction limited optical system, the improvement therewith comprising: 
       filtering the noisy blurred background data (D 2 ) of the same scene to obtain noise suppressed data;  
       applying estimations of point spread functions associated with the noise suppressed data and optical system in the noise suppressed data to obtain a reconstructed background image (I r (x)); and  
       low pass filtering the noisy blurred scene data containing the object to be reconstructed (D 1 ) and using the reconstructed background image (I r (x)) to eliminate the background data from said image data to obtain a reconstructed image of the object with increased spatial resolution;  
       wherein the step of eliminating the background data further comprises:  
       a) separating out a first estimate D 0 (x), in reconstructed object data from the filtered D 1  data;  
       b) replacing reconstructed background scene pixels by estimated reconstructed object pixels at predetermined object positions to obtain an image indicative of the reconstructed background and estimated reconstructed object (S(x));  
       c) blurring the combination of the reconstructed background and estimated reconstructed object to obtain a blurred image I s (x);  
       d) dividing on a pixel basis the filtered D 1  scene data by the blurred combination of the reconstructed background and estimated reconstructed object data to obtain a new array of image data N(x);  
       e) correlating said new array N(x) with the optical system point spread function and multiplying for each pixel specified by the current estimate of the reconstructed object to provide a new estimate of said reconstructed object;  
       f) taking the new estimate of the reconstructed object to be the reconstructed image of the object having increased spatial resolution.  
     
     
       10. The method according to claim  9 , further comprising the step of determining whether a threshold number of iterations has been met and repeating steps b-f of claim  9  until said threshold is achieved. 
     
     
       11. The method according to claim  9 , wherein the step of separating out a first estimate of the reconstructed object from the filter data D 1 , further comprises using binmap values specifying the position in a scene of the object to be resolved in combination with the filtered data (D f (x)) to obtain D 0 (x), where 
       
         
             D   0 ( x )=binmap( x )· D   f ( x ).  
         
       
     
     
       12. The method according to claim  11 , wherein the predetermined object positions for replacing the reconstructed background scene pixels by the estimated reconstructed object pixels S(x) are specified by the binmap values, where 
       
         
             S ( x )= I   r ( x )(binmap( x )−1)+binmap( x ) D   0 ( x ).  
         
       
     
     
       13. The method according to claim  12 , wherein the step of blurring the combination of the reconstructed background and estimated reconstructed object data uses the optical system point spread function, where 
         I   s ( x )=Σ y   h   0 ( x−y ) S ( y ). 
     
     
       14. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved method for increasing the spatial resolution of the imaging data produced by the optical system, the improvement therewith comprising: 
       obtaining filtered data by filtering the noisy blurred background data (D 2 ) of the same scene;  
       obtaining a reconstructed background image (I r (x)) by applying estimations of point spread functions associated with the filtered data and the optical system to the filtered data; and  
       low pass filtering the noisy blurred scene data containing the object to be reconstructed (D 1 ) and eliminating the background data from said image data by using the reconstructed background image (I r (x)), to obtain a reconstructed image of the object with increased spatial resolution.  
     
     
       15. The method according to claim  14  wherein the optical system is diffraction limited. 
     
     
       16. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved apparatus for increasing the spatial resolution of the image data produced by the optical system, the improvement therewith comprising: 
       means for filtering the noisy blurred background data (D 2 ) of the same scene to obtain noise suppressed data:  
       means for applying estimations of point spread functions associated with the noise suppressed data and optical system to the noise suppressed data to obtain a reconstructed background image (I r (X)); and  
       means for low pass filtering the noisy blurred scene data containing the object to be reconstructed and using the reconstructed background image to eliminate the background data from the image data to obtain a reconstructed image of the object with increased spatial resolution.  
     
     
       17. The apparatus of claim  16  wherein said optical system has a numerical aperture and said detector means has at least five detectors spread across the central lobe of the diffraction pattern determined by said aperture. 
     
     
       18. The apparatus of claim  17  wherein the optical system is diffraction limited. 
     
     
       19. The apparatus of claim  16 , wherein the reconstructed background I r (x) is obtained using a means for implementing the Richardson-Lucy reconstruction technique. 
     
     
       20. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved apparatus for increasing the spatial resolution of the image data produced by the optical system, the improvement therewith comprising: 
       means for filtering the noisy blurred background data (D 2 ) of the same scene to obtain noise suppressed data:  
       means for applying estimations of point spread functions associated with the noise suppressed data and optical system to the noise suppressed data to obtain a reconstructed background image (I r (X); and  
       means for low pass filtering the noisy blurred scene data containing the object to be reconstructed and using the reconstructed background image to eliminate the background data from the image data to obtain a reconstructed image of the object with increased spatial resolution;  
       wherein the means for filtering to remove noise from D 2  comprises means for using the modified method of sieves to remove said noise, using the equations:  
       
         
             I ( x )=Σ y   h   0 ( x−y ) D   2 ( y )  
         
       
       
         
             D   3 ( x )=Σ y   h   3 ( x−y ) D   2 ( y )  
         
       
       
         
             D   4 =Σ y   h   4 ( x−y ) D   2 ( y )  
         
       
       where h 3  and h 4  are two and three pixel wide point spread functions for removing noise by averaging adjacent pixels together, and where h 0  represents the optical system point spread function.  
     
     
       21. The apparatus of claim  20 , further comprising means for constructing new point spread functions (h 3T , h 4T ) to account for the combined effect of the method of sieves two and three pixel wide point spread functions, and the optical system point spread functions. 
     
     
       22. The apparatus of claim  21 , wherein said means to obtain the reconstructed background I r (x) further comprises: 
       (a) means for blurring the estimate of the true background scene data;  
       (b) means for dividing on a pixel by pixel basis the noisy scene data (D 3 , D 4 ) by the blurred estimate of the true background scene data to create a new array (T 3 , T 4 );  
       (c) means for correlating said new array with the complete method of sieves and optical system point spread functions and multiplying the result on a pixel by pixel basis with the current estimate of the true background scene data to provide a new estimate of said true background scene Z(x); and  
       (d) means for taking a new estimate of the true background scene to be the reconstructed background scene I r (x).  
     
     
       23. The apparatus of claim  22 , further comprising means for determining whether a threshold number of iterations has been met and repeating the steps of claim  22  until said threshold is achieved. 
     
     
       24. The apparatus of claim  22  wherein the optical system is diffraction limited. 
     
     
       25. The apparatus of claim  24  wherein said optical system has a numerical aperture and said detector means has at least five detectors spread across the central lobe of the diffraction pattern determined by said aperture. 
     
     
       26. The apparatus of claim  22 , wherein the noise suppressed values D 3  and D 4  are used as the first estimate of the true background scene. 
     
     
       27. The apparatus of claim  22 , wherein the means for correlating the new array to obtain a new estimate of the true background scene further comprises means for using said new point spread functions, h 3T  and h 4T , where 
       
         
             Z   3 ( x )=Σ y   T   3 ( y ) h   3T ( y−x )  
         
       
       
         
             Z   4 ( x )=Σ y   T   4 ( y ) h   4T ( y−x )  
         
       
       
         
             Z ( x )= I   n ( x )[ Z   3 ( x )+ Z   4 ( x]/ 2.  
         
       
     
     
       28. In an optical system having a detector means and processor means in which image data is obtained comprising noisy blurred scene data containing an object to be reconstructed (D 1 ), and noisy blurred background data (D 2 ) of the same scene, an improved apparatus for increasing the spatial resolution of the image data produced by the optical system, the improvement therewith comprising: 
       means for filtering the noisy blurred background data (D 2 ) of the same scene to obtain noise suppressed data:  
       means for applying estimations of point spread functions associated with the noise suppressed data and optical system to the noise suppressed data to obtain a reconstructed background image (I r (X); and  
       means for low pass filtering the noisy blurred scene data containing the object to be reconstructed and using the reconstructed background image to eliminate the background data from the image data to obtain a reconstructed image of the object with increased spatial resolution;  
       (a) means for separating out a first estimate, D 0 (x), of reconstructed object data from filtered D 1  data;  
       (b) means for replacing reconstructed background scene pixels by estimated reconstructed object pixels at predetermined object positions to obtain an image indicative of the reconstructed background and estimated reconstructed object (S(x));  
       (c) means for blurring the combination o the reconstructed background and estimated reconstructed object to obtain a blurred image I s (x);  
       (d) means for dividing on a pixel basis the filtered D 1  scene data by the blurred combination of the reconstructed background and estimated reconstructed object data to obtain a new array of image data N(x);  
       (e) means for correlating said new array N(x) with the optical system point spread function and multiplying for each pixel specified by the current estimate of the reconstructed object to provide a new estimate of said reconstructed object; and  
       (f) means for taking the new estimate of the reconstructed object to be the reconstructed image of the object having increased spatial resolution.  
     
     
       29. The apparatus of claim  28 , further comprising means for determining whether a threshold number of iterations has been met and repeating steps b-f of claim  28  until said threshold is achieved. 
     
     
       30. The apparatus of claim  29 , wherein the means for separating out a first estimate of the reconstructed object from the filtered data D 1  further comprises means for using binmap values specifying the position in a scene of the object to be resolved in combination with the filtered data (D f (x)) to obtain D 0 (X), where 
       
         
             D   0 ( x )=binmap( x )· D   f ( x ).  
         
       
     
     
       31. The apparatus of claim  30 , wherein the predetermined object positions for replacing the reconstructed background scene pixels by the estimated reconstructed object pixels S(x) are specified by the binmap values, where 
       
         
             S ( x )= I   r ( x )(binmap( x )−1)+binmap( x )D 0 ( x ).  
         
       
     
     
       32. The apparatus of claim  31 , wherein the means of blurring the combination of the reconstructed background and estimated reconstructed object data uses the optical system point spread function, where 
       
         
             I   s ( x )=Σ y   h   0 ( x−y ) S ( y ).  
         
       
     
     
       33. The apparatus of claim  31  wherein the optical system is diffraction limited. 
     
     
       34. The apparatus of claim  33  wherein said optical system has a numerical aperture and said detector means has at least five detectors spread across the central lobe of the diffraction pattern determined by said aperture.