Method and apparatus for processing images

Abstract

A method and apparatus for accurately computing parallax information as captured by imagery of a scene. The method computes the parallax information of each point in an image by computing the parallax within windows that are offset with respect to the point for which the parallax is being computed. Additionally, parallax computations are performed over multiple frames of imagery to ensure accuracy of the parallax computation and to facilitate correction of occluded imagery.

What is claimed is:  
     
         1 . A method for computing image flow information from a plurality of images comprising: 
 aligning a plurality of images to form an aligned image;    defining a plurality of windows, where each of said windows circumscribe an image region containing a point within said aligned image;    offsetting at least one of said windows from said point;    computing a flow estimation within each of said windows;    identifying the flow estimation having the lowest error; and    deeming said flow estimation associated with said lowest error as said flow information for said point.    
     
     
         2 . The method of  claim 1  wherein said flow information is constrained to produce parallax information.  
     
     
         3 . The method of  claim 1  wherein one of said windows is centered upon said point.  
     
     
         4 . The method of  claim 1  wherein said windows have different sizes.  
     
     
         5 . The method of  claim 1  wherein said plurality of images comprises a plurality of images and said windows are defined in said aligned images.  
     
     
         6 . The method of  claim 1  wherein said plurality of images are tiled and pairs of tiles form said plurality of images.  
     
     
         7 . The method of  claim 1  wherein each said plurality of images are imaged contemporaneously.  
     
     
         8 . The method of  claim 1  further comprising the steps of: 
 computing a flow estimate for each of said aligned images;  
 identifying a flow estimate having a lowest error;  
 identifying, in response to said flow estimate, errant information in first aligned image; and  
 repairing said errant information in said first aligned image with information from at least one other aligned image.  
 
     
     
         9 . The method of  claim 1  wherein said flow estimate is constrained to form a parallax estimate.  
     
     
         10 . The method of  claim 1  wherein said flow estimation is corrected.  
     
     
         11 . A method for enhancing regions within a plurality of images comprising: 
 aligning a plurality of images to form a plurality of aligned images;    computing a flow estimation for each of said aligned images;    identifying flow estimation having the lowest error;    identifying, in response to said flow estimation, regions in a first aligned image; and    enhancing said regions in said first aligned image with information from at least one other aligned image.    
     
     
         12 . The method of  claim 11  wherein said flow estimation is constrained to form a parallax estimation.  
     
     
         13 . The method of  claim 11  wherein computing step further comprises: 
 computing an epipolar constraint for each of said aligned images; and  
 computing a flow field representing image changes from aligned image to aligned image.  
 
     
     
         14 . The method of  claim 11  wherein said computing step further comprises the step of: 
 computing a temporal constraint.  
 
     
     
         15 . The method of  claim 11  further comprising the steps of: 
 computing a flow estimation for a second aligned image; and  
 using the flow estimation from said second aligned image to correct a flow estimation for said first aligned image.  
 
     
     
         16 . The method of  claim 11  wherein said region is caused by noise and said enhancing said step reduces said noise.  
     
     
         17 . A method of determining image flow comprising the steps of: 
 aligning a plurality of pairs of images in said plurality of images to form a plurality of aligned images;    computing a flow estimation for each of said aligned images to produce a plurality of flow estimates;    weighting the flow estimates; and    compositing an image by combining the weighted flow estimates.    
     
     
         18 . The method of  claim 17  wherein said flow estimation is constrained to produce parallax estimation.  
     
     
         19 . The method of  claim 17  wherein said flow estimation is corrected.  
     
     
         20 . The method of  claim 17  wherein the weighting step weights flow estimates for images over time.  
     
     
         21 . Apparatus for generating a enhancing an image comprising: 
 a first imaging device for producing first images at a first resolution;    a second imaging device for producing second images at a second resolution;    an image processor coupled to said first and said second imaging devices, for using said second image to enhance said first image.    
     
     
         22 . The apparatus of  claim 21  wherein said image processor comprises: 
 an image flow generator.  
 
     
     
         23 . The apparatus of  claim 22  wherein said image flow generator is a parallax computer.  
     
     
         24 . The apparatus of  claim 23  wherein said parallax computer further comprises one or more augmentation modules selected from the group consisting of: 
 a module for dividing the images into tiles, a depth correlator, a module which performs Just Noticeable Differences, a correspondence checker, and a blank area avoidance module.