Method of matching stereo images and method of measuring disparity between these items

Abstract

In the image pickup phase (A), right and left images are taken in through two image-pickup devices (S101, S102). Then, in the next feature extraction phase (B), right and left images are respectively subjected to feature extraction (S103, S104). Thereafter, in the succeeding matching phase (C), the extracted features of right and left images are compared to check how they match with each other (step S105). More specifically, in the matching phase (C), a one-dimensional window is set, this one-dimensional window is shifted along the left image in accordance with a predetermined scanning rule so as to successively set overlapped one-dimensional windows, and a matching operation is performed by comparing the image features within one window and corresponding image features on the right image. Subsequently, in the disparity determination phase (D), the left image is dissected or divided into plural blocks each having a predetermined size, a histogram in each block is created from disparities obtained by the matching operation based on one-dimensional windows involving pixels of a concerned block, and a specific disparity just corresponding to the peak of thus obtained histogram is identified as a valid disparity representing the concerned block (S106).

What is claimed is: 
     
       1. A method of matching stereo images, comprising the steps of: inputting first and second images IL and IR;   developing said images IL and IR into a plurality of frequency component images FL1, FL2, FL3 . . . , FLk, FLk+1, . . . , FLn and a plurality of frequency component images FR1, FR2, FR3, . . . , FRk, FRk+1, . . . , FRn, respectively;   applying a secondary differential processing to each of said frequency component images;   converting each frequency component image, after being applied the secondary differential processing, into ternary values pixel by pixel, thereby obtaining ternary-valued frequency component images TL1, TL2, TL3, . . . , TLk, TLk+1, . . . , TLn and ternary-valued frequency component images TR1, TR2, TR3 . . . TRk, TRk+1, . . . , TRn; and   performing a matching operation between said first and second images based on said ternary-valued frequency component images,   wherein pixels in a one-dimensional window of the ternary-valued frequency component image TLk of said first image IL are compared in a one-to-one manner with pixels in a designated region of the ternary-valued frequency component image TRk of said second image IR, when said ternary-valued frequency component images TLk and TRk are identical in their frequency components,   an evaluation raesult "P" is obtained when corresponding two pixels are both "p" or "m", while an evaluation result "Z" is obtained when the corresponding two pixels are both "z", and   a similarity between two ternary-valued frequency component images TLk and TRk is evaluated by using the following equation:   Eall=Σβk(PN)k+Σγk(ZN)k     where PN represents a total number of pixels having the evaluation result "P", ZN represents a total number of pixels having the evaluation result "Z", and βk and γk represent weighing factors.     
     
     
       2. The method defined by claim 1, wherein said first image IL is designated as a reference image for the matching operation, a one-dimensional window capable of encompassing N pixels therein is set on the ternary-valued frequency component image of said first image IL, and a matching region having the same ternary-value pattern as said N pixels in said one-dimensional window is searched from the ternary-valued frequency component image of said second image IR. 
     
     
       3. The method defined by claim 1, wherein one of said first and second images IL and IR is designated as a reference image for the matching operation, a plurality of one-dimensional windows are set on the entire surface of said ternary-valued frequency component image of said reference image through a scanning operation along an epipolar line, so that said one-dimensional windows are successively overlapped at the same intervals of N/2 when each of said one-dimensional windows has a size equivalent to N pixels, and said matching operation is carried out with respect to each of said one-dimensional windows. 
     
     
       4. The method defined by claim 1, wherein a matching result in said matching operation is validated only when Σβk(PN)k is larger than a predetermined threshold TH3(>0).   
     
     
       5. A method of matching stereo images, comprising the steps of: inputting first and second images IL and IR;   developing said images IL and IR into a plurality of frequency component images FL1, FL2, FL3, . . . , FLk, FLk+1, . . . , FLn and a plurality of frequency component images FR1, FR2, FR3, . . . , FRk, FRk+1, . . . , FRn, respectively;   applying a secondary differential processing to each of said frequency component images;   converting each frequency component image, after being applied the secondary differential processing, into ternary values pixel by pixel by using a positive threshold TH1(>0) and a negative threshold TH2(<0) in such a manner that a pixel larger than TH1 is designated to "p", a pixel in a range between TH1 and TH2 is designated to "z", and a pixel smaller than TH2 is designated to "m", thereby obtaining ternary-valued frequency component images TL1, TL2, TL3, . . . , TLk, TLk+1, . . . , TLn and ternary-valued frequency component images TR1, TR2, TR3 . . . , TRk, TRk+1, . . . , TRn; and   performing a matching operation between said first and second images based on said ternary-valued frequency component images,   wherein pixels in a one-dimensional window of the ternary-valued frequency component image TLk of said first image IL are compared in a one-to-one manner with pixels in a designated region of the ternary-valued frequency component image TRk of said second image IR, when said ternary-valued frequency component images TLk and TRk are identical in their frequency components,   an evaluation raesult "P" is obtained when corresponding two pixels are both "p" or "m", while an evaluation result "Z" is obtained when the corresponding two pixels are both "z", and   a similarity between two ternary-valued frequency component images TLk and TRk is evaluated by using the following equation:   Eall=Σβk(PN)k+Σγk(ZN)k       where PN represents a total number of pixels having the evaluation result "P", ZN represents a total number of pixels having the evaluation result "Z", and βk and γk represent weighing factors.   
     
     
       6. The method defined by claim 5, wherein said first image IL is designated as a reference image for the matching operation, a one-dimensional window capable of encompassing N pixels therein is set on the ternary-valued frequency component image of said first image IL, and a matching region having the same ternary-value pattern as said N pixels in said one-dimensional window is searched from the ternary-valued frequency component image of said second image IR. 
     
     
       7. The method defined by claim 5, wherein one of said first and second images IL and IR is designated as a reference image for the matching operation, a plurality of one-dimensional windows are set on the entire surface of said ternary-valued frequency component image of said reference image through a scanning operation along an epipolar line, so that said one-dimensional windows are successively overlapped at the same intervals of N/2 when each of said one-dimensional windows has a size equivalent to N pixels, and said matching operation is carried out with respect to each of said one-dimensional windows. 
     
     
       8. The method defined by claim 5, wherein a matching result in said matching operation is validated only when Σβk(PN)k is larger than a predetermined threshold TH3(>0).   
     
     
       9. A method of matching stereo images, comprising the steps of: inputting first and second images IL and IR;   developing said images IL and IR into a plurality of frequency component images FL1, FL2, FL3 . . . FLk, FLk+1, . . . , FLn and a plurality of frequency component images FR1, FR2, FR3 . . . FRk, FRk+1, . . . , FRn, respectively;   applying a secondary differential processing to each of said frequency component images;   converting each frequency component image, after being applied the secondary differential processing, into ternary values pixel by pixel in such a manner that a pixel not related to a zero-crossing point is designated to "z", a pixel related to a zero-crossing point and having a positive gradient is designated to "p", and a pixel related to a zero-crossing point and having a negative gradient is designated to "m", thereby obtaining ternary-valued frequency component images TL1, TL2, TL3, . . . , TLk, TLk+1, . . . , TLn and ternary-valued frequency component images TR1, TR2, TR3, . . . , TRk, TRk+1, . . . , TRn; and   performing a matching operation between said first and second images based an said ternary-valued frequency component images,   wherein pixels in a one-dimensional window of the ternary-valued frequency component image TLk of said first image IL are compared in a one-to-one manner with pixels in a designated region of the ternary-valued frequency component image TRk of said second image IR, when said ternary-valued frequency component images TLk and TRk are identical in their frequency components,   an evaluation raesult "P" is obtained when corresponding two pixels are both "p" or "m", while an evaluation result "Z" is obtained when the corresponding two pixels are both "z", and   a similarity between two ternary-valued frequency component images TLk and TRk is evaluated by using the following equation:   Eall=Σβk(PN)k+Σγk(ZN)k     where PN represents a total number of pixels having the evaluation result "P", ZN represents a total number of pixels having the evaluation result "Z", and βk and γk represent weighing factors.     
     
     
       10. The method defined by claim 9, wherein said first image IL is designated as a reference image for the matching operation, a one-dimensional window capable of encompassing N pixels therein is set on the ternary-valued frequency component image of said first image IL, and a matching region having the same ternary-value pattern as said N pixels in said one-dimensional window is searched from the ternary-valued frequency component image of said second image IR. 
     
     
       11. The method defined by claim 9, wherein one of said first and second images IL and IR is designated as a reference image for the matching operation, a plurality of one-dimensional windows are set on the entire surface of said ternary-valued frequency component image of said reference image through a scanning operation along an epipolar line, so that said one-dimensional windows are successively overlapped at the same intervals of N/2 when each of said one-dimensional windows has a size equivalent to N pixels, and said matching operation is carried out with respect to each of said one-dimensional windows. 
     
     
       12. The method defined by claim 9, wherein a matching result in said matching operation is validated only when Σβk(PN)k is larger than a predetermined threshold TH3(>0).   
     
     
       13. A method of matching stereo images, comprising the steps of: inputting first and second images IL and IR;   developing said images IL and IR into a plurality of frequency component images FL1, FL2, FL3, . . . , FLk, FLk+1, . . . , FLn and a plurality of frequency component images FR1, FR2, FR3, . . . , FRk, FRk+1, . . . , FRn, respectively;   applying a secondary differential processing to each of said frequency component images;   converting each low frequency component image of said frequency component images, after being applied the secondary differential processing, into ternary values pixel by pixel by using a positive threshold TH1(>0) and a negative threshold TH2(<0) in such a manner that a pixel larger than TH1 is designated to "p" a pixel in a range between TH1 and TH2 is designated to "z", and a pixel smaller than TH2 is designated to "m",   and converting each high frequency component image of said frequency component images, after being applied the secondary differential processing, into ternary values pixel by pixel in such a manner that a pixel not related to a zero-crossing point is designated to "z", a pixel related to a zero-crossing point and having a positive gradient is designated to "p", and a pixel related to a zero-crossing point and having a negative gradient is designated to "m", thereby obtaining ternary-valued frequency component images TL1, TL2, TL3, . . . , TLk, TLk+1, . . . , TLn and ternary-valued frequency component images TR1, TR2, TR3, . . . , TRk, TRk+1, . . . , TRn; and   performing a matching operation between said first and second images based on said ternary-valued frequency component images,   wherein pixels in a one-dimensional window of the ternary-valued frequency component image TLk of said first image IL are compared in a one-to-one manner with pixels in a designated region of the ternary-valued frequency component image TRk of said second image IR, when said ternary-valued frequency component images TLk and TRk are identical in their frequency components,   an evaluation raesult "P" is obtained when corresponding two pixels are both "p" or "m", while an evaluation result "Z" is obtained when the corresponding two pixels are both "z", and   a similarity between two ternary-valued frequency component images TLk and TRk is evaluated by using the following equation:   Eall=Σβk(PN)k+Σγk(ZN)k     where PN represents a total number of pixels having the evaluation result "P", ZN represents a total number of pixels having the evaluation result "Z", and βk and γk represent weighing factors.     
     
     
       14. The method defined by claim 13, wherein said first image IL is designated as a reference image for the matching operation, a one-dimensional window capable of encompassing N pixels therein is set on the ternary-valued frequency component image of said first image IL, and a matching region having the same ternary-value pattern as said N pixels in said one-dimensional window is searched from the ternary-valued frequency component image of said second image IR. 
     
     
       15. The method defined by claim 13, wherein one of said first and second images IL and IR is designated as a reference image for the matching operation, a plurality of one-dimensional windows are set on the entire surface of said ternary-valued frequency component image of said reference image through a scanning operation along an epipolar line, so that said one-dimensional windows are successively overlapped at the same intervals of N/2 when each of said one-dimensional windows has a size equivalent to N pixels, and said matching operation is carried out with respect to each of said one-dimensional windows. 
     
     
       16. The method defined by claim 13, wherein a matching result in said matching operation is validated only when Σβk(PN)k is larger than a predetermined threshold TH3(>0).