System and method for night vision object detection and driver assistance

Abstract

A stereo vision system includes a first camera sensor and a second camera sensor. The first camera sensor is configured to sense first reflected energy and generate first sensor signals based on the sensed first reflected energy. The second camera sensor is configured to sense second reflected energy and generate second sensor signals based on the sensed second reflected energy. The stereo vision system further includes a processor configured to receive the first sensor signals from the first camera sensor and configured to receive the second sensor signals from the second camera sensor. The processor is configured to perform stereo matching based on the first sensor signals and the second sensor signals. The first camera sensor is configured to sense reflected energy that is infrared radiation. The second camera sensor is configured to sense reflected energy that is infrared radiation.

What is claimed is: 
     
         1 . A stereo vision system for use in a vehicle, the stereo vision system comprising:
 a first camera sensor configured to sense first reflected energy and generate first sensor signals based on the sensed first reflected energy;   a second camera sensor configured to sense second reflected energy and generate second sensor signals based on the sensed second reflected energy; and   a processor configured to receive the first sensor signals from the first camera sensor and configured to receive the second sensor signals from the second camera sensor,   wherein the processor is configured to perform stereo matching based on the first sensor signals and the second sensor signals,   wherein the first camera sensor is configured to sense reflected energy that is infrared radiation, and   wherein the second camera sensor is configured to sense reflected energy that is infrared radiation.   
     
     
         2 . The stereo vision system of  claim 1 ,
 wherein the processor is configured to perform the stereo matching by producing a stereo range map,   wherein the processor is configured to perform object detection using the stereo range map,   wherein the processor is configured to perform object tracking using a result of the object detection, and   wherein the processor is configured to provide an output signal based on a result of the object tracking in order to provide assistance to a driver of the vehicle.   
     
     
         3 . The stereo vision system of  claim 1 ,
 wherein the first camera sensor is configured to sense reflected energy that is short-wavelength infrared radiation, and   wherein the second camera sensor is configured to sense reflected energy that is short-wavelength infrared radiation.   
     
     
         4 . The stereo vision system of  claim 1 ,
 wherein an energy sensitive area of the first camera sensor is constructed using indium gallium arsenide, and   wherein an energy sensitive area of the second camera sensor is constructed using indium gallium arsenide.   
     
     
         5 . The stereo vision system of  claim 1 ,
 wherein the stereo vision system does not include an active illumination component for emitting electromagnetic radiation that can be sensed by the stereo vision system upon reflection off of objects in an environment sensed by the stereo vision system.   
     
     
         6 . The stereo vision system of  claim 1 ,
 wherein the stereo vision system does not include a component for emitting infrared radiation.   
     
     
         7 . The stereo vision system of  claim 1 , the stereo vision system further comprising:
 an active illumination component configured to emit infrared radiation.   
     
     
         8 . The stereo vision system of  claim 7 ,
 wherein the active illumination component is configured to alternate between emitting infrared radiation and not emitting infrared radiation, and   wherein the active illumination component is configured to emit infrared radiation in synchronization with an exposure interval of the first camera sensor and an exposure interval of the second camera sensor.   
     
     
         9 . The stereo vision system of  claim 7 ,
 wherein the active illumination component comprises:
 one or more laser diodes configured to emit infrared radiation in one or more collimated beams; and 
 one or more optical filters configured to produce one or more diffused conic beams from the one or more collimated beams. 
   
     
     
         10 . The stereo vision system of  claim 9 ,
 wherein the one or more laser diodes comprises:
 a first laser diode configured to emit infrared radiation in a first collimated beam; 
 a second laser diode configured to emit infrared radiation in a second collimated beam; and 
 a third laser diode configured to emit infrared radiation in a third collimated beam, 
   wherein the one or more optical filters comprises:
 a first optical filter configured to produce a first diffused conic beam at a first dispersion angle from the first collimated beam; 
 a second optical filter configured to produce a second diffused conic beam at a second dispersion angle from the second collimated beam; and 
 a third optical filter configured to produce a third diffused conic beam at a third dispersion angle from the third collimated beam, 
   wherein the first dispersion angle is different from the second dispersion angle and the third dispersion angle, and   wherein the second dispersion angle is different from the third dispersion angle.   
     
     
         11 . The stereo vision system of  claim 1 , the stereo vision system further comprising:
 a third camera sensor configured to sense third reflected energy and generate third sensor signals based on the sensed third reflected energy,   wherein the processor is configured to receive the third sensor signals from the third camera sensor, and   wherein the third camera sensor is configured to sense reflected energy that is infrared radiation.   
     
     
         12 . The stereo vision system of  claim 11 ,
 wherein the second camera sensor is positioned between the first camera sensor and the third camera sensor,   wherein the processor is configured to perform first stereo matching based on the first sensor signals and the second sensor signals but not the third sensor signals, and   wherein the processor is configured to perform second stereo matching based on the second sensor signals and the third sensor signals but not the first sensor signals.   
     
     
         13 . The stereo vision system of  claim 12 ,
 wherein the processor performs the first stereo matching for a first downrange distance range having a first minimum downrange distance and a first maximum downrange distance,   wherein the processor performs the second stereo matching for a second downrange distance range having a second minimum downrange distance and a second maximum downrange distance, and   wherein the first minimum downrange distance is substantially the same as the second minimum downrange distance.   
     
     
         14 . The stereo vision system of  claim 12 ,
 wherein the processor is configured to perform first object tracking based on a result of the first stereo matching but not based on a result of the second stereo matching, and   wherein the processor is configured to perform second object tracking based on a result of the second stereo matching but not based on a result of the first stereo matching.   
     
     
         15 . The stereo vision system of  claim 14 ,
 wherein the processor is configured to perform merging of a result of the first object tracking and a result of the second object tracking.   
     
     
         16 . The stereo vision system of  claim 11 ,
 wherein the second camera sensor is positioned between the first camera sensor and the third camera sensor,   wherein the processor is configured to perform first stereo matching based on the first sensor signals and the second sensor signals but not the third sensor signals, and   wherein the processor is configured to perform second stereo matching based on the first sensor signals and the third sensor signals but not the second sensor signals.   
     
     
         17 . The stereo vision system of  claim 16 ,
 wherein the processor performs the first stereo matching for a first downrange distance range having a first minimum downrange distance and a first maximum downrange distance,   wherein the processor performs the second stereo matching for a second downrange distance range having a second minimum downrange distance and a second maximum downrange distance, and   wherein the first maximum downrange distance is substantially the same as the second minimum downrange distance.   
     
     
         18 . The stereo vision system of  claim 16 ,
 wherein the processor is configured to perform merging of a result of the first stereo matching and a result of the second stereo matching.   
     
     
         19 . The stereo vision system of  claim 18 ,
 wherein the processor performs the merging by performing a union of a first stereo range map resulting from the first stereo mapping and a second stereo range map resulting from the second stereo mapping.   
     
     
         20 . A stereo vision system for use in a vehicle, the stereo vision system comprising:
 a first camera sensor configured to sense first reflected energy and generate first sensor signals based on the sensed first reflected energy;   a second camera sensor configured to sense second reflected energy and generate second sensor signals based on the sensed second reflected energy;   a third camera sensor configured to sense third energy and generate third sensor signals based on the sensed third reflected energy; and   a processor configured to receive the first sensor signals from the first camera sensor, configured to receive the second sensor signals from the second camera sensor, and configured to receive the third sensor signals from the third camera sensor,   wherein the processor is further configured to perform stereo matching based on at least one of the first sensor signals, the second sensor signals, and the third sensor signals,   wherein the first camera sensor is configured to sense reflected energy that is visible radiation,   wherein the second camera sensor is configured to sense reflected energy that is visible radiation,   wherein the third camera sensor is configured to sense energy that is infrared radiation.   
     
     
         21 . The stereo vision system of  claim 20 ,
 wherein the third camera sensor is configured to sense energy that is thermal emitted energy.   
     
     
         22 . The stereo vision system of  claim 20 ,
 wherein the processor is configured to perform merging of the first sensor signal, the second sensor signals, and the third sensor signals after performing image rectification but prior to performing stereo matching.   
     
     
         23 . The stereo vision system of  claim 20 ,
 wherein the processor is configured to perform combining of the first sensor signal, the second sensor signals, and the third sensor signals after performing image rectification but prior to performing stereo matching.   
     
     
         24 . The stereo vision system of  claim 20 ,
 wherein the processor is configured to perform stereo matching based on the first sensor signals and the second sensor signals in order to produce a stereo range map, and   wherein the processor is configured to perform combining of the third sensor signals with the stereo range map.   
     
     
         25 . The stereo vision system of  claim 20 ,
 wherein the processor is configured to perform first stereo matching based on the first sensor signals and the second sensor signals,   wherein the processor is configured to perform first object tracking based on a result of the first stereo matching,   wherein the processor is configured to perform second object tracking based on the third sensor signals, and   wherein the processor is configured to perform combining of a result of the first object tracking and a result of the second object tracking.   
     
     
         26 . A method for stereo vision in a vehicle, the method comprising:
 sensing first reflected energy using a first camera sensor;   generating first sensor signals based on the sensed first reflected energy;   sensing second reflected energy using a second camera sensor;   generating second sensor signals based on the sensed second reflected energy; and   performing stereo matching based on the first sensor signals and the second sensor signals,   wherein the first reflected energy is infrared radiation, and   wherein the second reflected energy is infrared radiation.