Method for processing image data for display using backward projection

Abstract

A method for processing image data for a vehicular vision system includes providing a plurality of cameras at a vehicle so as to have respective fields of view exterior the vehicle and providing a video display screen in the vehicle. A control has a data processor for processing image data captured by the plurality of cameras. Image data captured by at least one camera of the plurality of cameras is processed at the control utilizing backward projection from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of the at least one camera. Responsive to the processing of captured image data, video images derived at least in part from the processed image data are displayed.

1 . A method for processing image data for a vehicular vision system, said method comprising:
 providing a plurality of cameras at a vehicle so as to have respective fields of view exterior the vehicle;   wherein each camera of the plurality of cameras comprises a two dimensional array of photosensing pixels;   wherein providing the plurality of cameras comprises (i) mounting a rear camera at a rear portion of the vehicle so as to have a field of view at least rearward of the vehicle, (ii) mounting a left-side camera at a left side portion of the vehicle so as to have a field of view at least sideward of the vehicle, (iii) mounting a right-side camera at a right side portion of the vehicle so as to have a field of view at least sideward of the vehicle and (iv) mounting a front camera at a front portion of the vehicle so as to have a field of view at least forward of the vehicle;   wherein, with the plurality of cameras mounted at the vehicle, the field of view of the left-side camera partially overlaps the field of view of the front camera and partially overlaps the field of view of the rear camera, and the field of view of the right-side camera partially overlaps the field of view of the front camera and partially overlaps the field of view of the rear camera;   providing a video display screen in the vehicle, wherein the video display screen comprises a two dimensional grid of display pixels;   providing a control having a data processor for processing image data captured by the plurality of cameras;   processing, at the control, image data captured by at least one camera of the plurality of cameras utilizing backward projection from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of the at least one camera of the plurality of cameras; and   responsive to the processing at the control of the image data captured by the at least one camera of the plurality of cameras, displaying, at the video display screen, video images derived at least in part from the processed image data.   
     
     
         2 . The method of  claim 1 , wherein each camera of the plurality of cameras comprises at least one million photosensing pixels. 
     
     
         3 . The method of  claim 2 , wherein processing image data captured by the at least one camera of the plurality of cameras comprises utilizing ray tracing from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of the at least one camera of the plurality of cameras. 
     
     
         4 . The method of  claim 1 , wherein processing image data captured by the at least one camera of the plurality of cameras comprises restricting processing of photosensing pixels of the at least one camera of the plurality of cameras to processing of only those photosensing pixels of the at least one camera of the plurality of cameras that are used for the displayed video images. 
     
     
         5 . The method of  claim 1 , wherein processing image data captured by the at least one camera of the plurality of cameras comprises processing image data captured by all of the plurality of cameras utilizing backward projection from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of each respective camera of the plurality of cameras. 
     
     
         6 . The method of  claim 5 , wherein displaying video images comprises displaying video images derived from the processed image data captured by all of the plurality of cameras to provide a surround view display. 
     
     
         7 . The method of  claim 1 , wherein processing image data captured by the at least one camera of the plurality of cameras comprises utilizing backward projection from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of the one camera of the plurality of cameras. 
     
     
         8 . The method of  claim 7 , wherein displaying video images comprises displaying video images derived from the processed image data captured by the one camera of the plurality of cameras. 
     
     
         9 . The method of  claim 1 , comprising buffering data from only those photosensing pixels of the at least one camera of the plurality of cameras that are used in the backward projection. 
     
     
         10 . The method of  claim 1 , wherein processing image data captured by the at least one camera of the plurality of cameras comprises utilizing unwarping. 
     
     
         11 . A method for processing image data for a vehicular vision system, said method comprising:
 providing a plurality of cameras at a vehicle so as to have respective fields of view exterior the vehicle;   wherein each camera of the plurality of cameras comprises a two dimensional array of photosensing pixels;   wherein providing the plurality of cameras comprises (i) mounting a rear camera at a rear portion of the vehicle so as to have a field of view at least rearward of the vehicle, (ii) mounting a left-side camera at a left side portion of the vehicle so as to have a field of view at least sideward of the vehicle, (iii) mounting a right-side camera at a right side portion of the vehicle so as to have a field of view at least sideward of the vehicle and (iv) mounting a front camera at a front portion of the vehicle so as to have a field of view at least forward of the vehicle;   wherein, with the plurality of cameras mounted at the vehicle, the field of view of the left-side camera partially overlaps the field of view of the front camera and partially overlaps the field of view of the rear camera, and the field of view of the right-side camera partially overlaps the field of view of the front camera and partially overlaps the field of view of the rear camera;   providing a video display screen in the vehicle, wherein the video display screen comprises a two dimensional grid of display pixels;   providing a control having a data processor for processing image data captured by the plurality of cameras;   processing, at the control, image data captured by the plurality of cameras utilizing backward projection from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of each respective camera of the plurality of cameras;   responsive to the processing at the control of image data captured by each camera of the plurality of cameras, displaying, at the video display screen, video images derived at least in part from the processed image data to provide a surround view display; and   wherein processing image data captured by each camera of the plurality of cameras comprises restricting processing of photosensing pixels of each camera of the plurality of cameras to processing of only those photosensing pixels of each camera of the plurality of cameras that are used for the displayed video images.   
     
     
         12 . The method of  claim 11 , wherein each camera of the plurality of cameras comprises at least one million photosensing pixels. 
     
     
         13 . The method of  claim 12 , wherein processing image data captured by each camera of the plurality of cameras comprises utilizing ray tracing from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of each respective camera of the plurality of cameras. 
     
     
         14 . The method of  claim 11 , comprising buffering data from only those photosensing pixels of each camera of the plurality of cameras that are used in the backward projection. 
     
     
         15 . The method of  claim 11 , wherein processing image data captured by each camera of the plurality of cameras comprises utilizing unwarping. 
     
     
         16 . A method for processing image data for a vehicular vision system, said method comprising:
 providing a plurality of cameras at a vehicle so as to have respective fields of view exterior the vehicle;   wherein each camera of the plurality of cameras comprises a two dimensional array of photosensing pixels;   wherein providing the plurality of cameras comprises (i) mounting a rear camera at a rear portion of the vehicle so as to have a field of view at least rearward of the vehicle, (ii) mounting a left-side camera at a left side portion of the vehicle so as to have a field of view at least sideward of the vehicle, (iii) mounting a right-side camera at a right side portion of the vehicle so as to have a field of view at least sideward of the vehicle and (iv) mounting a front camera at a front portion of the vehicle so as to have a field of view at least forward of the vehicle;   wherein, with the plurality of cameras mounted at the vehicle, the field of view of the left-side camera partially overlaps the field of view of the front camera and partially overlaps the field of view of the rear camera, and the field of view of the right-side camera partially overlaps the field of view of the front camera and partially overlaps the field of view of the rear camera;   providing a video display screen in the vehicle, wherein the video display screen comprises a two dimensional grid of display pixels;   providing a control having a data processor for processing image data captured by the plurality of cameras;   processing, at the control, image data captured by one camera of the plurality of cameras utilizing backward projection from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of the one camera of the plurality of cameras;   responsive to the processing at the control of the image data captured by the one camera of the plurality of cameras, displaying, at the video display screen, video images derived at least in part from the processed image data; and   wherein processing image data captured by the one camera of the plurality of cameras comprises restricting processing of photosensing pixels of the one camera of the plurality of cameras to processing of only those photosensing pixels of the one camera of the plurality of cameras that are used for the displayed video images.   
     
     
         17 . The method of  claim 16 , wherein each camera of the plurality of cameras comprises at least one million photosensing pixels. 
     
     
         18 . The method of  claim 17 , wherein processing image data captured by the one camera of the plurality of cameras comprises utilizing ray tracing from individual display pixels of the two dimensional grid of display pixels of the video display screen to respective photosensing pixels of the two dimensional array of photosensing pixels of the one camera of the plurality of cameras. 
     
     
         19 . The method of  claim 16 , comprising buffering data from only those photosensing pixels of the one camera of the plurality of cameras that are used in the backward projection. 
     
     
         20 . The method of  claim 16 , wherein processing image data captured by the one camera of the plurality of cameras comprises utilizing unwarping.