Intelligent vehicles with advanced vehicle camera systems for underbody hazard and foreign object detection

Abstract

A method for operating an advanced driver assistance (ADAS) system of a motor vehicle includes a vehicle controller receiving, from side and end cameras mounted to the vehicle, camera signals indicative of real-time images of outboard-facing side and end views of the vehicle. The controller determines a region of interest (ROI) inset within each end/side view within which is expected foreign objects and/or hazards. These ROIs are analyzed to detect if a foreign object/hazard is present in the vehicle's end and/or side views. Responsive to detecting the foreign object/hazard, movement of the foreign object/hazard is tracked to determine if the foreign object/hazard moves towards or away from the vehicle's underbody region. If the foreign object/hazard moves to the underbody region, control signals are transmitted to the vehicle's propulsion and/or steering system to automate preventative action that prevents collision of the vehicle with and/or removes the foreign object/hazard from the underbody.

What is claimed: 
     
         1 . A motor vehicle, comprising:
 a vehicle body with a plurality of road wheels mounted to the vehicle body;   a propulsion system with a prime mover mounted to the vehicle body and operable to drive one or more of the road wheels to thereby propel the motor vehicle;   a steering system mounted to the vehicle body and operable to steer the vehicle;   a camera system including an end camera and a side camera, the end camera mounted proximate a front or rear end of the vehicle body, and the side camera mounted proximate a left or right side of the vehicle body, the end and side cameras each being configured to capture images of a respective outboard-facing camera view from the motor vehicle; and   a vehicle controller communicatively connected to the propulsion system the steering system, and the camera system, the vehicle controller being programmed to:
 receive, from the camera system, camera signals indicative of real-time images of the outboard-facing camera views captured by the end and side cameras, the end and side cameras each having a respective full-frame area; 
 determine a respective region of interest (ROI) inset within and fixed at a predefined location of each of the outboard-facing camera views of the end and side cameras, the ROIs each including a respective delineated camera frame having an ROI area smaller than the respective full-frame area of the corresponding camera; 
 analyze the ROIs to detect if a foreign object and/or hazard is present in any of the outboard-facing camera views of the end and side cameras; 
 responsive to detecting the foreign object and/or hazard, track movement of the foreign object and/or hazard to determine if the foreign object and/or hazard moves towards or is in an underbody region of the motor vehicle; and 
 responsive to determining the foreign object and/or hazard is moving towards or is in the underbody region, transmit an alert to the driver and/or a control signal to the propulsion system and/or the steering system to take preventative action calibrated to prevent collision of the motor vehicle with and/or remove the foreign object and/or hazard from the underbody region of the motor vehicle. 
   
     
     
         2 . The motor vehicle of  claim 1 , wherein the ROI of the side camera is immediately adjacent and contains a lateral portion of the vehicle body, and the ROI of the end camera is immediately adjacent and contains an end portion of the vehicle body. 
     
     
         3 . The motor vehicle of  claim 2 , wherein the ROI of the side camera includes a first portion of a ground surface laterally adjacent the motor vehicle, and the ROI of the end camera includes a second portion of the ground surface fore or aft of the motor vehicle. 
     
     
         4 . The motor vehicle of  claim 1 , wherein tracking the movement of the foreign object and/or hazard via the vehicle controller is responsive to detecting the foreign object and/or hazard in the ROIs of both the end and side cameras. 
     
     
         5 . The motor vehicle of  claim 1 , wherein the side camera includes a left side camera mounted proximate the left side of the vehicle body and a right side camera mounted proximate the right side of the vehicle body. 
     
     
         6 . The motor vehicle of  claim 2 , wherein the end camera includes a front camera mounted proximate the front end of the vehicle body and a back camera mounted proximate the rear end of the vehicle body. 
     
     
         7 . The motor vehicle of  claim 1 , further comprising an underbody camera mounted to the vehicle body proximate an undercarriage thereof, the underbody camera being configured to capture an outboard-facing downward view and generate signals indicative thereof. 
     
     
         8 . The motor vehicle of  claim 7 , wherein the vehicle controller is further programmed to, responsive to detecting the foreign object and/or hazard in any of the ROIs of the end and side cameras, analyze the outboard-facing downward view of the underbody camera to determine if the foreign object and/or hazard is located in the underbody region of the vehicle. 
     
     
         9 . The motor vehicle of  claim 8 , wherein the vehicle controller is further programmed to, responsive to detecting the foreign object and/or hazard in any of the ROIs of the end and side cameras, determine if the foreign object and/or hazard has subsequently disappeared from the ROIs of the end and side cameras, wherein analyzing the outboard-facing downward view of the underbody camera is responsive to the foreign object and/or hazard disappearing from all of the ROIs of the end and side cameras. 
     
     
         10 . The motor vehicle of  claim 1 , wherein analyzing the ROIs to detect if the foreign object is present in the outboard-facing camera views of the end and side cameras includes scanning the real-time images for any of a plurality of designated foreign objects each having a respective target marker, the respective target markers each including a shape, a size, a color, a face, eyes, and/or a behavioral pattern associated with the designated foreign object. 
     
     
         11 . The motor vehicle of  claim 9 , further comprising a memory device communicatively connected to the vehicle controller and storing therein a lookup table with the respective target markers associated with the designated foreign objects, wherein the predetermined target markers are derived via a trained neural network, the vehicle controller being further programmed to retrieve, from the memory device prior to analyzing the ROIs, the predetermined target markers from the lookup table. 
     
     
         12 . The motor vehicle of  claim 1 , wherein determining the respective ROIs includes calibrating the ROIs specific to the motor vehicle based on a make/model of the motor vehicle, respective mounting locations of the end and side cameras on the vehicle body, and/or a predefined protection region around the motor vehicle. 
     
     
         13 . The motor vehicle of  claim 1 , wherein the vehicle controller is further programmed to:
 responsive to detecting the foreign object, assign an index identification (ID) and a start location to the foreign object; and   monitor the foreign object in subsequent frames of the real-time images captured by the end and side cameras by scanning for the index ID with respect to the start location.   
     
     
         14 . The motor vehicle of  claim 13 , further comprising an underbody camera mounted to the vehicle body proximate an undercarriage thereof, wherein the vehicle controller is further programmed to:
 determine if the foreign object disappears from the subsequent frames of the real-time images captured by the end and side cameras; and   responsive to the foreign object disappearing from the subsequent frames, analyze real-time images of the underbody region captured by the underbody camera to detect if the foreign object is present in the underbody region.   
     
     
         15 . A method for operating a motor vehicle, the motor vehicle including a propulsion system, a steering system, and a camera system, the method comprising:
 receiving, via a vehicle controller from the camera system of the motor vehicle, camera signals indicative of real-time images of outboard-facing camera views captured by an end camera and a side camera of the camera system, the end camera mounted proximate a front or rear end of a vehicle body of the motor vehicle, and the side camera mounted proximate a left or right side of the vehicle body, the end and side cameras each having a respective full-frame area;   determining a respective region of interest (ROI) inset within and fixed at a predefined location of each of the outboard-facing camera views of the end and side cameras, the ROIs each including a respective delineated camera frame having an ROI area smaller than the respective full-frame area of the corresponding camera;   analyzing, via the vehicle controller, the ROIs to detect if the foreign object and/or hazard is present in the outboard-facing camera views of the end and side cameras;   tracking, responsive to detecting the foreign object and/or hazard, movement of the foreign object and/or hazard to determine if the foreign object and/or hazard moves towards or is in an underbody region of the motor vehicle; and   transmitting, responsive to determining the foreign object and/or hazard is moving towards or is in the underbody region, an alert to the driver and/or a control signal to the propulsion system and/or the steering system to take preventative action calibrated to prevent collision of the motor vehicle with and/or remove the foreign object and/or hazard from the underbody region.   
     
     
         16 . The method of  claim 15 , wherein the side camera includes left and right side cameras mounted proximate left and right sides the vehicle body, respectively, and the end camera includes front and back cameras mounted proximate front and rear ends of the vehicle body, wherein tracking the movement of the foreign object and/or hazard is responsive to detecting the foreign object and/or hazard in the ROI of any one of the left, right, front, and back cameras. 
     
     
         17 . The method of  claim 15 , wherein the camera system further includes an underbody camera mounted proximate an undercarriage of the vehicle body and configured to capture an outboard-facing downward view from the motor vehicle, the method further comprising analyzing, responsive to detecting the foreign object and/or hazard, an underbody ROI inset within the outboard-facing downward view of the underbody camera to determine if the foreign object and/or hazard is located in the underbody region of the motor vehicle. 
     
     
         18 . The method of  claim 17 , further comprising determining, responsive to detecting the foreign object and/or hazard in any of the ROIs of the end and side cameras, if the foreign object and/or hazard has subsequently disappeared from each of the ROIs, wherein analyzing the outboard-facing downward view of the underbody camera is responsive to the foreign object and/or hazard disappearing from all of the ROIs. 
     
     
         19 . The method of  claim 15 , wherein defining the respective ROIs includes calibrating, specific to the motor vehicle, the ROIs based on a make/model of the motor vehicle, respective mounting locations of the camera, and/or a predefined protection region around the motor vehicle. 
     
     
         20 . The method of  claim 15 , further comprising:
 assigning, responsive to detecting the foreign object, an index ID and a start location to the foreign object; and   monitoring the foreign object in subsequent frames in the real-time images captured by the end and side cameras by scanning for the index ID with respect to the start location.