Collision Avoidance Using Auditory Data Augmented With Map Data

Abstract

A controller for an autonomous vehicle receives audio signals from one or more microphones and identifies sounds. The controller further identifies an estimated location of the sound origin and the type of sound, i.e. whether the sound is a vehicle and/or the type of vehicle. The controller analyzes map data and attempts to identify a landmark within a tolerance from the estimated location. If a landmark is found corresponding to the estimated location and type of the sound origin, then the certainty is increased that the source of the sound is at that location and is that type of sound source. Collision avoidance is then performed with respect to the location of the sound origin and its type with the certainty as augmented using the map data. Collision avoidance may include automatically actuating brake, steering, and accelerator actuators in order to avoid the location of the sound origin.

1 . A controller for an autonomous vehicle comprising:
 one or more processing devices programmed to:
 receive one or more audio streams from one or more microphones; 
 retrieve map data in a region about the autonomous vehicle; 
 detect, in the one or more audio streams, a probable vehicle-originated sound; 
 identify a first predicted location of the probable vehicle-originated sound; 
 retrieve map data in a region proximate the first predicted location; and 
 when the map data indicates a vehicle-specific landmark within a threshold distance from the first predicted location, perform obstacle avoidance with respect to the first predicted location. 
   
     
     
         2 . The controller of  claim 1 , wherein the one or more processing devices are further programmed to invoke obstacle avoidance with respect to the first predicted location by actuating at least one of a steering actuator, accelerator actuator, and brake actuator of the autonomous vehicle effective to avoid the first predicted location. 
     
     
         3 . The controller of  claim 1 , wherein:
 the one or more microphones are an array of microphones and the one or more audio streams are a plurality of audio streams from the array of microphones;   the one or more processing devices are further programmed to identify the first predicted location of the probable vehicle-originated sound by comparing at least one of time of arrival and intensity of the probable vehicle-originated sound in the plurality of audio streams.   
     
     
         4 . The controller of  claim 1 , wherein the one or more microphones are directional microphones. 
     
     
         5 . The controller of  claim 1 , wherein the one or more processing devices are further programmed to:
 identify landmarks in the region proximate the first predicted location in the map data;   identify, from the map data, the vehicle-specific landmark among the landmarks in the region proximate the first predicted location; and   select, as a probable location of another vehicle, the vehicle-specific landmark.   
     
     
         6 . The controller of  claim 5 , wherein the vehicle-specific landmark is at least one of a parking lot and emergency vehicle station. 
     
     
         7 . The controller of  claim 6 , wherein the one or more processing devices are further programmed to:
 identify a predicted vehicle type from the probable vehicle-originated sound;   if the predicted vehicle type corresponds to the vehicle specific landmark, confirm the predicted vehicle type.   
     
     
         8 . The controller of  claim 7 , wherein the one or more processing devices are further programmed to:
 if the predicted vehicle type corresponds to the vehicle specific landmark and the vehicle specific landmark is an emergency vehicle station, invoke pulling over and stopping of the autonomous vehicle.   
     
     
         9 . The controller of  claim 1 , wherein the first predicted location is not within a line of sight of an imaging system of the autonomous vehicle. 
     
     
         10 . The controller of  claim 1 , wherein the one or more processing devices are further programmed to:
 receive outputs from one or more sensors, the one or more sensors including at least one of a Light Detection and Ranging (LIDAR) sensor and a Radio Detection and Ranging (RADAR) sensor;   detect an obstacle set in the outputs from the one or more sensors;   add the first predicted location to the obstacle set; and   perform obstacle avoidance with respect to the obstacle set.   
     
     
         11 . An autonomous vehicle comprising:
 a vehicle including an engine and wheels selectively coupled to the engine;   at least one of a steering actuator, accelerator actuator, and brake actuator;   one or more microphones;   a controller operably coupled to the one or more microphones and the at least one of the steering actuator, the accelerator actuator, and the brake actuator, the controller including one or more processing devices programmed to—
 receive one or more audio streams from the one or more microphones; 
 detect, in the one or more audio streams, a probable vehicle-originated sound; 
 identify a first predicted location of the probable vehicle-originated sound; 
 retrieve map data in a region proximate the first predicted location; 
 when the map data indicates a vehicle-specific landmark within a threshold distance from the first predicted location, perform obstacle avoidance with respect to the first predicted location. 
   
     
     
         12 . The autonomous vehicle of  claim 11 , wherein the one or more processing devices are further programmed to invoke obstacle avoidance with respect to the first predicted location by actuating at least one of a steering actuator, accelerator actuator, and brake actuator of the autonomous vehicle effective to avoid the first predicted location. 
     
     
         13 . The autonomous vehicle of  claim 11 , wherein:
 the one or more microphones are an array of microphones and the one or more audio streams are a plurality of audio streams from the array of microphones;   the one or more processing devices are further programmed to identify the first predicted location of the probable vehicle-originated sound by comparing at least one of time of arrival and intensity of the probable vehicle-originated sound in the plurality of audio streams.   
     
     
         14 . The autonomous vehicle of  claim 11 , wherein the one or more microphones are directional microphones. 
     
     
         15 . The autonomous vehicle of  claim 11 , wherein the one or more processing devices are further programmed to:
 identify landmarks in the region proximate the first predicted location;   identify, from the map data, the vehicle-specific landmark among the landmarks in the region proximate the first predicted location; and   select, as a probable location of another vehicle, the vehicle-specific landmark.   
     
     
         16 . The autonomous vehicle of  claim 15 , wherein the vehicle specific landmark is at least one of a parking lot and emergency vehicle station. 
     
     
         17 . The autonomous vehicle of  claim 16 , wherein the one or more processing devices are further programmed to:
 identify a predicted vehicle type from the probable vehicle-originated sound;   if the predicted vehicle type corresponds to the vehicle specific landmark, confirm the predicted vehicle type.   
     
     
         18 . The autonomous vehicle of  claim 17 , wherein the one or more processing devices are further programmed to:
 if the predicted vehicle type corresponds to the vehicle specific landmark and the vehicle specific landmark is an emergency vehicle station, invoke pulling over and stopping of the autonomous vehicle.   
     
     
         19 . The autonomous vehicle of  claim 11 , wherein the first predicted location is not within a line of sight of an imaging system of the autonomous vehicle. 
     
     
         20 . The autonomous vehicle of  claim 11 , wherein the one or more processing devices are further programmed to:
 receive outputs from one or more sensors, the one or more sensors including at least one of a Light Detection and Ranging (LIDAR) sensor and a Radio Detection and Ranging (RADAR) sensor;   detect an obstacle set in the outputs from the one or more sensors;   add the first predicted location to the obstacle set; and   perform obstacle avoidance with respect to the obstacle set.