Detecting and responding to sounds for autonomous vehicles

Abstract

The technology relates to detecting and responding to sounds for a vehicle having an autonomous driving mode. In one example, an audible signal corresponding to a sound received at one or more microphones of the vehicle may be received. Sensor data generated by a perception system of the vehicle identifying objects in an environment of the vehicle may be received. A type of sound may be determined by inputting the audible signal into a classifier. A set of additional signals may be determined based on the determined type of sound. The sensor data may be processed in order to identify one or more additional signals of the identified set of additional signals. The vehicle may be controlled in the autonomous driving mode in order to respond to the sound based on the one or more additional signals and the type of sound.

1 . A method of detecting and responding to sounds for a vehicle having an autonomous driving mode, the method comprising:
 receiving, by one or more processors, an audible signal corresponding to a sound received at one or more microphones of the vehicle;   determining a type of the sound; and
 controlling, by the one or more processors, the vehicle in the autonomous driving mode in order to: (1) initially respond to the sound based on the type of the sound, and (2) continue to respond to the sound based on one or more additional signals and the type of the sound, wherein the one or more additional signals are identified based on the type of the sound. 
   
     
     
         2 . The method of  claim 1 , further comprising, determining that the one or more additional signals are a predetermined combination of additional signals, and wherein controlling the vehicle is further based on the determination that the one or more additional signals are the predetermined combination of additional signals. 
     
     
         3 . The method of  claim 1 , further comprising;
 receiving, by the one or more processors, sensor data from one or more sensors of the vehicle, the sensor data identifying objects in an environment of the vehicle; and   processing, by the one or more processors, the sensor data in order to identify the one or more additional signals.   
     
     
         4 . The method of  claim 1 , wherein the type of the sound is associated with a set of additional signals. 
     
     
         5 . The method of  claim 1 , further comprising inputting the audible signal into a classifier. 
     
     
         6 . The method of  claim 1 , wherein the type of the sound is a train whistle. 
     
     
         7 . The method of  claim 1 , wherein the type of the sound is a reverse beeping sound. 
     
     
         8 . The method of  claim 1 , wherein the type of the sound is a crosswalk chirp. 
     
     
         9 . The method of  claim 1 , wherein the controlling further comprises automatically slowing down the vehicle in response to identifying the type of the sound as a train whistle. 
     
     
         10 . The method of  claim 1 , wherein the controlling further comprises automatically yielding to all larger vehicles in response to identifying the type of the sound as a a reverse beeping sound. 
     
     
         11 . A system for detecting and responding to sounds for a vehicle having an autonomous driving mode, the system comprising:
 one or more processors configured to:
 receive an audible signal corresponding to a sound received at one or more microphones of the vehicle; and 
 control the vehicle in the autonomous driving mode in order to: (1) initially respond to the sound based on a type of the sound, and (2) continue to respond to the sound based on one or more additional signals and the type of the sound, wherein the one or more additional signals are identified based on the type of the sound. 
   
     
     
         12 . The system of  claim 11 , wherein the one or more processors are configured to determine that the one or more additional signals are a predetermined combination of additional signals, and wherein the vehicle is controlled based on the determination that the one or more additional signals are the predetermined combination of additional signals. 
     
     
         13 . The system of  claim 11 , further comprising a memory including a classifier, wherein the classifier is configured to determine the type of the sound when the audible signal is input into the classifier. 
     
     
         14 . The system of  claim 11 , further comprising one or more sensors configured to generate sensor data identifying objects in an environment of the vehicle, wherein the one or more processors are configured to receive and process the sensor data in order to identify the one or more additional signals. 
     
     
         15 . The system of  claim 11 , wherein the type of the sound is associated with a set of additional signals. 
     
     
         16 . The system of  claim 11 , wherein the type of the sound is a train whistle. 
     
     
         17 . The system of  claim 11 , wherein the type of the sound is a reverse beeping sound. 
     
     
         18 . The system of  claim 11 , wherein the type of the sound is a crosswalk chirp. 
     
     
         19 . The system of  claim 11 , wherein the one or more processors control the vehicle to automatically slow the vehicle down when the type of the sound is a train whistle. 
     
     
         20 . The system of  claim 11 , wherein the one or more processors control the vehicle to automatically yield to all larger vehicles when the type of the sound is a reverse beeping sound.