Microphone Array for Sound Source Detection and Location

Abstract

Systems, methods, tangible non-transitory computer-readable media, and devices associated with detecting and locating sounds are provided. For example, sound data associated with sounds can be received. The sounds can include source sounds and background sounds received by microphones. Based on the sound data, time differences can be determined. Each of the time differences can include a time difference between receipt of a source sound and receipt of a background sound at each of the microphones respectively. A set of the source sounds can be synchronized based on the time differences. An amplified source sound can be generated based on a combination of the synchronized set of the source sounds. A source location of the source sounds can be determined based on the amplified source sound. Based on the source location, control signals can be generated in order to change actions performed by an autonomous vehicle.

1 .- 20 . (canceled) 
     
     
         21 . An autonomous vehicle (AV) control system comprising:
 one or more processors;   a memory comprising one or more tangible non-transitory computer-readable media, the memory storing computer-readable instructions that are executable by the one or more processors to cause the AV control system to perform operations comprising:
 receiving sound data associated with a source sound received by a plurality of microphones associated with an autonomous vehicle; 
 providing the sound data as input to a machine-learned model having been trained to recognize the source sound by analyzing one or more features of the source sound received by the plurality of microphones; 
 receiving, as an output of the machine-learned model and based on receipt of the sound data as input, data indicative of a sound type associated with the source sound; and 
 generating, based on the sound type, a control signal to control an action of the autonomous vehicle. 
   
     
     
         22 . The AV control system of  claim 21 , wherein the sound type comprises an ambulance signal sound, a police signal sound, or a fire engine signal sound. 
     
     
         23 . The AV control system of  claim 21 , wherein the sound type is determined based on one or more patterns associated with the superposition of sound waves being within a predetermined range of similarity to a sound profile associated with a known sound. 
     
     
         24 . The AV control system of  claim 21 , comprising receiving, as another output of the machine-learned model and based on receipt of the sound data as input, data indicative of a source location associated with the source sound, and wherein the control signal is further based on the source location. 
     
     
         25 . The AV control system of  claim 24 , wherein the control signal indicates that the autonomous vehicle is to pull to a side of a road based on a determination that the source location is approaching the autonomous vehicle. 
     
     
         26 . The AV control system of  claim 24 , wherein the control signal indicates the autonomous vehicle is to continue along its planned course based on a determination that the source location is moving farther away from the autonomous vehicle. 
     
     
         27 . The AV control system of  claim 24 , wherein the data indicative of the source location comprises at least one of:
 a distance of the source location from the plurality of microphones;   a direction of the source location from the plurality of microphones; or   a geographic location identified for the source location.   
     
     
         28 . The AV control system of  claim 24 , wherein the source location is determined based on a triangulation of the source sound received by the plurality of microphones. 
     
     
         29 . The AV control system of  claim 24 , wherein the source location is determined based on differences among at least one of constructive interference or destructive interference of the source sound received by the plurality of microphones. 
     
     
         30 . The AV control system of  claim 21 , the machine-learned model having been further trained to generate an amplified source sound based on the synchronized source sound, and to determine the sound type associated with the source sound based on the amplified source sound. 
     
     
         31 . The AV control system of  claim 30 , the machine-learned model having been further trained to recognize the source sound by analyzing at least one of a phase or an amplitude of the source sound received by the plurality of microphones to determine time differences in receiving the source sound by the plurality of microphones, and to determine a synchronized source sound comprising a superposition of sound waves associated with the source sound received by the plurality of microphones. 
     
     
         32 . The AV control system of  claim 21 , the machine-learned model having been further trained to filter background sounds from the sound data provided as input to the machine-learned model. 
     
     
         33 . The AV control system of  claim 21 , wherein the generating of the control signal comprises generating an audio output identifying the sound type. 
     
     
         34 . The AV control system of  claim 21 , wherein the generating of the control signal comprises changing a position of one or more of the plurality of microphones. 
     
     
         35 . An autonomous vehicle comprising:
 one or more processors;   a plurality of microphones;   a memory comprising one or more tangible non-transitory computer-readable media, the memory storing computer-readable instructions that are executable by the one or more processors to cause the one or more processors to perform operations comprising:
 receiving sound data associated with a source sound received by the plurality of microphones; 
 providing the sound data as input to a machine-learned model having been trained to recognize the source sound by analyzing one or more features of the source sound received by the plurality of microphones; 
 receiving, as an output of the machine-learned model and based on receipt of the sound data as input, data indicative of a sound type associated with the source sound; and 
 generating, based on the sound type, a control signal to control an action of the autonomous vehicle. 
   
     
     
         36 . The autonomous vehicle of  claim 35 , wherein:
 the plurality of microphones are configured to receive the source sound in a three-hundred and sixty degree radius around the autonomous vehicle; and   the one or more features comprise at least one of a phase or an amplitude of the source sound.   
     
     
         37 . The autonomous vehicle of  claim 35 , wherein a first microphone and a second microphone of the plurality of microphones are at least one meter apart or arranged in an orientation in which a sound detecting component of the first microphone is at least perpendicular to a sound detecting component of the second microphone. 
     
     
         38 . A method comprising:
 receiving sound data associated with a source sound received by a plurality of microphones positioned on an autonomous vehicle;   providing the sound data as input to a machine-learned model having been trained to recognize the source sound by analyzing one or more features of the source sound received by the plurality of microphones;   receiving, as an output of the machine-learned model and based on receipt of the sound data as input, data indicative of a sound type associated with the source sound; and   generating, based on the sound type, a control signal to control an action of the autonomous vehicle.   
     
     
         39 . The method of  claim 38 , comprising providing the control signal as an input to a motion planning system of the autonomous vehicle. 
     
     
         40 . The method of  claim 38 , wherein the action comprises at least one of: (i) modifying a velocity of the autonomous vehicle, (ii) modifying an acceleration of the autonomous vehicle, or (iii) modifying a travel path of the autonomous vehicle.