Intelligent information capturing in sound devices

Abstract

Sound devices such as hearing aids and headphones configured for intelligent information capturing are disclosed herein. In one embodiment, a sound device is a hearing aid or a noise-canceling headphone. The sound device includes a microphone, a speaker, a processor, and a memory containing a set of sound models each corresponding to a known sound. Upon receiving a digital sound signal representing an ambient sound captured via the microphone, the sound device can determine whether the digital sound signal includes a signal profile that matches the sound signature of one of the sound models stored in the memory. In response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, the sound device can output, via the speaker, an audio message to the user identifying the known sound while suppressing the captured ambient sound from the environment.

1 . A method of intelligent information capturing by a sound device having a microphone, a speaker, a memory, and a processor operatively coupled to one another, the memory containing records of sound models each corresponding to a known sound and having a sound signature, wherein the method comprising:
 capturing, via the microphone, an ambient sound from an environment in which a user wearing the sound device is in, the captured ambient sound having a background noise of a first frequency range represented by data of a digital sound signal and a target sound of a second frequency range;   suppressing, with the sound device, the background noise in the captured ambient sound of the first frequency range while allowing the target sound at the second frequency range to pass through the sound device; and   while suppressing the background noise at the sound device,
 determining, with the processor, whether at least a part of the digital sound signal of the background noise has a signal profile that matches the sound signature of one of the sound models stored in the memory, the signal profile including one or more of a range of frequency, a pattern of frequency, a range of frequency distribution, or a pattern of frequency distribution of the digital sound signal; and 
 in response to determining that the digital sound signal has a sound profile that matches the sound signature of one of the sound models, outputting, via the speaker of the sound device, an audio message to the user identifying the known sound corresponding to the one of the sound models while suppressing the background noise at the first frequency range in the captured ambient sound from the environment. 
   
     
     
         2 . The method of  claim 1  wherein:
 the one of the sound models also includes a text message corresponding to the known sound; 
 the method further includes performing, at the processor, text to speech conversion of the text message to generate the audio message; and 
 wherein outputting the audio message includes outputting, via the speaker, the generated audio message to the user. 
 
     
     
         3 . The method of  claim 1  wherein:
 the one of the sound models also includes a sound file corresponding to the known sound; and 
 outputting the audio message includes playing, via the speaker, the sound file to produce the audio message to the user. 
 
     
     
         4 . The method of  claim 1  wherein:
 the known sound includes one of an approaching vehicle, an emergency siren, or an alarm; and 
 outputting the audio message includes outputting, via the speaker, an audio warning regarding the approaching vehicle, the emergency siren, or the alarm while at least partially suppressing a sound made by the approaching vehicle, the emergency siren, or the alarm. 
 
     
     
         5 . The method of  claim 1 , further comprising:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models,   determining, at the processor, whether the known sound corresponding to the one of the sound models is human speech; and   in response to determining that the known sound is human speech, performing speech to text conversion of the digital sound signal to derive a text string.   
     
     
         6 . The method of  claim 1 , further comprising:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determining, at the processor, whether the known sound corresponding to the one of the sound models is human speech; and   in response to determining that the known sound is human speech,   performing speech to text conversion of the digital sound signal to derive a text string;
 determining whether the text string represents a command to the sound device; and 
 in response to determining that the text string represents a command to the sound device, executing the command with the processor. 
   
     
     
         7 . The method of  claim 1 , further comprising:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determining, at the processor, whether the known sound corresponding to the one of the sound models is human speech; and   in response to determining that the known sound is human speech,
 performing speech to text conversion of the digital sound signal to derive a text string; 
 determining whether the text string represents a command to a digital assistant; and 
 in response to determining that the text string represents a command to a digital assistant, transmitting the command to the digital assistant via a computer network. 
   
     
     
         8 . The method of  claim 1 , further comprising:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determining, at the processor, whether the known sound corresponding to the one of the sound models is human speech;   in response to determining that the known sound is human speech, performing speech to text conversion of the digital sound signal to derive a text string; and   wherein outputting the audio message includes:
 determining whether the text string includes one or more keywords pre-identified by the user; and 
 in response to determining that the text string includes one or more keywords pre-identified by the user, outputting the audio message to the user informing the user that the one or more keywords have been detected. 
   
     
     
         9 . The method of  claim 1  wherein:
 the captured ambient sound has multiple frequency ranges with corresponding amplitude; and 
 the method further includes:
 modifying, the amplitude of one or more of the multiple frequency ranges at the captured ambient sound; and 
 outputting the captured ambient sound, via the speaker, with the modified amplitude at one or more of the multiple frequency ranges along with the audio message. 
 
 
     
     
         10 . A sound device, comprising:
 a microphone;   a speaker;   a processor operatively coupled to the microphone and speaker; and   a memory containing data representing a set of sound models each corresponding to a known sound and having a sound signature, wherein the memory also contains instructions executable by the processor to cause the sound device to:
 receive a digital sound signal representing a background noise of a first frequency range of an ambient sound captured via the microphone from an environment in which a user wearing the sound device is in, the ambient sound also including a target sound of a second frequency range; and 
 in response to receiving the digital sound signal, suppress the background noise of the first frequency range in the captured ambient sound while allowing the target sound at the second frequency range to pass through the sound device;
 determine whether the digital sound signal representing the background noise includes a signal profile that matches the sound signature of one of the sound models stored in the memory, the signal profile including one or more of a range of frequency, a pattern of frequency, a range of frequency distribution, or a pattern of frequency distribution of the digital sound signal; and 
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, output, via the speaker of the sound device, an audio message to the user identifying the known sound corresponding to the one of the sound models while suppressing the background noise of the first frequency range in the captured ambient sound from the environment. 
 
   
     
     
         11 . The sound device of  claim 10  wherein:
 the known sound includes one of an approaching vehicle, an emergency siren, or an alarm; and 
 to output the audio message includes to output, via the speaker, an audio warning regarding the approaching vehicle, the emergency siren, or the alarm while at least partially suppressing a sound made by the approaching vehicle, the emergency siren, or the alarm. 
 
     
     
         12 . The sound device of  claim 10  wherein the memory includes additional instructions executable by the processor to cause the sound device to:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models,
 determine whether the known sound corresponding to the one of the sound models is human speech; and 
 in response to determining that the known sound is human speech, perform speech to text conversion of the digital sound signal to derive a text string. 
 
 
     
     
         13 . The sound device of  claim 10  wherein the memory includes additional instructions executable by the processor to cause the sound device to:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determine whether the known sound corresponding to the one of the sound models is human speech; and 
 in response to determining that the known sound is human speech, perform speech to text conversion of the digital sound signal to derive a text string;
 determine whether the text string represents a command to the sound device; and 
 in response to determining that the text string represents a command to the sound device, execute the command with the processor. 
 
 
     
     
         14 . The sound device of  claim 10  wherein the memory includes additional instructions executable by the processor to cause the sound device to:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determine whether the known sound corresponding to the one of the sound models is human speech; and 
 in response to determining that the known sound is human speech, perform speech to text conversion of the digital sound signal to derive a text string;
 determine whether the text string represents a command to a digital assistant; and 
 in response to determining that the text string represents a command to a digital assistant, transmit the command to the digital assistant via a computer network. 
 
 
     
     
         15 . The sound device of  claim 10  wherein the memory includes additional instructions executable by the processor to cause the sound device to:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determine whether the known sound corresponding to the one of the sound models is human speech; 
 in response to determining that the known sound is human speech, perform speech to text conversion of the digital sound signal to derive a text string; and 
 wherein to output the audio message includes to:
 determine whether the text string includes one or more keywords pre-identified by the user; and 
 in response to determining that the text string includes one or more keywords pre-identified by the user, output the audio message to the user informing the user that the one or more keywords have been detected. 
 
 
     
     
         16 . The sound device of  claim 10  wherein:
 the captured ambient sound has multiple frequency ranges with corresponding amplitude; and 
 the memory includes additional instructions executable by the processor to cause the sound device to:
 modify, the amplitude of one or more of the multiple frequency ranges at the captured ambient sound; and 
 output the captured ambient sound, via the speaker, with the modified amplitude at one or more of the multiple frequency ranges along with the audio message. 
 
 
     
     
         17 . The sound device of  claim 10  wherein:
 the captured ambient sound has multiple frequency ranges with corresponding first amplitude; and 
 the memory includes additional instructions executable by the processor to cause the sound device to:
 generate another digital sound signal having the multiple frequency ranges with corresponding second amplitude opposite the first amplitude of the captured ambient sound; and 
 output, via the speaker, the generated another digital sound signal along with the audio message, thereby at least partially canceling the captured ambient sound. 
 
 
     
     
         18 . A method of intelligent information capturing by a computing device having a processor and a memory operatively coupled to the processor, the memory containing records of sound models each corresponding to a known sound with a sound signature, wherein the method comprising:
 receiving, a digital sound signal representing a background noise of a first frequency range of an ambient sound captured using a microphone from an environment in which a user is in, the ambient sound also including a target sound of a second frequency range;   suppressing the background noise of the first frequency range in the captured ambient sound while allowing the target sound at the second frequency range to pass through the computing device;   determining, with the processor, whether the received digital sound signal representing the background noise has a signal profile that matches the sound signature of one of the sound models stored in the memory, the signal profile including one or more of a range of frequency, a pattern of frequency, a range of frequency distribution, or a pattern of frequency distribution of the digital sound signal; and   in response to determining that the received digital sound signal has a signal profile that matches the sound signature of one of the sound models, transmitting, a command to a speaker, the command instructing the speaker to playback an audio message to the user identifying the known sound corresponding to the one of the sound models while suppressing the background noise of the first frequency range in the ambient sound.   
     
     
         19 . The method of  claim 18 , further comprising:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determining, at the processor, whether the known sound corresponding to the one of the sound models is human speech; and   in response to determining that the known sound is human speech,
 performing speech to text conversion of the digital sound signal to derive a text string; 
 determining whether the text string represents a command to the sound device; and 
 in response to determining that the text string represents a command to the sound device, executing the command with the processor. 
   
     
     
         20 . The method of  claim 18 , further comprising:
 in response to determining that the digital sound signal has a signal profile that matches the sound signature of one of the sound models, determining, at the processor, whether the known sound corresponding to the one of the sound models is human speech;   in response to determining that the known sound is human speech, performing speech to text conversion of the digital sound signal to derive a text string; and   wherein outputting the audio message includes:
 determining whether the text string includes one or more keywords pre-identified by the user; and 
 in response to determining that the text string includes one or more keywords pre-identified by the user, outputting the audio message to the user informing the user that the one or more keywords have been detected.