Somatic, auditory and cochlear communication system and method

Abstract

Methods and devices ( 620, 1220, 1410 ) to deliver a tactile speech analog to a person's skin ( 404, 604, 1082, 1440 ) providing a silent, invisible, hands-free, eyes-free, and ears-free way to receive and directly comprehend electronic communications ( 1600 b ). Embodiments include an alternative to hearing aids that will enable people with hearing loss to better understand speech. A device ( 1410 ), worn like watch or bracelet, supplements a person's remaining hearing to help identify and disambiguate those sounds he or she can not hear properly. Embodiments for hearing aids ( 620 ) and hearing prosthetics ( 1220 ) are also described.

1 . A method of transforming a sequence of symbols representing phonemes into a sequence of arrays of nerve stimuli, the method comprising:
 establishing a correlation between each member of a phoneme symbol set and an assignment of one or more channels of a multi-electrode array;   accessing a sequence of phonetic symbols corresponding to a message; and   activating a sequence of one or more electrodes corresponding to each phonetic symbol of the message identified by the correlation.   
     
     
         2 . The method of  claim 1 , wherein the phonetic symbols belong to one of SAMPA, Kirshenbaum, or IPA Unicode digital character sets. 
     
     
         3 . The method of  claim 1 , wherein the symbols belong to the cmudict phoneme set. 
     
     
         4 . The method of  claim 1 , wherein the correlation is a one to one correlation. 
     
     
         5 . The method of  claim 1 , wherein activating a sequence of one or more electrodes includes an energizing period for each electrode, wherein the energizing period comprises a begin time parameter and an end time parameter. 
     
     
         6 . The method of  claim 5 , wherein the begin time parameter is representative of a time from an end of components of a previous energizing period of a particular electrode. 
     
     
         7 . The method of  claim 1 , wherein the electrodes are associated with a hearing prosthesis. 
     
     
         8 . The method of  claim 1 , wherein the hearing prosthesis comprises a cochlear implant. 
     
     
         9 . A method of processing a sequence of spoken words into a sequence of sounds, the method comprising:
 converting a sequence of spoken words into electrical signals;   digitizing the electrical signals representative of the speech sounds;   transforming the speech sounds into digital symbols representing corresponding phonemes;   transforming the symbols representing the corresponding phonemes into sound representations; and   transforming the sound representations into sounds.   
     
     
         10 . The method of  claim 9 , wherein transforming the symbols representing the phonemes into sound representations comprises:
 accessing a data structure configured to map phonemes to sound representations;   locating the symbols representing the corresponding phonemes in the data structure; and   mapping the phonemes to sound representations.   
     
     
         11 . The method of  claim 10 , additionally comprising creating the data structure, comprising:
 identifying phonemes corresponding to a language used by a user of the method;   establishing a set of allowed sound frequencies;   generating a correspondence mapping the identified phonemes to the set of allowed sound frequencies such that each constituent phoneme of the identified phonemes is assigned a subset of one or more frequencies from the set of allowed sound frequencies; and   mapping each constituent phoneme of the identified phonemes to a set of one or more sounds.   
     
     
         12 . The method of  claim 11 , wherein establishing a set of allowed sound frequencies comprises selecting a set of sound frequencies that are in a hearing range of the user. 
     
     
         13 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises an initial frequency parameter. 
     
     
         14 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises a begin time parameter. 
     
     
         15 . The method of  claim 14 , wherein the begin time parameter is representative of a time from an end of components of a previous sound representation. 
     
     
         16 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises an end time parameter. 
     
     
         17 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises a power parameter. 
     
     
         18 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises a power shift parameter. 
     
     
         19 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises a frequency shift parameter. 
     
     
         20 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises a pulse rate parameter. 
     
     
         21 . The method of  claim 11 , wherein each sound of the set of one more sounds comprises a duty cycle parameter. 
     
     
         22 . A method of processing a sequence of spoken words into a sequence of nerve stimuli, the method comprising:
 converting a sequence of spoken words into electrical signals;   digitizing the electrical signals representative of the speech sounds;   transforming the speech sounds into digital symbols representing corresponding phonemes;   transforming the symbols representing the corresponding phonemes into stimulus definitions; and   transforming the stimulus definitions into a sequence of nerve stimuli.   
     
     
         23 . The method of  claim 22 , wherein the nerve stimuli are associated with a hearing prosthesis. 
     
     
         24 . The method of  claim 23 , wherein the hearing prosthesis comprises a cochlear implant. 
     
     
         25 . The method of  claim 22 , wherein the nerve stimuli are associated with a skin interface. 
     
     
         26 . The method of  claim 25 , wherein the skin interface is located on the wrist and/or hand of the user. 
     
     
         27 . The method of  claim 25 , wherein the skin interface is located on the ankle and/or foot of the user. 
     
     
         28 . The method of  claim 22 , wherein the nerve stimuli are mechanical. 
     
     
         29 . The method of  claim 22 , wherein the nerve stimuli are electrical. 
     
     
         30 . The method of  claim 22 , wherein transforming the symbols representing the phonemes into stimulus definitions comprises:
 accessing a data structure configured to map phonemes to stimulus definitions;   locating the symbols representing the corresponding phonemes in the data structure; and   mapping the phonemes to stimulus definitions.   
     
     
         31 . The method of  claim 22 , wherein the stimulus definitions comprise sets of one or more stimuli. 
     
     
         32 . The method of  claim 31 , wherein the sets of one or more stimuli correspond to one or more locations on the skin. 
     
     
         33 . The method of  claim 31 , wherein the sets of one or more stimuli correspond to one or more locations in the cochlea. 
     
     
         34 . The method of  claim 31 , wherein each stimulus of the sets of one or more stimuli comprises a begin time parameter. 
     
     
         35 . The method of  claim 34 , wherein the begin time parameter is representative of a time from an end of components of a previous stimulus definition. 
     
     
         36 . The method of  claim 31 , wherein each stimulus of the sets of one or more stimuli comprises an end time parameter. 
     
     
         37 . A method of transforming a sequence of symbols representing phonemes into a sequence of arrays of nerve stimuli, the method comprising:
 establishing a correlation between each member of a phoneme symbol set and an assignment of one or more channels of a multi-stimulator array;   accessing a sequence of phonetic symbols corresponding to a message; and   activating a sequence of one or more stimulators corresponding to each phonetic symbol of the message identified by the correlation.   
     
     
         38 . A method of  claim 37 , wherein the stimulators are vibrators affixed to the user's skin. 
     
     
         39 . A method of  claim 37 , wherein the phonetic symbols belong to one of SAMPA, Kirshenbaum, or IPA Unicode digital character sets. 
     
     
         40 . A method of  claim 37 , wherein the symbols belong to the cmudict phoneme set. 
     
     
         41 . A method of  claim 37 , wherein the correlation is a one to one correlation. 
     
     
         42 . The method of  claim 37 , wherein activating a sequence of one or more stimulators includes an energizing period for each stimulator, wherein the energizing period comprises a begin time parameter and an end time parameter. 
     
     
         43 . The method of  claim 42 , wherein the begin time parameter is representative of a time from an end of components of a previous energizing period of a particular stimulator. 
     
     
         44 . A method of training a user, the method comprising:
 providing a set of somatic stimulations to a user, wherein the set of somatic stimulations is indicative of a plurality of phonemes, and wherein the phonemes are based at least in part on an audio communication;   providing the audio communication concurrently to the user with the plurality of phonemes; and   selectively modifying at least portions of the audio communication to the user during the providing of the set of somatic stimulations to the user.   
     
     
         45 . The method of  claim 44 , wherein selectively modifying at least portions of the audio communication comprises reducing an audio property of the audio communication. 
     
     
         46 . The method of  claim 45 , wherein the audio property comprises a volume of the audio. 
     
     
         47 . The method of  claim 45 , wherein the audio property comprises omitting selected words from the audio. 
     
     
         48 . The method of  claim 45 , wherein the audio property comprises attenuating a volume of selected words from the audio. 
     
     
         49 . The method of  claim 45 , wherein the audio property comprises omitting selected phonemes from the audio. 
     
     
         50 . The method of  claim 45 , wherein the audio property comprises attenuating a volume of selected phonemes from the audio. 
     
     
         51 . The method of  claim 45 , wherein the audio property comprises omitting selected sound frequencies from the audio. 
     
     
         52 . The method of  claim 45 , wherein the audio property comprises attenuating a volume of selected sound frequencies from the audio. 
     
     
         53 . A method of training a user, the method comprising:
 providing a set of somatic stimulations to a user, wherein the set of somatic stimulations is indicative of a plurality of phonemes, and wherein the phonemes are based at least in part on an audiovisual communication;   providing the audiovisual communication concurrently to the user with the plurality of phonemes; and   selectively modifying at least portions of the audiovisual communication to the user during the providing of the set of somatic stimulations to the user.   
     
     
         54 . The method of  claim 53 , wherein selectively modifying at least portions of the audiovisual communication comprises reducing an audio or video property of the audiovisual communication. 
     
     
         55 . The method of  claim 54 , wherein the audio property comprises a volume of the audio. 
     
     
         56 . The method of  claim 54 , wherein the audio property comprises omitting selected words from the audio. 
     
     
         57 . The method of  claim 54 , wherein the audio property comprises attenuating a volume of selected words from the audio. 
     
     
         58 . The method of  claim 54 , wherein the audio property comprises omitting selected phonemes from the audio. 
     
     
         59 . The method of  claim 54 , wherein the audio property comprises attenuating a volume of selected phonemes from the audio. 
     
     
         60 . The method of  claim 54 , wherein the audio property comprises omitting selected sound frequencies from the audio. 
     
     
         61 . The method of  claim 54 , wherein the audio property comprises attenuating a volume of selected sound frequencies from the audio. 
     
     
         62 . The method of  claim 54 , wherein the video property comprises a presence or brightness of the video. 
     
     
         63 . A system for processing a sequence of spoken words into a sequence of sounds, the system comprising:
 a first converter configured to digitize electrical signals representative of a sequence of spoken words;   a speech recognizer configured to receive the digitized electrical signals and generate a sequence of phonemes representative of the sequence of spoken words;   a mapper configured to assign sound sets to phonemes utilizing an audiogram so as to generate a map;   a transformer configured to receive the sequence of phonemes representative of the sequence of spoken words and the map and to generate a sequence of sound representations corresponding to the sequence of phonemes; and   a second converter configured to convert the sequence of sound representations into a sequence of audible sounds.   
     
     
         64 . The system of  claim 63 , wherein the map is a user-specific map based on a particular user's audiogram. 
     
     
         65 . A system for processing a sequence of spoken words into a sequence of sounds, the system comprising:
 a first converter configured to digitize electrical signals representative of a sequence of spoken words;   a speech recognizer configured to receive the digitized electrical signals and generate a sequence of phonemes representative of the sequence of spoken words;   a data structure comprising sound sets mapped to phonemes;   a transformer configured to receive the sequence of phonemes representative of the sequence of spoken words and the data structure and to generate a sequence of sound representations corresponding to the sequence of phonemes; and   a second converter configured to convert the sequence of sound representations into a sequence of audible sounds.   
     
     
         66 . The system of  claim 65 , wherein the data structure is generated utilizing a user's audiogram. 
     
     
         67 . A system for processing a sequence of spoken words into a sequence of nerve stimuli, the system comprising:
 a converter configured to digitize electrical signals representative of a sequence of spoken words;   a speech recognizer configured to receive the digitized electrical signals and generate a sequence of phonemes representative of the sequence of spoken words;   a mapper configured to assign nerve stimuli arrays to phonemes utilizing an audiogram so as to generate a map; and   a transformer configured to receive the sequence of phonemes representative of the sequence of spoken words and the map and to generate a sequence of stimulus definitions corresponding to the sequence of phonemes.   
     
     
         68 . The system of  claim 67 , additionally comprising:
 a receiver configured to convert the sequence of stimulus definitions into electrical waveforms; and   an electrode array configured to receive the electrical waveforms.   
     
     
         69 . The system of  claim 68 , wherein the electrode array is surgically placed in the user's cochlea. 
     
     
         70 . The system of  claim 67 , wherein the sequence of stimulus definitions comprise digital representations of nerve stimulation patterns. 
     
     
         71 . A system for processing a sequence of spoken words into a sequence of nerve stimuli, the system comprising:
 a converter configured to digitize electrical signals representative of a sequence of spoken words;   a speech recognizer configured to receive the digitized electrical signals and generate a sequence of phonemes representative of the sequence of spoken words;   a data structure comprising nerve stimuli arrays mapped to phonemes; and   a transformer configured to receive the sequence of phonemes representative of the sequence of spoken words and the data structure and to generate a sequence of stimulus definitions corresponding to the sequence of phonemes.   
     
     
         72 . The system of  claim 71 , wherein the data structure is generated utilizing a user's audiogram. 
     
     
         73 . The system of  claim 71 , additionally comprising:
 a receiver configured to convert the sequence of stimulus definitions into electrical waveforms; and   an electrode array configured to receive the electrical waveforms.   
     
     
         74 . The system of  claim 73 , wherein the electrode array is surgically placed in the user's cochlea. 
     
     
         75 . The system of  claim 71 , wherein the sequence of stimulus definitions comprise digital representations of nerve stimulation patterns. 
     
     
         76 . A system for processing a sequence of spoken words into a sequence of nerve stimuli, the system comprising:
 a processor configured to generate a sequence of phonemes representative of a sequence of spoken words and to transform the sequence of phonemes using a data structure comprising nerve stimuli arrays mapped to phonemes to produce a sequence of stimulus definitions corresponding to the sequence of phonemes; and   an electrode array configured to play the sequence of stimulus definitions.   
     
     
         77 . The system of  claim 76 , wherein the data structure is generated utilizing a user's audiogram. 
     
     
         78 . The system of  claim 76 , wherein the electrode array comprises a converter configured to convert the sequence of stimulus definitions into electrical waveforms. 
     
     
         79 . The system of  claim 76 , wherein the electrode array is surgically placed in the user's cochlea. 
     
     
         80 . The system of  claim 76 , wherein the electrode array comprises a plurality of mechanical stimulators. 
     
     
         81 . The system of  claim 76 , wherein the electrode array comprises a plurality of electrodes. 
     
     
         82 . The system of  claim 76 , wherein the sequence of stimulus definitions comprise digital representations of nerve stimulation patterns. 
     
     
         83 . A system for processing a sequence of spoken words into a sequence of sounds, the system comprising:
 a processor configured to generate a sequence of phonemes representative of the sequence of spoken words and to transform the sequence of phonemes using a data structure comprising sound sets mapped to phonemes to produce sound representations corresponding to the sequence of phonemes; and   a converter configured to convert the sound representations into audible sounds.   
     
     
         84 . The system of  claim 83 , wherein the data structure is generated utilizing a user's audiogram. 
     
     
         85 . A system for processing a sequence of text into a sequence of sounds, the system comprising:
 a first converter configured to receive a sequence of text and generate a sequence of phonemes representative of the sequence of text;   a mapper configured to assign sound sets to phonemes utilizing a hearing audiogram so as to generate a map;   a transformer configured to receive the sequence of phonemes representative of the sequence of text and the map and to generate sound representations corresponding to the sequence of phonemes; and   a second converter configured to convert the sound representations into audible sounds.   
     
     
         86 . The system of  claim 85 , wherein the hearing audiogram is representative of a normal human hearing range. 
     
     
         87 . The system of  claim 85 , wherein the hearing audiogram is representative of a hearing range for a specific individual. 
     
     
         88 . A system for processing a sequence of text into a sequence of sounds, the system comprising:
 a text converter configured to receive a sequence of text and generate a sequence of phonemes representative of the sequence of text;   a data structure comprising sound sets mapped to phonemes;   a transformer configured to receive the sequence of phonemes representative of the sequence of text and the data structure and to generate sound representations corresponding to the sequence of phonemes; and   a second converter configured to convert the sound representations into audible sounds.   
     
     
         89 . The system of  claim 88 , wherein the data structure is generated utilizing a user's audiogram. 
     
     
         90 . A system for processing a sequence of text into a sequence of nerve stimuli, the system comprising:
 a converter configured to receive a sequence of text and generate a sequence of phonemes representative of the sequence of text;   a data structure comprising nerve stimuli arrays mapped to phonemes; and   a transformer configured to receive the sequence of phonemes representative of the sequence of text and the data structure and to generate a sequence of stimulus definitions corresponding to the sequence of phonemes.   
     
     
         91 . The system of  claim 90 , wherein the data structure is generated utilizing a user's abilities. 
     
     
         92 . The system of  claim 91 , wherein the user's abilities comprise useable channels of a cochlear implant of the user. 
     
     
         93 . The system of  claim 91 , wherein the user's abilities comprise the ability to distinguish between two or more unique stimuli. 
     
     
         94 . A method of processing a sequence of text into a sequence of sounds, the method comprising:
 transforming the sequence of text into digital symbols representing corresponding phonemes;   transforming the symbols representing the corresponding phonemes into sound representations; and   transforming the sound representations into a sequence of sounds.   
     
     
         95 . A method of processing a sequence of text into a sequence of nerve stimuli, the method comprising:
 transforming the sequence of text into digital symbols representing corresponding phonemes;   transforming the symbols representing the corresponding phonemes into stimulus definitions; and   transforming the stimulus definitions into a sequence of nerve stimuli.   
     
     
         96 . The method of  claim 95 , wherein the nerve stimuli are associated with a cochlear implant. 
     
     
         97 . The method of  claim 95 , wherein the nerve stimuli are associated with a skin interface. 
     
     
         98 . The method of  claim 97 , wherein the skin interface is located on the wrist and/or hand of the user. 
     
     
         99 . The method of  claim 95 , wherein transforming the symbols representing the phonemes into stimulus definitions comprises:
 accessing a data structure configured to map phonemes to stimulus definitions;   locating the symbols representing the corresponding phonemes in the data structure; and   mapping the phonemes to stimulus definitions.   
     
     
         100 . A method of creating a data structure configured to transform symbols representing phonemes into sound representations, the method comprising:
 identifying phonemes corresponding to a language utilized by a user;   establishing a set of allowed sound frequencies;   generating a correspondence mapping the identified phonemes to the set of allowed sound frequencies such that each constituent phoneme of the identified phonemes is assigned a subset of one or more frequencies from the set of allowed sound frequencies; and   mapping each constituent phoneme of the identified phonemes to a set of one or more sounds.   
     
     
         101 . The method of  claim 100 , wherein establishing a set of allowed sound frequencies comprises selecting a set of sound frequencies that are in a hearing range of the user. 
     
     
         102 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises an initial frequency parameter. 
     
     
         103 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises a begin time parameter. 
     
     
         104 . The method of  claim 103 , wherein the begin time parameter is representative of a time from an end of components of a previous sound representation. 
     
     
         105 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises an end time parameter. 
     
     
         106 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises a power parameter. 
     
     
         107 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises a power shift parameter. 
     
     
         108 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises a frequency shift parameter. 
     
     
         109 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises a pulse rate parameter. 
     
     
         110 . The method of  claim 100 , wherein each sound of the set of one more sounds comprises a duty cycle parameter.