Monitoring Environmental Noise and Data Packets to Display a Transcription of Call Audio

Abstract

Various embodiments provide a communication system that monitors environmental noise at a computing device. The communication system additionally monitors call audio received at the computing device. Based on the environmental noise and the received call audio, the communication system determines that a user of the computing device is unlikely to hear the received call audio when played back by the computing device. In response to determining that the user of the computing device is unlikely to hear the received call audio, the communication system visually displays a transcription of at least a portion of the received call audio.

1 . A method, comprising:
 monitoring, using a communication system, environmental noise at a computing device;   monitoring, using the communication system, call audio received at the computing device;   determining, using the communication system, that a user of the computing device is unlikely to hear the call audio received at the computing device;   responsive to determining that the user of the computing device is unlikely to hear the call audio received at the computing device, visually displaying a transcription of at least a portion of the call audio received at the computing device; and   stopping the visual display of the transcription of at least a portion of the call audio in response to determining that the user of the computing device is likely to hear the call audio received at the computing device.   
     
     
         2 . The method as recited in  claim 1 , wherein the monitoring the environmental noise is performed by monitoring background noise using a microphone of the computing device, the background noise being different from speech of the user. 
     
     
         3 . The method as recited in  claim 1 , wherein the call audio comprises audio of a voice call or a video call. 
     
     
         4 . The method as recited in  claim 1 , wherein determining that the user of the computing device is unlikely to hear the received call audio comprises determining that a level of the environmental noise satisfies a threshold level of environmental noise. 
     
     
         5 . The method as recited in  claim 1 , wherein determining that the user of the computing device is likely to hear the call audio received at the computing device comprises determining that a level of the environmental noise no longer satisfies a threshold level of environmental noise. 
     
     
         6 . The method as recited in  claim 4 , wherein the threshold level of environmental noise is a decibel level specified by a manufacturer of the computing device or a user of the computing device. 
     
     
         7 . The method as recited in  claim 4 , wherein the threshold level of environmental noise is dependent on a decibel level of the call audio received at the computing device. 
     
     
         8 . The method as recited in  claim 1 , wherein the call audio is received as a series of data packets at the computing device, the method further comprising monitoring, for each set of two sequential data packets of the series of data packets, a latency between the two sequential data packets. 
     
     
         9 . The method as recited in  claim 7 , wherein determining that the user of the computing device is unlikely to hear the received call audio comprises determining that the latency between two sequential data packets satisfies a threshold level of data packet latency. 
     
     
         10 . The method as recited in  claim 7 , wherein determining that the user of the computing device is likely to hear the call audio received at the computing device comprises determining that the latency between two sequential data packets fails to satisfy a threshold level of data packet latency. 
     
     
         11 . The method as recited in  claim 1 , further comprising generating, by the communication system, synthesized speech of the transcription of at least the portion of the call audio and playing back the synthesized speech while the transcription is displayed. 
     
     
         12 . One or more computer readable media storing computer-readable instructions which, when executed, perform operations comprising:
 monitoring call audio received in a series of multiple data packets at a computing device;   monitoring, a latency between sequential data packets in the series of multiple data packets;   determining that the latency between a set of two sequential data packets in the series of multiple data packets satisfies a data packet latency threshold; and   visually displaying a transcription of audio data included in the set of two sequential data packets in response to determining that the latency between the set of two sequential data packets satisfies the data packet latency threshold.   
     
     
         13 . The one or more computer readable media as recited in  claim 12 , further comprising visually displaying a transcription of audio data included in data packets received subsequent to the set of two sequential data packets until determining that the latency between the subsequently received data packets no longer satisfies the data packet latency threshold. 
     
     
         14 . The one or more computer readable media as recited in  claim 12 , wherein the data packet latency threshold comprises a threshold amount of time between received data packets that is specified by a manufacturer of the computing device or a user of the computing device. 
     
     
         15 . The one or more computer readable media as recited in  claim 12 , wherein visually displaying the transcription of audio data included in the set of two sequential data packets comprises extracting speech data from the set of two sequential data packets, converting the speech data to text, and displaying the text in real-time during the received audio call. 
     
     
         16 . The one or more computer readable media as recited in  claim 12 , further comprising determining that latencies between sequential data packets in the series of multiple data packets is approaching the data packet latency threshold and notifying a user of the computing device that the visual display of the transcription of audio data is about to begin. 
     
     
         17 . The one or more computer readable media as recited in  claim 16 , wherein notifying the user of the computing device that the visual display of the transcription of audio data is about to begin comprises at least one of vibrating the computing device, playing an audible tone from the computing device, or displaying a visual notification at a device of the computing device. 
     
     
         18 . A computing device comprising:
 one or more processors; and   one or more computer readable storage media storing computer-readable instructions which, when executed, perform operations comprising:
 monitoring, using a communication system, environmental noise at a computing device; 
 monitoring, using the communication system, call audio received at the computing device; 
 determining, using the communication system, that a user of the computing device is unlikely to hear the call audio received at the computing device; 
 responsive to determining that the user of the computing device is unlikely to hear the call audio received at the computing device, generating, using the communication system, synthesized speech of the call audio and playing back the synthesized speech at the computing device. 
   
     
     
         19 . The computing device as recited in  claim 18 , wherein determining that the level of the environmental noise no longer satisfies the environmental noise level threshold is performed in response to receiving user input at the computing device indicating that a user of the device can hear the call audio from the computing device. 
     
     
         20 . The computing device as recited in  claim 18 , wherein determining that the user of the computing device is unlikely to hear the received call audio comprises:
 determining that a level of the environmental noise satisfies a threshold level of environmental noise; or   determining that a latency between two sequential data packets of the series of data packets satisfies a threshold level of data packet latency.