Representation of contextual information by projecting different participants' audio from different positions in a 3D soundscape

Abstract

An overlay network platform facilitates a multi-party conference. End users participate in the conference using client-based web browser software. According to this disclosure, an enhanced “audio” experience for a user participating in the conference is provided by rendering different participants' audio in the conference at different positions in a three-dimensional (3D) space. In operation, and given a set of audio streams that comprise the conference and that are being received, a three-dimensional (3D) position for each audio stream in a 3D soundscape is computed. In one embodiment, the 3D position for each audio stream is computed by identifying a target area region of space, and then determining the 3D position for each stream in the target area region of space. The relative 3D positions may be determined based on contextual information related to the content of an audio stream, as well as a computed “relevance” of one stream versus another. The streams are then rendered to the listener based on the computed 3D positions.

Having described the subject matter, what we claim follows below. 
     
         1 . A method carried out at a given client machine, the given client machine being one of a set of client machines participating in a multi-party conference using an overlay network platform, the multi-party conference comprising a set of audio streams received from multiple users each associated with a respective client machine, comprising:
 computing a position in a soundscape for each of the received audio streams by identifying a target area region of space in front of the user of the given client machine, and identifying a relative position of each of a set of remote users in the multi-party conference as follows: (a) applying a function to a unique identifier associated with the remote user of the set of remote users to generate a number, (b) based on the number generated for each of the set of remote users, assigning each of the remote users a relative left-right position within the target area region of space in front of the user;   rendering the audio streams of the remote users at the left-right position computed;   as the multi-party conference continues, selectively adjusting a front-back position of at least one of the remote users relative to the assigned left-right positions based on a context associated with a given user or a given audio stream; and   re-rendering the audio streams of the remote users including at the adjusted front-back position.   
     
     
         2 . The method as described in  claim 1  wherein, with respect to a location of a listener, the target area region is defined by a 45° cone. 
     
     
         3 . The method as described in  claim 1  wherein the position of the at least one of the remote users also is based on a position on a display screen where a video of the speaker that generates that stream is located. 
     
     
         4 . The method as described in  claim 1  wherein, with respect to the relative left-right positions, audio streams of the remote users are evenly spaced within the target area region of space. 
     
     
         5 . The method as described in  claim 4  wherein the audio streams are evenly spaced by applying a hash function to the unique identifier, the unique identifier being the remote user's name. 
     
     
         6 . The method as described in  claim 1  further including re-adjusting the front-back position of the at least one of the remote users relative to the assigned left-right positions. 
     
     
         7 . The method as described in  claim 1  wherein the audio streams are rendered in a browser. 
     
     
         8 . The method as described in  claim 7  wherein the browser renders the audio streams using an application programming interface (API). 
     
     
         9 . The method as described in  claim 8  wherein the API is a Hypertext Markup Language Version 5 (HTML5) Web Audio Application Programming Interface.