Spatial audio teleconferencing

Abstract

Embodiments of the present invention allow participants in a networked virtual world to converse naturally as in a real-life situation. Each participant can hear “nearby” participants localized according to their positions in the virtual world, desirably with reverberation and environmental audio effects derived from properties of the virtual world.

What is claimed is: 
     
         1 .- 25 . (canceled) 
     
     
         26 . A spatial sound conferencing server comprising:
 an input processor for receiving audio and position information over a network from a plurality of network clients;   a conference processor for spatializing the audio according to the received position and orientation information;   a connection manager for defining a client-specific conference for each network client, each client-specific conference including a mix of audio from other network clients; and   an output interface for transmitting, as spatialized audio, the client-specific conferences to the corresponding network clients over the network.   
     
     
         27 . The conference server of  claim 26  wherein:
 each client corresponds to a user; 
 position information corresponding to each user is represented in a world model; and 
 the conference processor is configured to spatialize each client-specific conference based on a listener position and orientation relative to the positions of the users. 
 
     
     
         28 . The conference server of  claim 27  wherein:
 for a first client, the associated conference comprises audio from other clients corresponding to users who are proximate to the first-client user in the world model. 
 
     
     
         29 . The conference server of  claim 27  wherein the conference processor is configured to spatialize audio by:
 scaling by a gain inversely proportional to the distance between a client user and a sound source to model distance cues; 
 filtering by a pair of HRTFs to encode localization cues; and 
 summing the resulting stereo signal with stereo reverberation. 
 
     
     
         30 . The conference server of  claim 29  wherein:
 the server is configured to receive audio from a plurality of monophonic and stereophonic capable clients; 
 the conference processor is configured, for each client, to:
 store a monophonic buffer, 
 spatialize the monophonic buffer to create a stereophonic buffer, 
 mix monophonic buffers for a monophonic-capable client, and 
 mix stereophonic buffers for a stereophonic-capable client. 
 
 
     
     
         31 . A spatial sound conferencing server comprising:
 an input processor for receiving audio and position and orientation information over a network from a plurality of network clients;   a conference processor for spatializing the audio according to the received position and orientation information; and   an output interface for transmitting the spatialized audio to the network clients over the network.   
     
     
         32 . The conference server of  claim 31  wherein the input processor receives audio and position and orientation information from a plurality of network clients, the conference processor creating, for each client, a mix of audio from other clients spatialized according to the received position and orientation information. 
     
     
         33 . A method of spatializing sound, the method comprising the steps of:
 receiving audio and position information over a network from a plurality of network clients;   spatializing the audio according to the received position and orientation information;   defining a client-specific conference for each network client, each client-specific conference including a mix of audio from other network clients; and   transmitting, as spatialized audio, the client-specific conferences to the corresponding network clients over the network.   
     
     
         34 . The method of  claim 33  wherein:
 each client corresponds to a user; 
 position information corresponding to each user is represented in a world model; and 
 each client-specific conference is spatialized based on a listener position and orientation relative to the positions of the users. 
 
     
     
         35 . The method of  claim 34  wherein:
 for a first client, the associated conference comprises audio from other clients corresponding to users who are proximate to the first-client user in the world model. 
 
     
     
         36 . The method of  claim 34  wherein the audio is spatialized by:
 scaling by a gain inversely proportional to the distance between a client user and a sound source to model distance cues; 
 filtering by a pair of HRTFs to encode localization cues; and 
 summing the resulting stereo signal with stereo reverberation. 
 
     
     
         37 . The method of  claim 33  further comprising:
 receiving audio from a plurality of monophonic and stereophonic capable clients; and 
 for each client,
 storing a monophonic buffer, 
 spatializing the monophonic buffer to create a stereophonic buffer, 
 mixing monophonic buffers for a monophonic-capable client, and 
 mixing stereophonic buffers for a stereophonic-capable client. 
 
 
     
     
         38 . A method of spatializing sound, the method comprising the steps of:
 receiving, on a server, audio and position and orientation information over a network from a plurality of network clients;   spatializing the audio on the server according to the received position and orientation information; and   transmitting the spatialized audio from the server to the network clients over the network.   
     
     
         39 . The method of  claim 38  further comprising the steps of receiving audio and position and orientation information from a plurality of network clients and creating, for each client, a mix of audio from other clients spatialized according to the received position and orientation information. 
     
     
         40 . The method  claim 38  wherein the positional information is represented in a world model. 
     
     
         41 . The method of  claim 38  further comprising the step of defining conferences associated with each client, the audio mix transmitted to a client corresponding to a conference associated with that client. 
     
     
         42 . The method of  claim 38  wherein:
 each client corresponds to a user; 
 the positional information is represented in a world model; and 
 for a first client, the associated conference comprises audio from other clients corresponding to users who are proximate to the first-client user in the world model.