Augmented reality audio mixing

Abstract

Augmented reality enables an operator to visualize values of parameters of audio channels during audio mixing. A heads-up display worn by the operator, such as smart glasses, displays virtual graphical objects representing the parameters so that they appear within a three-dimensional space surrounding the operator and an audio mixing console. Parameter values are represented by the location, size, and other attributes of the virtual graphical objects. The operator adjusts the parameter values using physical and touch controls on the console and by manipulating the virtual objects with a body part such as a finger. Sensors mounted on the heads-up display and on other body parts capture position and movement of the operator in real time and send the captured data to a system running augmented reality control software. Graphical user interfaces of a media processing application controlled by the console and of plug-in modules may also be displayed on the heads-up display.

What is claimed is: 
     
         1 . A method of mixing a plurality of audio channels of a media project, the method comprising:
 providing a physical mixing console for mixing the plurality of audio channels of the media project;   providing smart glasses for an operator of the physical mixing console, wherein the physical mixing console and the smart glasses are in data communication with a computer hosting augmented reality software;   providing to the augmented reality software a mapping of a physical space surrounding the operator and the physical mixing console, wherein the mapping includes a mapping of a physical location of the physical mixing console within the physical space; and   while the operator is wearing the smart glasses, displaying on the smart glasses a graphical representation in virtual space of a first element of a user interface for mixing the plurality of audio channels, wherein a location within the virtual space at which the first element of the user interface is displayed is chosen to avoid covering an object which needs to be visible to the operator in at least one of the physical space and the virtual space.   
     
     
         2 . The method of  claim 1 , wherein the object which needs to be visible to the operator is the physical mixing console. 
     
     
         3 . The method of  claim 1 , further comprising displaying on the smart glasses a graphical representation in virtual space of a second element of the user interface for mixing the plurality of audio channels, wherein the object which needs to be visible to the operator is the first element of the user interface for mixing the plurality of audio channels. 
     
     
         4 . The method of  claim 1 , wherein the first user interface element comprises a virtual monitor displaying a graphical user interface of a digital audio workstation that is being used to mix the plurality of audio channels. 
     
     
         5 . The method of  claim 4 , wherein the virtual monitor appears to be displayed on a wall of the physical space. 
     
     
         6 . The method of  claim 1 , wherein the physical space includes a screen for displaying a synchronized video component of the media project while the operator is mixing the plurality of audio channels of the media project. 
     
     
         7 . The method of  claim 1 , wherein the first user interface element enables the operator to interact with one or more parameters of the plurality of audio channels. 
     
     
         8 . The method of  claim 1 , further comprising displaying on the smart glasses a user interface element in the virtual space enabling the operator to control equalization parameters of the plurality of audio channels. 
     
     
         9 . The method of  claim 1 , further comprising displaying on the smart glasses a user interface element in virtual space enabling the operator to control a dynamics parameter of the one or more audio channels. 
     
     
         10 . The method of  claim 1 , further comprising displaying on the smart glasses a user interface element in virtual space showing three-dimensional spectrograms of one or more of the one or more audio channels. 
     
     
         11 . The method of  claim 1 , further comprising displaying on the smart glasses a user interface element in virtual space showing filter response curves of one or more of the one or more audio channels. 
     
     
         12 . A system for audio mixing comprising:
 a control system in data communication with augmented reality smart glasses and with a physical mixing console, wherein the control system includes augmented reality software, which when executed on the control system, cause the control system to enable a method of mixing a plurality of audio channels of a media project, the method comprising:
 the control system receiving a mapping of a physical space surrounding an operator and the physical mixing console, wherein the mapping includes a mapping of a physical location of the physical mixing console within the physical space; and 
 while the operator is wearing the smart glasses, displaying on the smart glasses a graphical representation in virtual space of a first element of a user interface for mixing the plurality of audio channels, wherein a location within the virtual space at which the first element of the user interface is displayed is chosen to avoid covering an object which needs to be visible to the operator in at least one of the physical space and the virtual space. 
   
     
     
         13 . The system of  claim 12 , wherein the object which needs to be visible to the operator is the physical mixing console. 
     
     
         14 . The system of  claim 12 , further comprising displaying on the smart glasses a graphical representation in virtual space of a second element of the user interface for mixing the plurality of audio channels, wherein the object which needs to be visible to the operator is the first element of the user interface for mixing the plurality of audio channels. 
     
     
         15 . The system of  claim 12 , wherein the first user interface element comprises a virtual monitor displaying a graphical user interface of a digital audio workstation that is being used to mix the plurality of audio channels. 
     
     
         16 . The system of  claim 15 , wherein the virtual monitor appears to be displayed on a wall of the physical space. 
     
     
         17 . The system of  claim 12 , wherein the physical space includes a screen for displaying a synchronized video component of the media project while the operator is mixing the plurality of audio channels of the media project. 
     
     
         18 . The system of  claim 12 , wherein the first user interface element enables the operator to interact with one or more parameters of the plurality of audio channels. 
     
     
         19 . The system of  claim 12 , further comprising displaying on the smart glasses a user interface element in the virtual space enabling the operator to control equalization parameters of the plurality of audio channels. 
     
     
         20 . The system of  claim 12 , further comprising displaying on the smart glasses a user interface element in virtual space enabling the operator to control a dynamics parameter of the one or more audio channels.