Modification of far end audio signals

Abstract

Techniques for modification of far end audio signals are provided. In an example method, a computing device establishes a conference meeting including a first user equipment (UE) among a plurality of UE. The computing device then receives far end audio (FEA) data from the first UE. The computing device generates first modified FEA comprising determining a time-domain sum-of-absolute-difference (SAD) from the FEA data and determines a difference between the first modified FEA and a predefined value. Responsive to the difference exceeding a predefined threshold, the computing device determines a second modified FEA comprising determining a frequency-domain SAD from the FEA data and stores the second modified FEA in a buffer. The second modified is then output from the buffer.

What is claimed is: 
     
         1 . A method, comprising:
 establishing a conference meeting including a first user equipment (UE) among a plurality of UE;   receiving far end audio (FEA) data from the first UE;   generating first modified FEA;   determining a first difference between the first modified FEA and a predefined value;   responsive to the first difference satisfying a predefined threshold, storing the first modified FEA in a buffer; and   outputting, from the buffer, the first modified FEA.   
     
     
         2 . The method of  claim 1 , wherein the FEA data comprises a fetched audio signal and an FEA signal, wherein the fetched audio signal comprises a shared audio signal and the FEA signal and the shared audio signal is determined by removing a modified FEA signal from the fetched audio signal. 
     
     
         3 . The method of  claim 2 , wherein generating the first modified FEA comprises determining a time-domain sum-of-absolute-difference (SAD) from the FEA data, comprising minimizing a second difference between the fetched audio signal and the FEA signal. 
     
     
         4 . The method of  claim 3 , wherein the fetched audio signal and the FEA signal at a time t are represented by vectors, wherein the magnitude of each vector corresponds to an energy. 
     
     
         5 . The method of  claim 4 , wherein the FEA signal is delayed with respect to the shared audio signal. 
     
     
         6 . The method of  claim 5 , wherein the time-domain SAD is determined by minimizing ∥z(n)−f(x(n+t))∥, wherein z(n) is the fetched audio signal at a discrete time n; x(n+t) is the FEA signal at a discrete delayed time n+t; and ∥z(n)−f(x(n+t))∥ is the L1 norm of the vector elements z(n) and x(n+t). 
     
     
         7 . The method of  claim 6 , wherein the second difference is minimized over a range of discrete time values between −T and T, inclusive, wherein T is a predetermined positive integer. 
     
     
         8 . A method, comprising:
 establishing a conference meeting including a first user equipment (UE) among a plurality of UE;   receiving far end audio (FEA) data from the first UE;   generating first modified FEA;   determining a difference between the first modified FEA and a predefined value;   responsive to the difference exceeding a predefined threshold, generating a second modified FEA;   storing the second modified FEA in a buffer; and   outputting, from the buffer, the second modified FEA.   
     
     
         9 . The method of  claim 8 , wherein:
 generating the first modified FEA comprises determining a time-domain sum-of-absolute-difference (SAD) from the FEA data; and   generating the second modified FEA comprises determining a frequency-domain SAD from the FEA data.   
     
     
         10 . The method of  claim 9 , wherein:
 the FEA data comprises a fetched audio signal and an FEA signal, wherein the fetched audio signal comprises a shared audio signal and the FEA signal and the shared audio signal is determined by removing a modified FEA signal from the fetched audio signal;   the fetched audio signal and the FEA signal at a time t are represented by vectors, wherein the magnitude of each vector corresponds to an energy;   determining the time-domain SAD comprises minimizing a first difference between the fetched audio signal and the FEA signal, comprising minimizing ∥z(n)−f(x(n+t))∥, wherein z(n) is the fetched audio signal at discrete time n; x(n+t) is the FEA signal at discrete delayed time n+t; and ∥z(n)−f(x(n+t))∥ is the L1 norm of the vector elements z(n) and x(n+t); and   determining the frequency-domain SAD comprises minimizing a second difference between a first Fourier transform of the fetched audio signal and a second Fourier transform of the FEA signal, comprising minimizing ∥Z(f)−X t (f)∥, wherein Z(f) is the first Fourier transform of fetched audio signal at a discrete time t; X t (f) is the second Fourier transform of the FEA signal at a discrete delayed time n+t; f is a frequency index, and ∥Z(f)−X t (f)∥ is the L1 norm of the vector elements Z(f) and X t (f).   
     
     
         11 . The method of  claim 10 , wherein the FEA signal is filtered by a linear or nonlinear filter. 
     
     
         12 . The method of  claim 10 , wherein the first Fourier transform is a Discrete-Time Fourier Transform (DTFT) and the second Fourier transform is a DTFT. 
     
     
         13 . The method of  claim 10 , wherein minimization of ∥Z(f)−X t (f)∥ and ∥z(n)−f(x(n+t))∥ is a two-dimensional minimization over a range of discrete time values between −T and T, inclusive, wherein T is a predetermined positive integer. 
     
     
         14 . The method of  claim 10 , wherein the energy of the FEA signal is attenuated or boosted. 
     
     
         15 . The method of  claim 14 , wherein:
 minimizing the difference between the fetched audio signal and the FEA signal further comprises minimizing ∥z(n)−γ·f(x(n+t))∥, wherein z(n) is the fetched audio signal at a discrete time n; x(n+t) is the FEA signal at a discrete delayed time n+t; γ is a scaling factor;   and ∥z(n)−γ·f(x(n+t))∥ is the L1 norm of the vector elements z(n) and γ·f(x(n+t)); and   minimizing the difference between the first Fourier transform of the fetched audio signal and the second Fourier transform of the FEA signal, further comprises minimizing ∥Z(f)−γ·X t (f)∥, wherein Z(f) is the first Fourier transform of the fetched audio signal at a discrete time t; X t (f) is the second Fourier transform of the FEA signal at a discrete delayed time n+t; γ is the scaling factor; f is a frequency index, and ∥Z(f)−γ·X t (f)∥ is the L1 norm of the vector elements Z(f) and γ·X t (f).   
     
     
         16 . The method of  claim 15 , wherein minimization of ∥Z(f)−γ·X t (f)∥ and ∥z(n)−γ·f(x(n+t))∥ is a two-dimensional minimization over a range of discrete time values between −T and T, inclusive, wherein T is a predetermined positive integer and a prespecified list of gamma candidates, {γ 0 , γ 1 , . . . , γ N }. 
     
     
         17 . The method of  claim 10 , wherein determining the second modified FEA is further responsive to the time-domain SAD exceeding a predetermined threshold. 
     
     
         18 . A system comprising:
 one or more processors; and   one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including:
 establishing a conference meeting including a first user equipment (UE) among a plurality of UE; 
 receiving far end audio (FEA) data from the first UE, wherein the FEA data comprises a fetched audio signal and an FEA signal, wherein the fetched audio signal comprises a shared audio signal and the FEA signal and the shared audio signal is determined by removing a modified FEA signal from the fetched audio signal; 
 generating first modified FEA; 
 determining a second difference between the first modified FEA and a predefined value; 
 responsive to the second difference exceeding a predefined threshold, generating a second modified FEA; 
 storing the second modified FEA in a buffer; and 
 outputting, from the buffer, the second modified FEA. 
   
     
     
         19 . The system of  claim 18 , wherein:
 generating the first modified FEA comprises determining a time-domain sum-of-absolute-difference (SAD) from the FEA data, comprising minimizing a first difference between the fetched audio signal and the FEA signal; and   generating the second modified FEA comprises determining a frequency-domain SAD from the FEA data, comprising minimizing a third difference between a first Fourier transform of the fetched audio signal and a second Fourier transform of the FEA signal.   
     
     
         20 . The system of  claim 19 , wherein:
 the fetched audio signal and the FEA signal at a time t are represented by vectors, wherein the magnitude of each vector corresponds to an energy;   minimizing the first difference between the fetched audio signal and the FEA signal, comprises minimizing ∥z(n)−f(x(n+t)∥, wherein z(n) is the fetched audio signal at discrete time n; x(n+t) is the FEA signal at discrete delayed time n+t; and ∥z(n)−f(x(n+t))∥ is the L1 norm of the vector elements z(n) and x(n+t); and   minimizing the third difference between the first Fourier transform of the fetched audio signal and the second Fourier transform of the FEA signal comprises minimizing ∥X(f)−X t (f)∥, wherein Z(f) is the first Fourier transform of the fetched audio signal at a discrete time t; X t (f) is the second Fourier transform of the FEA signal at a discrete delayed time n+t; f is a frequency index, and ∥Z(f)−X t (f)∥ is the L1 norm of the vector elements Z(f) and X t (f).