System, method, and computer program for transmitting face models based on face data points

Abstract

A system, method, and computer program are provided for receiving face models based on face nodal points. In use, a real-time face model is received, wherein the real-time face model includes one or more face nodal points. Real-time face nodal points are received, including additional one or more face nodal points. The real-time face model is manipulated based on the real-time face nodal points.

What is claimed is: 
     
         1 . A device, comprising:
 a non-transitory memory storing instructions; and   one or more processors in communication with the non-transitory memory, wherein the one or more processors execute the instructions to:
 receive an audio input signal comprising one or more spoken words; 
 estimate one or more animation parameters for animating movements of a face model based on the one or more spoken words; 
 transmit the one or more animation parameters through a data network to a second device as a real-time stream. 
   
     
     
         2 . The device of  claim 1 , wherein the one or more processors execute further instructions to transmit the face model through the data network to the second device. 
     
     
         3 . The device of  claim 1 , wherein the face model includes one or more facial features. 
     
     
         4 . The device of  claim 1 , wherein the one or more facial features include one or more face data points, and one or more face parameters. 
     
     
         5 . The device of  claim 1 , wherein the one or more face data points and the one or face parameters are correlated. 
     
     
         6 . The device of  claim 1 , wherein the face model includes a geometric mesh. 
     
     
         7 . The device of  claim 6 , wherein the geometric mesh is based on a structure of a face. 
     
     
         8 . The device of  claim 1 , wherein the face model includes texture-mapped image. 
     
     
         9 . The device of  claim 1 , wherein to estimate the one or more animation parameters, the one or more processors execute further instructions to:
 deconstruct the audio input signal of the one or more spoken words into phonetic elements; and   map the phonetic elements through a correlation map configured to generate the one or more animation parameters.   
     
     
         10 . The device of  claim 1 , wherein the one or more animation parameters comprise one or more data points. 
     
     
         11 . The device of  claim 1 , wherein the one or more animation parameters correspond to Eigenfaces comprising a set of canonical facial expressions. 
     
     
         12 . The device of  claim 1 , wherein the one or more processors execute further instructions to transmit the audio input signal to the second device. 
     
     
         13 . The device of  claim 1 , wherein the one or more processors execute further instructions to synchronize the transmitted audio signal with the one or more transmitted animation parameters. 
     
     
         14 . The device of  claim 1 , wherein the device comprises a reality augmentation device. 
     
     
         15 . The device of  claim 1 , wherein the real-time stream is used to modify an associated environment. 
     
     
         16 . The device of  claim 1 , wherein the one or more processors execute the instructions to create an associated environment comprising the real-time stream and at least one other digital overlay or underlay. 
     
     
         17 . The device of  claim 1 , wherein the real-time stream includes a video chat. 
     
     
         18 . The device of  claim 1 , wherein to estimate the one or more animation parameters, the one or more processors execute further instructions to deconstruct the audio input signal of the one or more spoken words into one or more of phonetics, intonations, or emotions. 
     
     
         19 . A device, comprising:
 a non-transitory memory storing instructions; and   one or more processors in communication with the non-transitory memory, wherein the one or more processors execute the instructions to:
 receive one or more animation parameters through a data network as a real-time stream; 
 modify a face model according to the one or more animation parameters; and 
 render a video frame of the modified face model. 
   
     
     
         20 . The device of  claim 19 , wherein the face model is animated according to the one or more animation parameters, and the rendered video frame comprises one of a sequence of video frames depicting the face model animation. 
     
     
         21 . The device of  claim 19 , wherein the one or more processors execute further instructions to receive an audio signal through the data network. 
     
     
         22 . The device of  claim 21 , wherein the one or more processors execute further instructions to synchronize the audio signal with the sequence of video frames. 
     
     
         23 . The device of  claim 19 , wherein the face model includes one or more facial features. 
     
     
         24 . The device of  claim 19 , wherein the one or more facial features include one or more face data points, and one or more face parameters. 
     
     
         25 . The device of  claim 19 , wherein the one or more face data points and the one or face parameters are correlated. 
     
     
         26 . The device of  claim 19 , wherein the face model includes a geometric mesh. 
     
     
         27 . The device of  claim 19 , wherein the geometric mesh is based on a structure of a face. 
     
     
         28 . The device of  claim 19 , wherein the one or more processors execute further instructions to receive the face model through the data network. 
     
     
         29 . The device of  claim 19 , wherein the device comprises a reality augmentation device. 
     
     
         30 . The device of  claim 19 , wherein the modified face model depicts one of a human face, an avatar face, or an inanimate object. 
     
     
         31 . The device of  claim 19 , wherein the real-time stream is used to modify a digital environment. 
     
     
         32 . The device of  claim 19 , wherein the one or more processors execute the instructions to create a digital environment comprising the real-time stream and at least one other digital overlay or underlay. 
     
     
         33 . The device of  claim 19 , wherein the real-time stream includes a video chat.