Generating an Animation Rig for Use in Animating a Computer-Generated Character Based on Facial Scans of an Actor and a Muscle Model

Abstract

An animation system wherein scanned facial expressions are processed to form muscle models that can be used to generate expressions based on specification of a strain vector and a control vector of the muscle model.

What is claimed is: 
     
         1 . A computer-implemented method for generating a facial expression in an animation system, the method comprising:
 receiving data associated with a plurality of facial scans of a face of an actor over a plurality of facial expression poses;   obtaining a muscle model comprising a set of facial muscle structural parameters and a set of strain values corresponding to a set of facial muscles; extracting from the plurality of facial scans, a representation of physical deformations of a facial surface of the face;   determining, from the representation of physical deformations of the facial surface and the muscle model, a strain vector representing strains of the set of facial muscles from the muscle model;   generating a mesh representing the facial surface of the face that is formed into expressions based on the muscle model and strain vector values and their corresponding deformations; and   determining values for the set of vector values in the strain vector that adjust the mesh to form an animated facial expression that corresponds to a plausible facial expression from the actor.   
     
     
         2 . The method of  claim 1 , wherein the plurality of facial scans comprise data pertaining to the physical surface deformations, wherein the deformations are associated with the facial expression poses. 
     
     
         3 . The method of  claim 1 , wherein the set of facial muscle structural parameters in the muscle model comprise a location parameter for a bone attachment point for a facial muscle, a location parameter for a skin attachment point for the facial muscle, and a volume parameter for the facial muscle. 
     
     
         4 . The method of  claim 1 , wherein the set of facial muscle structural parameters in the muscle model further comprises parameters describing a joint. 
     
     
         5 . The method of  claim 1 , wherein determining the control vectors further comprises determining strain values associated with expansion and contraction of muscles of the set of facial muscles. 
     
     
         6 . The method of  claim 1  further comprising generating an animated image using a facial expression derived from a blending of strain vector values. 
     
     
         7 . The method of  claim 1 , wherein the muscle model is derived by:
 training a machine learning system based on a training dataset of the plurality of facial scans and information relating to the plurality of facial expression poses as ground-truth labels.   
     
     
         8 . An animation system comprising:
 at least one processor; and   medium storing instructions, which when executed by the at least one processor, cause the animation system to:   receive data associated with a plurality of facial scans of a face of an actor over a plurality of facial expression poses;   obtain a muscle model comprising a set of facial muscle structural parameters and a set of strain values corresponding to a set of facial muscles; extracting from the plurality of facial scans, a representation of physical deformations of a facial surface of the face;   determine, from the representation of physical deformations of the facial surface and the muscle model, a strain vector representing strains of the set of facial muscles from the muscle model;   generate a mesh representing the facial surface of the face that is formed into expressions based on the muscle model and strain vector values and their corresponding deformations; and   determine values for the set of vector values in the strain vector that adjust the mesh to form an animated facial expression that corresponds to a plausible facial expression from the actor.   
     
     
         9 . The animation system of  claim 8 , wherein the plurality of facial scans comprise data pertaining to the physical surface deformations, wherein the deformations are associated with the facial expression poses. 
     
     
         10 . The animation system of  claim 8 , wherein the set of facial muscle structural parameters in the muscle model comprise a location parameter for a bone attachment point for a facial muscle, a location parameter for a skin attachment point for the facial muscle, and a volume parameter for the facial muscle. 
     
     
         11 . The animation system of  claim 8 , wherein the set of facial muscle structural parameters in the muscle model further comprises parameters describing a joint. 
     
     
         12 . The animation system of  claim 8 , the control vectors are strain values associated with expansion and contraction of muscles of the set of facial muscles. 
     
     
         13 . The animation system of  claim 8 , wherein an animated image is generated using a facial expression derived from a blending of strain vector values. 
     
     
         14 . A non-transitory computer-readable storage medium storing instructions, which when executed by at least one processor of a computer system, causes the computer system to:
 receive data associated with a plurality of facial scans of a face of an actor over a plurality of facial expression poses;   obtain a muscle model comprising a set of facial muscle structural parameters and a set of strain values corresponding to a set of facial muscles; extracting from the plurality of facial scans, a representation of physical deformations of a facial surface of the face;   determine, from the representation of physical deformations of the facial surface and the muscle model, a strain vector representing strains of the set of facial muscles from the muscle model;   generate a mesh representing the facial surface of the face that is formed into expressions based on the muscle model and strain vector values and their corresponding deformations; and
 determine values for the set of vector values in the strain vector that adjust the mesh to form an animated facial expression that corresponds to a plausible facial expression from the actor. 
   
     
     
         15 . The non-transitory computer-readable storage medium of  claim 14 , further storing instructions, which when executed by at least one processor of a computer system, causes the computer system to process the plurality of facial scans as data pertaining to the physical surface deformations, wherein the deformations are associated with the facial expression poses. 
     
     
         16 . The non-transitory computer-readable storage medium of  claim 14 , further storing instructions, which when executed by at least one processor of a computer system, causes the computer system to process the plurality of facial scans based on an anatomical model that corresponds to human anatomy. 
     
     
         17 . The non-transitory computer-readable storage medium of  claim 16 , further storing instructions, which when executed by at least one processor of a computer system, causes the computer system to process the animated facial expression using an alternative anatomical model that is distinct from an anatomy of the actor. 
     
     
         18 . The non-transitory computer-readable storage medium of  claim 16 , further storing instructions, which when executed by at least one processor of a computer system, causes the computer system to process the control vectors by determining strain values associated with expansion and contraction of muscles of the set of facial muscles. 
     
     
         19 . The non-transitory computer-readable storage medium of  claim 16 , further storing instructions, which when executed by at least one processor of a computer system, causes the computer system to process a facial expression derived from a blending of strain vector values. 
     
     
         20 . The non-transitory computer-readable storage medium of  claim 14 , wherein the muscle model is derived by:
 training a machine learning system based on a training dataset of the plurality of facial scans and information relating to the plurality of facial expression poses as ground-truth labels.