Performance driven facial animation

Abstract

A method of animating a digital facial model, the method including: defining a plurality of action units; calibrating each action unit of the plurality of action units via an actor's performance; capturing first facial pose data; determining a plurality of weights, each weight of the plurality of weights uniquely corresponding to the each action unit, the plurality of weights characterizing a weighted combination of the plurality of action units, the weighted combination approximating the first facial pose data; generating a weighted activation by combining the results of applying the each weight to the each action unit; applying the weighted activation to the digital facial model; and recalibrating at least one action unit of the plurality of action units using input user adjustments to the weighted activation.

1 . A method of animating a digital facial model, the method comprising:
 defining a plurality of action units;   calibrating each action unit of said plurality of action units via an actor's performance;   capturing first facial pose data;   determining a plurality of weights, each weight of said plurality of weights uniquely corresponding to said each action unit, said plurality of weights characterizing a weighted combination of said plurality of action units, said weighted combination approximating said first facial pose data; and   recalibrating at least one action unit of said plurality of action units using input user adjustments.   
     
     
         2 . The method of  claim 1 , wherein said each action unit includes a second facial pose data and an activation. 
     
     
         3 . The method of  claim 2 , wherein said calibrating each action unit includes
 calibrating said second facial pose data of said each action unit using calibration pose data derived from a calibration performance corresponding with said each action unit.   
     
     
         4 . The method of  claim 3 , further comprising
 cleaning and stabilizing said calibration pose data.   
     
     
         5 . The method of  claim 2 , wherein said weighted combination includes
 a weighted combination of said second facial pose data of said each action unit.   
     
     
         6 . The method of  claim 5 , wherein said determining a plurality of weights includes
 an optimization of a correspondence between said first facial pose data and said weighted combination of said second facial pose data.   
     
     
         7 . The method of  claim 6 , wherein said optimization includes a linear optimization. 
     
     
         8 . The method of  claim 7 , wherein said linear optimization includes a least-squares method. 
     
     
         9 . The method of  claim 6 , wherein said optimization includes a non-linear optimization. 
     
     
         10 . (canceled) 
     
     
         11 . The method of  claim 2 , wherein said recalibrating at least one action unit includes
 recalibrating said second facial pose data.   
     
     
         12 . The method of  claim 2 , wherein said recalibrating at least one action unit includes
 recalibrating said activation.   
     
     
         13 . The method of  claim 2 , wherein said activation of said each action unit is directed to a fascia layer. 
     
     
         14 . The method of  claim 13 , wherein said fascia layer includes a muscle layer. 
     
     
         15 . The method of  claim 13 , wherein said fascia layer includes a jaw layer. 
     
     
         16 . The method of  claim 13 , wherein said fascia layer includes a volume layer. 
     
     
         17 . The method of  claim 13 , wherein said fascia layer includes an articulation layer. 
     
     
         18 . The method of  claim 1 , wherein said plurality of action units comprises a FACS matrix. 
     
     
         19 . The method of  claim 1 , further comprising
 cleaning and stabilizing said first facial pose data.   
     
     
         20 . A method of animating a digital facial model, the method comprising:
 defining a plurality of action units, each action unit of including first facial pose data and an activation;   calibrating said first facial pose data using calibration pose data derived from a plurality of captured calibration performances, each calibration performance of said plurality of captured calibration performances corresponding with said each action unit;   deriving second facial pose data from another calibration performance of said plurality of captured calibration performances;   determining a plurality of weights, each weight of said plurality of weights uniquely corresponding to said each action unit, said plurality of weights characterizing a weighted combination of said facial pose data, said weighted combination approximating said second facial pose data; and   recalibrating said first facial pose data and said activation using input user adjustments.   
     
     
         21 . A system for retargeting facial motion capture data to a digital facial model, the system comprising:
 a FACS module to manage a plurality of action units;   a calibration module to calibrate each action unit of the plurality of action units via an actor's performance; and   a tuning interface module to generate recalibrated action units for said FACS module in accordance with input user adjustments to a facial animation frame.   
     
     
         22 . The system of  claim 21 , wherein said animation module includes
 a rigging unit to generate said digital facial model.   
     
     
         23 . The system of  claim 22 , wherein said rigging unit generates at least one fascia layer on said digital facial model. 
     
     
         24 . The system of  claim 23 , wherein said animation module includes
 a transfer module to apply said at least one weighted activation to said at least one fascia layer.   
     
     
         25 . The system of  claim 21 , wherein said tuning interface module includes
 a frame selection unit to select said facial animation frame for tuning.   
     
     
         26 . (canceled) 
     
     
         27 . A method of digital facial animation, the method comprising:
 defining a plurality of action units in a FACS matrix;   calibrating each action unit of said plurality of action units via an actor's performance;   capturing facial motion data;   labeling said facial motion data;   stabilizing said facial motion data;   cleaning said facial motion data using said FACS matrix;   normalizing said facial motion data;   retargeting said facial motion data onto a digital facial model using said FACS matrix; and   recalibrating using multidimensional tuning of said FACS matrix.   
     
     
         28 . The method of  claim 27 , wherein multidimensional tuning uses tuning feedback provided by an animator to reduce the effects of incorrect mathematical solutions in the FACS matrix associated with poses in selected frames. 
     
     
         29 . The method of  claim 28 , wherein the tuning feedback is performed by modifying weights resulting from the solutions in the FACS matrix associated with the poses in the selected frames. 
     
     
         30 . The method of  claim 29 , wherein the modified weights are used to update and optimize the FACS matrix, resulting in the FACS matrix including action units based on actual marker ranges of motion as well as the modified weights.