Dynamic real-time generation of three-dimensional avatar models of users based on live visual input of users' appearance and computer systems and computer-implemented methods directed to thereof

Abstract

In some embodiments, the present invention provides for a computer system that may include a camera component configured to acquire a visual content, where the visual content includes a plurality of frames having a visual representation of a person's face; and a processor configured to: train a face detection regressor with a synthetic face model database to obtain a face detection trained regressor; apply, for each frame, the face detection trained regressor to detect or to track the face based on facial features, local features, and a pre-defined hyperparameter; construct an intermediate multi-dimensional face model; apply machine learning to determine features of an intermediate multi-dimensional head model; construct a multi-dimensional avatar; and utilize the multi-dimensional avatar to perform an activity associated with the person.

What is claimed is: 
     
         1 . A method, comprising:
 training, by at least one processor, at least one face detection regressor with at least one synthetic face model database to obtain at least one face detection trained regressor;   wherein the at least one synthetic face model database comprises a plurality of facial features;   wherein each facial feature is identified by a pre-defined set of parameters;   obtaining, by the at least one processor, a plurality of frames having a visual representation of a face of at least one person;   extracting, by the at least one processor, a plurality of local features from each frame of the plurality of frames;   applying, by the at least one processor, for each frame, the at least one face detection trained regressor to the plurality of local features to detect or to track a presence of a face of a particular individual, based, at least in part, on:
 i) the plurality of facial features, 
 ii) the plurality of local features, and 
 iii) at least one pre-defined hyperparameter; 
   constructing, by the at least one processor, an intermediate multi-dimensional face model of the face of the particular individual based on the detected or tracked presence of the face of the particular individual;   applying, by the at least one processor, machine learning to determine a plurality of features of an intermediate multi-dimensional head model of the particular individual based on the detected or tracked presence of the face of the particular individual;   constructing, by the at least one processor, at least one multi-dimensional avatar of the particular individual based, at least in part, on:
 i) the intermediate multi-dimensional face model of the particular individual, and 
 ii) the plurality of features of the intermediate multi-dimensional head model of the particular individual; and 
   utilizing, by the at least one processor, the at least one multi-dimensional avatar of the particular individual to perform at least one activity associated with the particular individual.   
     
     
         2 . A system comprising:
 a camera component, wherein the camera component is configured to acquire a visual content, wherein the visual content comprises a plurality of frames having a visual representation of a face of at least one person; and   at least one processor configured to:   train at least one face detection regressor with at least one synthetic face model database to obtain at least one face detection trained regressor;
 wherein the at least one synthetic face model database comprises a plurality of facial features; 
 wherein each facial feature is identified by a pre-defined set of parameters; 
   extract a plurality of local features from each frame of the plurality of frames;   apply, for each frame, the at least one face detection trained regressor to the plurality of local features to detect or to track a presence of a face of a particular individual, based, at least in part, on:
 i) the plurality of facial features, 
 ii) the plurality of local features, and 
 iii) at least one pre-defined hyperparameter; 
   construct an intermediate multi-dimensional face model of the face of the particular individual based on the detected or tracked presence of the face of the particular individual;   apply machine learning to determine a plurality of features of an intermediate multi-dimensional head model of the particular individual based on the detected or tracked presence of the face of the particular individual;   construct at least one multi-dimensional avatar of the particular individual based, at least in part, on:
 i) the intermediate multi-dimensional face model of the particular individual, and 
 ii) the plurality of features of the intermediate multi-dimensional head model of the particular individual; and 
   utilize the at least one multi-dimensional avatar of the particular individual to perform at least one activity associated with the particular individual.