Method and apparatus for registering face, and method and apparatus for recognizing face

Abstract

A method and an apparatus for registering a face, and a method and an apparatus for recognizing a face are disclosed, in which a face registering apparatus may change a stored three-dimensional (3D) facial model to an individualized 3D facial model based on facial landmarks extracted from two-dimensional (2D) face images, match the individualized 3D facial model to a current 2D face image of the 2D face images, and extract an image feature of the current 2D face image from regions in the current 2D face image to which 3D feature points of the individualized 3D facial model are projected, and a face recognizing apparatus may perform facial recognition based on image features of the 2D face images extracted by the face registering apparatus.

What is claimed is: 
     
         1 . A method of recognizing a face, comprising:
 receiving a two-dimensional (2D) input image;   transforming a three-dimensional (3D) facial model based on a user face appearing in the 2D input image;   extracting an image feature from positions where feature points of the transformed 3D facial model are projected to the 2D input image; and   determining a result of facial recognition based on the extracted image feature and a reference image feature.   
     
     
         2 . The method of  claim 1 , wherein the transforming comprising:
 adjusting at least one of a facial pose and a facial expression of the 3D facial model based on the user face appearing in the 2D input image.   
     
     
         3 . The method of  claim 1 , wherein the transforming comprising:
 extracting facial landmarks from the 2D input image; and   matching the three-dimensional (3D) facial model to the user face appearing in the 2D input image based on the extracted facial landmarks.   
     
     
         4 . The method of  claim 3 , wherein the extracting comprising:
 projecting the feature points of the matched 3D facial model to the 2D input image; and   extracting the image feature from positions where feature points of the matched 3D facial model are projected to the 2D input image.   
     
     
         5 . The method of  claim 1 , wherein the determining comprising:
 comparing the extract image feature to a reference image feature; and   determining the result of facial recognition based on the comparing.   
     
     
         6 . The method of  claim 1 , wherein the reference image feature comprises an image feature of a 2D face image used for registering a face, obtained by matching the 3D facial model to the 2D face image and extracting the image feature from positions where the feature points of the matched 3D facial model are projected to the 2D face image. 
     
     
         7 . The method of  claim 1 , wherein the determining comprises:
 selecting a set of reference image features to be used for the facial recognition based on a facial pose in the 2D input image, and   determining the result of the facial recognition based on a degree of similarity between the selected set of reference image features and a set of image features extracted from the 2D input image, the reference image feature being in the set of reference image features.   
     
     
         8 . The method of  claim 1 , wherein the determining comprises:
 determining a degree of similarity between the reference image feature and the extracted image feature; and   determining the facial recognition to be successful in response to the degree of similarity satisfying a condition.   
     
     
         9 . A non-transitory computer-readable medium comprising program code that, when executed by a processor, performs functions according to the method of  claim 1 . 
     
     
         10 . An apparatus for recognizing a face, comprising:
 at least one processor; and   a memory configured to communicate with the at least one processor, the memory configured to store instructions executable by the at least one processor such that   the at least one processor is configured to receive a two-dimensional (2D) input image,   transform a three-dimensional (3D) facial model based on a user face appearing in the 2D input image,   extract an image feature from positions where feature points of the transformed 3D facial model are projected to the 2D input image, and   determine a result of facial recognition based on the extracted image feature and a reference image feature.   
     
     
         11 . The apparatus of  claim 10 , wherein the at least one processor is configured to execute the computer-readable instructions to adjust at least one of a facial pose and a facial expression of the 3D facial model based on the user face appearing in the 2D input image. 
     
     
         12 . The apparatus of  claim 10 , wherein the at least one processor is configured to execute the computer-readable instructions to extract facial landmarks from the 2D input image and match the three-dimensional (3D) facial model to the user face appearing in the 2D input image based on the extracted facial landmarks. 
     
     
         13 . The apparatus of  claim 12 , wherein the at least one processor is configured to execute the computer-readable instructions to project the feature points of the matched 3D facial model to the 2D input image and extract the image feature from positions where feature points of the matched 3D facial model are projected to the 2D input image. 
     
     
         14 . The apparatus of  claim 10 , wherein the reference image feature comprises an image feature of a 2D face image used for registering a face, obtained by matching the 3D facial model to the 2D face image and extracting the image feature from positions where the feature points of the matched 3D facial model are projected to the 2D face image. 
     
     
         15 . A method of registering a face, comprising:
 receiving a two-dimensional (2D) face image for registering a face;   transforming a stored three-dimensional (3D) facial model to an individualized 3D facial model based on a user face appearing in the 2D face image;   extracting an image feature from positions where feature points of the individualized 3D facial model are projected to the 2D face image; and   storing the extracted image feature as a reference image feature.   
     
     
         16 . The method of  claim 15 , wherein the transforming comprising:
 extracting facial landmarks from the 2D face image; and   matching the stored three-dimensional (3D) facial model to the user face appearing in the 2D face image based on the extracted facial landmarks.   
     
     
         17 . The method of  claim 16 , wherein the extracting comprising:
 projecting the feature points of the matched 3D facial model to the 2D face image; and   extracting the image feature from positions where feature points of the matched 3D facial model are projected to the 2D face image.   
     
     
         18 . The method of  claim 16 , wherein the transforming comprises:
 determining shape control parameters to match facial landmarks of the stored 3D facial model to the extracted facial landmarks of the 2D face image; and   
       changing the stored 3D facial model to the individualized 3D facial model by applying the determined shape control parameters to the stored 3D facial model. 
     
     
         19 . The method of  claim 15 , wherein the transforming comprising:
 adjusting at least one of a facial pose and a facial expression of the stored 3D facial model based on the user face appearing in the 2D face image.   
     
     
         20 . The method of  claim 15 , wherein the 2D face image comprise at least one frontal face image including a frontal face of a user and at least one profile image including a profile of the user.