Systems and Methods for Voice Personalization of Video Content

Abstract

Systems and methods are disclosed for performing voice personalization of video content. The personalized media content may include a composition of a background scene having a character, head model data representing an individualized three-dimensional (3D) head model of a user, audio data simulating the user's voice, and a viseme track containing instructions for causing the individualized 3D head model to lip sync the words contained in the audio data. The audio data simulating the user's voice can be generated using a voice transformation process. In certain examples, the audio data is based on a text input or selected by the user (e.g., via a telephone or computer) or a textual dialogue of a background character.

1 . A method for generating an audio portion of media content, the method comprising:
 receiving a selection from a user of a piece of prerecorded media content, the prerecorded media content comprising a background scene having a character therein;   accessing an individualized three-dimensional (3D) head model;   accessing at least one voice sample of the user;   converting the at least one voice sample to at least one audio track;   detecting from the at least one audio track a plurality of phonemes;   creating at least one viseme track that associates the plurality of phonemes with a plurality of visemes, each of the plurality of visemes being indicative of an animated mouth movement of the individualized 3D head model;   synchronizing the at least one audio track and the at least one viseme track; and   generating personalized media content by,
 associating the individualized 3D head model with the character of the background scene, and 
 associating the synchronized at least one audio track and at least one viseme track with the individualized 3D head model to cause the animated mouth movement of the individualized 3D head model to correspond to the at least one audio track during playback of the personalized media content. 
   
     
     
         2 . The method of  claim 1 , wherein said converting the at least one voice sample further comprises:
 receiving a text that differs from the words of the at least one voice sample; and   generating the at least one audio track to correspond to the text.   
     
     
         3 . The method of  claim 2 , further comprising receiving the text from the user. 
     
     
         4 . The method of  claim 3 , further comprising prompting the user to select one of a plurality of predetermined text options. 
     
     
         5 . The method of  claim 2 , wherein the text is associated with an original script of the character of the background scene. 
     
     
         6 . The method of  claim 1 , wherein said accessing the at least one voice sample of the user comprises receiving the at least one voice sample via a phone network. 
     
     
         7 . The method of  claim 1 , wherein said accessing the at least one voice sample of the user comprises receiving the at least one voice sample via Voice over Internet Protocol (VoIP). 
     
     
         8 . The method of  claim 1 , wherein said accessing the at least one voice sample of the user comprises receiving multiple voice samples from the user at different times. 
     
     
         9 . The method of  claim 1 , additionally comprising transmitting the personalized media content to a user-selected destination. 
     
     
         10 . The method of  claim 1 , additionally comprising storing the at least one voice sample of the user in a memory. 
     
     
         11 . The method of  claim 1 , additionally comprising creating the individualized 3D head model from a plurality of two-dimensional (2D) still images. 
     
     
         12 . An animation system for performing voice personalization of media content, the animation system comprising:
 a piece of media content comprising a background scene having a character;   head model data representing an individualized three-dimensional (3D) head model;   audio data representing at least one voice sample of a user, the at least one voice sample corresponding to a first text;   a processor configured to receive the media content, the head model data and the audio data to generate personalized media content by
 processing the at least one voice sample to create at least one audio track; 
 detecting from the at least one audio track a plurality of phonemes; 
 creating at least one viseme track that associates the plurality of phonemes with a plurality of visemes, each of the plurality of visemes comprising instructions for a corresponding animated mouth movement of the individualized 3D head model; and 
 compositing the media content, the individualized 3D head model, the at least one audio track and the at least one viseme track such that the individualized 3D head model is associated with the character and such that the at least one audio track and the at least one viseme track are associated with the individualized 3D head model to cause the animated mouth movement of the individualized 3D head model to correspond to the at least one audio track during playback of the personalized media content. 
   
     
     
         13 . The animation system of  claim 12 , further comprising blendshape data that determines a plurality of facial expressions of the individualized 3D head model. 
     
     
         14 . The animation system of  claim 13 , wherein each of the plurality of visemes is associated with at least one of the plurality of facial expressions. 
     
     
         15 . The animation system of  claim 12 , further comprising script data associated with the character, and wherein a second text of the at least one audio track is associated with text of the script data. 
     
     
         16 . The animation system of  claim 15 , wherein the first text does not comprise the second text. 
     
     
         17 . The animation system of  claim 12 , wherein the individualized 3D head model comprises a 3D head model of the user. 
     
     
         18 . The animation system of  claim 12 , further comprising a user interface configured to receive expression data to be associated with the playback of the at least one audio track. 
     
     
         19 . A system for animating media content, the system comprising:
 means for receiving a selection of a piece of media content, the media content comprising a background scene having a character therein;   means for receiving an individualized three-dimensional (3D) head model of a user;   means for receiving at least one voice sample of the user;   means for converting the at least one voice sample to at least one audio track;   means for detecting from the at least one audio track a plurality of phonemes;   means for creating at least one viseme track that associates the plurality of phonemes with a plurality of visemes, each of the plurality of visemes being indicative of an animated mouth movement of the individualized 3D head model; and   means for generating personalized media content by
 associating the individualized 3D head model with the character of the background scene, and 
 associating the at least one audio track and the at least one viseme track with the individualized 3D head model to cause the animated mouth movement of the individualized 3D head model to correspond to the at least one audio track during playback of the personalized media content. 
   
     
     
         20 . The system of  claim 19 , wherein a text of the at least one audio track is different than a text of the at least one voice sample. 
     
     
         21 . The system of  claim 19 , the system further comprising means for generating the individualized 3D head model.