Motion Tracking and Image Recognition of Hand Gestures to Animate a Digital Puppet, Synchronized with Recorded Audio

Abstract

There is provided a system and method for creating a digital puppet show by animating a digital puppet using gestures and sound. The method includes presenting the digital puppet to a user on a display. The method further includes receiving motion data corresponding to a gesture, using a camera, from the user, translating the motion data into digital data using a motion tracking algorithm, and animating the digital puppet, on the display, using the digital data. The method can further include receiving audio data from the user using a microphone and playing the audio data, using a speaker, while animating the digital puppet on the display to create the digital puppet show.

1 - 20 . (canceled) 
     
     
         21 : A computer-readable non-transitory medium having stored thereon a digital puppet application, which when executed by a hardware processor, performs a method comprising:
 presenting a three-dimensional (3-D) digital puppet to a user on a display;   receiving motion data corresponding to a gesture made by the user;   receiving audio data corresponding to voice of the user for use in a 3-D digital puppet show;   translating the motion data into digital data using a motion tracking algorithm; and   creating the 3-D digital puppet show using the 3-D digital puppet by:
 animating the 3-D digital puppet, on the display, using the digital data; and 
 playing the audio data at the same time as the animating of the 3-D digital puppet on the display, wherein playing the audio data includes synchronizing the audio data with the animating of the 3-D digital puppet. 
   
     
     
         22 : The computer-readable non-transitory medium of  claim 21 , wherein when the digital puppet application is executed by the hardware processor, further performs the method comprising:
 before presenting the 3-D digital puppet to the user, presenting a plurality of 3-D digital puppets to the user on the display; and   receiving a user input for selecting the 3-D digital puppet from the plurality of 3-D digital puppets.   
     
     
         23 : The computer-readable non-transitory medium of  claim 21 , wherein the gesture made by the user is a hand gesture. 
     
     
         24 : The computer-readable non-transitory medium of  claim 23 , wherein a physical accessory corresponding to the 3-D digital puppet is used by the user for the hand gesture. 
     
     
         25 : The computer-readable non-transitory medium of  claim 23 , wherein a movement of fingers and a thumb of the hand gesture animates a mouth of the 3-D digital puppet. 
     
     
         26 : The computer-readable non-transitory medium of  claim 21 , wherein the digital data further includes rotational data. 
     
     
         27 : The computer-readable non-transitory medium of  claim 21 , wherein when the digital puppet application is executed by the hardware processor, further performs the method comprising:
 recording the animation of the 3-D digital puppet in a memory to create the 3-D digital puppet show.   
     
     
         28 : The computer-readable non-transitory medium of  claim 21 , wherein the digital data further includes mouth control data. 
     
     
         29 : A method comprising:
 presenting a three-dimensional (3-D) digital puppet to a user on a display;   receiving motion data corresponding to a gesture made by the user;   receiving audio data corresponding to voice of the user for use in a 3-D digital puppet show;   translating the motion data into digital data using a motion tracking algorithm; and   creating the 3-D digital puppet show using the 3-D digital puppet by:
 animating the 3-D digital puppet, on the display, using the digital data; and 
 playing the audio data at the same time as the animating of the 3-D digital puppet on the display, wherein playing the audio data includes synchronizing the audio data with the animating of the 3-D digital puppet. 
   
     
     
         30 : The method of  claim 29 , further comprising:
 before presenting the 3-D digital puppet to the user, presenting a plurality of 3-D digital puppets to the user on the display; and   receiving a user input for selecting the 3-D digital puppet from the plurality of 3-D digital puppets.   
     
     
         31 : The method of  claim 29 , wherein the gesture made by the user is a hand gesture. 
     
     
         32 : The method of  claim 31 , wherein a physical accessory corresponding to the 3-D digital puppet is used by the user for the hand gesture. 
     
     
         33 : The method of  claim 31 , wherein a movement of fingers and a thumb of the hand gesture animates a mouth of the 3-D digital puppet. 
     
     
         34 : The method of  claim 29 , wherein the digital data further includes rotational data. 
     
     
         35 : The method of  claim 29 , further comprising:
 recording the animation of the 3-D digital puppet in a memory to create the 3-D digital puppet show.   
     
     
         36 : The method of  claim 29 , wherein the digital data further includes mouth control data. 
     
     
         37 : A system for animating a three-dimensional (3-D) digital puppet, the system comprising:
 a display; and   a processor configured to:
 present a three-dimensional (3-D) digital puppet to a user on a display; 
 receive motion data corresponding to a gesture made by the user, 
 receive audio data corresponding to voice of the user for use in a 3-D digital puppet show; 
 translate the motion data into digital data using a motion tracking algorithm; and 
 create the 3-D digital puppet show using the 3-D digital puppet by:
 animating the 3-D digital puppet, on the display, using the digital data; and 
 playing the audio data at the same time as the animating of the 3-D digital puppet on the display, wherein playing the audio data includes synchronizing the audio data with the animating of the 3-D digital puppet. 
 
   
     
     
         38 : The system of  claim 37 , wherein the processor is further configured to:
 before presenting the 3-D digital puppet to the user, present a plurality of 3-D digital puppets to the user on the display; and   receive a user input for selecting the 3-D digital puppet from the plurality of 3-D digital puppets.   
     
     
         39 : The system of  claim 37 , wherein the gesture made by the user is a hand gesture. 
     
     
         40 : The system of  claim 39 , wherein a physical accessory corresponding to the 3-D digital puppet is used by the user for the hand gesture.