Skeletal control of three-dimensional virtual world

Abstract

A virtual skeleton includes a plurality of joints and provides a machine readable representation of a human target observed with a three-dimensional depth camera. A relative position of a hand joint of the virtual skeleton is translated as a gestured control, and a three-dimensional virtual world is controlled responsive to the gestured control.

1 . A data holding device holding instructions executable by a logic device, the instructions comprising:
 instructions to render a three-dimensional environment for display on a display device;   instructions to receive a machine-readable virtual skeleton structurally representing a human sighted by a depth camera having a field of view, the virtual skeleton including a hand-joint position in three dimensions corresponding to a position of a hand of the human anywhere within the field of view;   instructions to render a control cursor for display on the display device, a screen position of the control cursor moving based on corresponding movement of the hand-joint position;   instructions to lock the control cursor to an object in the three-dimensional environment if a grab threshold of the object is overcome;   instructions to, when the control cursor is locked to the object, change a screen position of the object based on corresponding movement of the hand-joint position, such that movement of the hand anywhere within the field of view effects a corresponding movement of the object in the three-dimensional environment; and   instructions to unlock the control cursor from the object at a release position within the three-dimensional environment if a release threshold of the object is overcome.   
     
     
         2 . The data holding device of  claim 1 , where the grab threshold of the object is overcome if the hand is closed. 
     
     
         3 . The data holding device of  claim 1 , where the grab threshold of the object is overcome if the control cursor is within a threshold distance of the object for a threshold duration. 
     
     
         4 . The data holding device of  claim 1 , where the grab threshold of the object is overcome if the control cursor is within a threshold distance of the object and a speed of the control cursor is below a threshold speed for a threshold duration. 
     
     
         5 . The data holding device of  claim 1 , where the release threshold is overcome if the hand is open. 
     
     
         6 . The data holding device of  claim 1 , wherein the instructions further comprise:
 instructions to, when the control cursor is locked to the object, rotate the object in the three-dimensional environment responsive to a rotation gesture of the virtual skeleton.   
     
     
         7 . The data holding device of  claim 1 , wherein the instructions further comprise:
 instructions to, when the control cursor is locked to the object, scale the object in the three-dimensional environment responsive to a scale gesture of the virtual skeleton.   
     
     
         8 . The data holding device of  claim 1 , wherein the object is one of a plurality of objects in the three-dimensional environment selected responsive to a selection gesture of the virtual skeleton, and wherein the instructions further comprise:
 instructions to lock the control cursor to the plurality of objects if a grab threshold of the plurality of objects is overcome;   instructions to, when the control cursor is locked to the plurality of objects, change a screen position of the plurality of objects based on corresponding movement of the hand-joint position; and   instructions to unlock the control cursor from the plurality of objects if a release threshold of the plurality of objects is overcome.   
     
     
         9 . The data holding device of  claim 8 , wherein the instructions further comprise:
 instructions to, when the control cursor is locked to the plurality of objects, rotate the plurality of objects in the three-dimensional environment responsive to a rotation gesture of the virtual skeleton.   
     
     
         10 . The data holding device of  claim 8 , wherein the instructions further comprise:
 instructions to, when the control cursor is locked to the plurality of objects, scale the plurality of objects in the three-dimensional environment responsive to a scale gesture of the virtual skeleton.   
     
     
         11 . The data holding device of  claim 1 , wherein the instructions further comprise:
 instructions to recognize a zoom-in gesture in which right and left hands of the human are separated; and   instructions to magnify a view of the three-dimensional environment on the display responsive to the zoom-in gesture.   
     
     
         12 . The data holding device of  claim 1 , wherein the instructions further comprise:
 instructions to recognize a zoom-out gesture in which right and left hands of the human are brought closer; and   instructions to demagnify a view of the three-dimensional environment on the display responsive to the zoom-out gesture.   
     
     
         13 . A method comprising:
 displaying a three-dimensional environment on a display device;   receiving a machine-readable virtual skeleton structurally representing a human sighted by a depth camera having a field of view, the virtual skeleton including a hand-joint position in three dimensions corresponding to a position of a hand of the human anywhere within the field of view;   displaying a control cursor on the display device, a screen position of the control cursor moving based on corresponding movement of the hand-joint position;   locking the control cursor to an object in the three-dimensional environment if a grab threshold of the object is overcome;   when the control cursor is locked to the object, changing a screen position of the object based on corresponding movement of the hand-joint position, such that movement of the hand anywhere within the field of view effects a corresponding movement of the object in the three-dimensional environment; and   unlocking the control cursor from the object at a release position within the three-dimensional environment if a release threshold of the object is overcome.   
     
     
         14 . The method of  claim 13 , where the grab threshold of the object is overcome if the hand is closed. 
     
     
         15 . The method of  claim 13 , where the grab threshold of the object is overcome if the control cursor is within a threshold distance of the object for a threshold duration. 
     
     
         16 . The method of  claim 13 , where the grab threshold of the object is overcome if the control cursor is within a threshold distance of the object and a speed of the control cursor is below a threshold speed for a threshold duration. 
     
     
         17 . The method of  claim 13 , where the release threshold is overcome if the hand is open. 
     
     
         18 . The method of  claim 13 , further comprising, when the control cursor is locked to the object, rotating the object in the three-dimensional environment responsive to a rotation gesture of the virtual skeleton. 
     
     
         19 . The method of  claim 13 , further comprising, when the control cursor is locked to the object, scaling the object in the three-dimensional environment responsive to a scale gesture of the virtual skeleton. 
     
     
         20 . A data holding device holding instructions executable by a logic device, the instructions comprising:
 instructions to render a three-dimensional environment for display on a display device;   instructions to receive a machine-readable virtual skeleton structurally representing a human sighted by a depth camera having a field of view, the virtual skeleton including a hand-joint position in three dimensions corresponding to a position of a hand of the human being anywhere within the field of view;   instructions to lock the hand joint to an object in the three-dimensional environment if a grab threshold of the object is overcome;   instructions to, when the hand joint is locked to the object, move the object in all three dimensions to track the hand-joint position, such that movement of the hand anywhere within the field of view effects a corresponding movement of the object in the three-dimensional environment; and   instructions to unlock the control cursor from the object at a release position of the object within the three-dimensional environment if a release threshold of the object is overcome.