Apparatus, display system, gesture recognition method, and non-transitory recording medium

Abstract

An apparatus includes circuitry. The circuitry acquires information related to a shape of a pointing object from a sensor that acquires the information. The circuitry recognizes a gesture operation based on the acquired information. The gesture operation corresponds to a motion of the pointing object, and is recognizable in one of at least three layers in which a plurality of gesture operations are classified. When the circuitry recognizes a first gesture operation in a top layer of the at least three layers, the circuitry becomes ready to recognize a gesture operation classified in a second layer of the at least three layers. When the circuitry recognizes a second gesture operation in the second layer, the circuitry becomes ready to recognize a gesture operation classified in a third layer of the at least three layers.

1 . An apparatus comprising circuitry configured to
 acquire information related to a shape of a pointing object from a sensor that acquires the information, and   recognize a gesture operation based on the acquired information, the gesture operation corresponding to a motion of the pointing object and being recognizable in one of at least three layers in which a plurality of gesture operations are classified, wherein   when the circuitry recognizes a first gesture operation in a top layer of the at least three layers, the circuitry becomes ready to recognize a gesture operation classified in a second layer of the at least three layers, and   when the circuitry recognizes a second gesture operation in the second layer, the circuitry becomes ready to recognize a gesture operation classified in a third layer of the at least three layers.   
     
     
         2 . The apparatus of  claim 1 , wherein when the circuitry recognizes, in the second layer, a gesture operation for ending the recognition of the gesture operation classified in the second layer, the circuitry becomes ready to recognize a gesture operation classified in the top layer, and
 wherein when the circuitry recognizes, in the third layer, a gesture operation for ending the recognition of the gesture operation classified in the third layer, the circuitry becomes ready to recognize the gesture operation classified in the second layer.   
     
     
         3 . The apparatus of  claim 1 , wherein the circuitry causes transition from a current layer to an upper layer or a lower layer in the at least three layers. 
     
     
         4 . The apparatus of  claim 1 , wherein when the circuitry recognizes the pointing object in an initial state, the circuitry identifies the top layer and becomes ready to recognize a gesture operation classified in the top layer. 
     
     
         5 . The apparatus of  claim 4 , wherein the second layer includes a plurality of modes for using a plurality of particular features of the apparatus, and
 wherein when the circuitry recognizes the first gesture operation in the top layer, the circuitry causes transition to the second layer and to a mode of the plurality of modes in the second layer, the mode in the second layer being associated with the recognized first gesture operation.   
     
     
         6 . The apparatus of  claim 5 , further comprising:
 a display including a touch panel to display a drawn line input in handwriting on the touch panel with a pen or finger; and   a plurality of image capturing devices to capture a plurality of images of the display of the apparatus and a hand of a user, the hand of the user being the pointing object,   wherein when the circuitry recognizes, in the top layer, the first gesture operation to transition to a pointer mode included in the plurality of modes in the second layer, the circuitry   causes transition from the top layer to the pointer mode in the second layer,   analyzes image data of the plurality of images captured by the plurality of image capturing devices,   detects a plurality of coordinates on the display pointed by a forefinger of the user, and   displays a pointer at the detected plurality of coordinates on the display.   
     
     
         7 . The apparatus of  claim 4 , wherein the third layer includes a plurality of modes for using a plurality of particular features of the apparatus, and
 wherein when the circuitry recognizes the second gesture operation in the second layer, the circuitry causes transition to the third layer and to a mode of the plurality of modes in the third layer, the mode in the third layer being associated with the recognized second gesture operation.   
     
     
         8 . The apparatus of  claim 7 , wherein the second layer includes a plurality of modes for using a plurality of particular features of the apparatus, each mode of the plurality of modes in the second layer being associated with one or more second gesture operations recognizable by the circuitry, and
 wherein when the circuitry recognizes, in a mode of the plurality of modes in the second layer, a second gesture operation associated with the mode, the circuitry causes transition to the third layer and to the mode of the plurality of modes in the third layer, the mode in the third layer associated with the recognized second gesture operation.   
     
     
         9 . The apparatus of  claim 7 , wherein when the circuitry causes transition to a pen mode included in the plurality of modes in the third layer, the circuitry recognizes a gesture operation for displaying a drawn line on a display. 
     
     
         10 . The apparatus of  claim 9 , further comprising:
 the display; and   a plurality of image capturing devices to capture a plurality of images of the display and a hand of a user, the hand of the user being the pointing object,   wherein the circuitry
 analyzes image data of the plurality of images captured by the plurality of image capturing devices, 
 detects a plurality of coordinates on the display pointed by a forefinger of the user, and 
 displays the drawn line on the display as a trajectory of the detected plurality of coordinates, 
   wherein when the circuitry recognizes, based on the analyzed image data of the captured plurality of images, a gesture operation of pointing the forefinger at the display and moving a pen down in the pen mode, the circuitry identifies a pen-down mode, and   wherein when the circuitry recognizes, based on the analyzed image data of the captured plurality of images, a gesture operation of moving the pen up in the pen mode, the circuitry identifies a pen-up mode and stops drawing the drawn line.   
     
     
         11 . The apparatus of  claim 10 , wherein the circuitry displays a pen icon at the detected plurality of coordinates in the pen mode. 
     
     
         12 . The apparatus of  claim 11 , wherein when the circuitry recognizes, based on the analyzed image data of the captured plurality of images, the gesture operation of pointing the forefinger at the display and moving the pen down in the pen mode, the circuitry identifies the pen-down mode and displays the pen icon in a style different from a style used in a mode previous to the pen-down mode. 
     
     
         13 . The apparatus of  claim 7 , wherein when the circuitry causes transition to a marker mode included in the plurality of modes in the third layer, the circuitry recognizes a gesture operation for displaying a marker line on a display. 
     
     
         14 . The apparatus of  claim 13 , further comprising:
 the display; and   a plurality of image capturing devices to capture a plurality of images of the display and a hand of a user, the hand of the user being the pointing object,   wherein the circuitry
 analyzes image data of the plurality of images captured by the plurality of image capturing devices, and 
 detects a plurality of coordinates on the display pointed by a forefinger of the user, 
   wherein when the circuitry recognizes, based on the analyzed image data of the captured plurality of images, a gesture operation of pointing the forefinger at the display and moving a pen down in the marker mode, the circuitry identifies a pen-down mode, detects a plurality of coordinates on the display pointed by the forefinger, and displays the marker line as a trajectory of the detected plurality of coordinates, and   wherein when the circuitry recognizes, based on the analyzed image data of the captured plurality of images, a gesture operation of moving the pen up in the marker mode, the circuitry identifies a pen-up mode and stops drawing the marker line.   
     
     
         15 . The apparatus of  claim 14 , wherein the circuitry displays a marker icon at the detected plurality of coordinates in the marker mode. 
     
     
         16 . The apparatus of  claim 15 , wherein when the circuitry recognizes, based on the analyzed image data of the captured plurality of images, the gesture operation of pointing the forefinger at the display and moving the pen down in the marker mode, the circuitry identifies the pen-down mode and displays the marker icon in a style different from a style used in a mode previous to the pen-down mode. 
     
     
         17 . The apparatus of  claim 7 , wherein the plurality of modes in the third layer include a pen mode, a marker mode, and an eraser mode,
 wherein the circuitry recognizes, from an identical gesture operation, a gesture operation for moving a pen down in the pen mode, a gesture operation for moving a pen down in the marker mode, and a gesture operation for bringing a virtual eraser into contact with a display of the apparatus in the eraser mode, and   wherein the circuitry recognizes, from another identical gesture operation, a gesture operation for moving the pen up in the pen mode, a gesture operation for moving the pen up in the marker mode, and a gesture operation for releasing the virtual eraser from the display in the eraser mode.   
     
     
         18 . A display system comprising:
 an apparatus to recognize a gesture operation corresponding to a motion of a pointing object and receive an operation according to the gesture operation; and   an information processing system to communicate with the apparatus via a network,   the apparatus including
 first circuitry configured to acquire information related to a shape of the pointing object from a sensor that acquires the information, and 
 a first network interface circuit to transmit the information to the information processing system, and 
   the information processing system including
 second circuitry configured to analyze the information received from the apparatus and recognize the gesture operation based on the information acquired by the sensor, the gesture operation being recognizable in one of at least three layers in which a plurality of gesture operations are classified, and 
 a second network interface circuit to report to the apparatus a layer of the at least three layers corresponding to the recognized gesture operation, wherein 
   when the second circuitry of the information processing system recognizes a first gesture operation in a top layer of the at least three layers, the second circuitry of the information processing system becomes ready to recognize a gesture operation classified in a second layer of the at least three layers,   when the second circuitry of the information processing system recognizes a second gesture operation in the second layer, the second circuitry of the information processing system becomes ready to recognize a gesture operation classified in a third layer of the at least three layers, and   the first circuitry of the apparatus executes a process in the layer reported from the information processing system.   
     
     
         19 . A gesture recognition method comprising:
 acquiring information related to a shape of a pointing object from a sensor; and   recognizing a gesture operation based on the acquired information, the gesture operation corresponding to a motion of the pointing object and being recognizable in one of at least three layers in which a plurality of gesture operations are classified,   wherein, when the recognizing recognizes a first gesture operation in a top layer of the at least three layers, the method further comprises:   becoming ready to recognize a gesture operation classified in a second layer of the at least three layers, and   wherein, when the recognizing recognizes a second gesture operation in the second layer, the method further comprises:   becoming ready to recognize a gesture operation classified in a third layer of the at least three layers.   
     
     
         20 . A non-transitory recording medium storing a plurality of instructions which, when executed by one or more processors, causes the one or more processors to perform the method according to  claim 19 .