Method and apparatus for interface control with prompt and feedback

Abstract

To prompt input and provide feedback on input to a user with an interface, inputs and graphical cursors associated with those inputs are defined. Each input may have several forms such as base, hover, engaged, completed, and error. User input is anticipated. The base form of the anticipated input cursor is displayed to prompt the user for the anticipated input. If user hover is detected that matches anticipated input, the hover form is displayed to confirm the match to the user. If user input is detected that matches anticipated input, the engaged form is displayed as confirmation. If user input is completed that matches anticipated input, the completed form is displayed as confirmation. If user hover or input does not match anticipated input, the error form is displayed to indicate mismatch. Not all cursors must have all forms, and some cursors may have multiples of some forms.

1 . A method, comprising:
 associating, by a processing device, a first graphical cursor with a first type of gesture;   associating, by the processing device, a second graphical cursor with a second type of gesture, wherein the first graphical cursor and the second graphical cursor are graphically distinct from each other;   associating, by the processing device, a first virtual object with the first graphical cursor and the first type of gesture, wherein the first type of gesture is an interaction with the first virtual object;   displaying, by a head-mounted display, the first virtual object with the first graphical cursor;   displaying a first base form of the first graphical cursor;   detecting, by a sensor, a first viewer input from a viewer of the head-mounted display;   determining, by the processing device, whether a form of the first viewer input detected by the sensor is:
 a first appearance that indicates the viewer is preparing to initiate the first type of gesture; 
 a second appearance that indicates the viewer is preparing to initiate the second type of gesture; or 
 a third appearance that indicates the viewer is preparing to initiate an unknown type of gesture; 
   in response to the first viewer input indicating the viewer is preparing to initiate the second type of gesture or the unknown type of gesture, displaying, by the head-mounted display, a first indicator that the first type of gesture is anticipated; and   iteratively detecting the first viewer input by the sensor and displaying, by the head-mounted display, the first indicator until the first viewer input indicates the viewer is preparing to initiate the first type of gesture; and   in response to the first viewer input indicating the viewer is preparing to initiate the first type of gesture, displaying a confirmation that the first viewer input corresponds to the anticipated first type of gesture, wherein the confirmation comprises an engaged form of the graphical cursor.   
     
     
         2 . The method of  claim 1 , wherein:
 the first graphical cursor comprises a first hover form, or a first engaged form; and   the second graphical cursor comprises a second base form that is different than the first base form, a second hover form that is different than the first hover form, or a second engaged form that is different than the first engaged form.   
     
     
         3 . The method of  claim 1 , further comprising, in response to the first viewer input indicating the viewer is preparing to initiate the first type of gesture, displaying, by the head-mounted display, a second indicator that the first viewer input is a correct form of input. 
     
     
         4 . The method of  claim 1 , wherein the first type of gesture is a hand gesture performed in a free space removed from a physical surface. 
     
     
         5 . The method of  claim 1 , further comprising, in response to the first viewer input indicating the viewer is preparing to initiate the second type of gesture or the unknown type of gesture, displaying, by the head-mounted display, a graphical emphasis of a portion of the first graphical cursor indicating a non-correspondence between the anticipated first type of gesture and the first viewer input detected by the sensor. 
     
     
         6 . The method of  claim 1 , wherein the first indicator is an error indication that indicates to the viewer a non-correspondence between the first viewer input detected by the sensor and the first type of gesture. 
     
     
         7 . The method of  claim 1 , further comprising:
 associating, by the processing device, a second virtual object with the first graphical cursor and the first type of gesture, wherein the first type of gesture is an interaction with the second virtual object;   displaying, by the head-mounted display, the second virtual object with the first graphical;   detecting, by the sensor, a second viewer input from the viewer of the head-mounted display;   determining whether a form of the second viewer input detected by the sensor is:
 a fourth appearance that indicates the viewer is preparing to initiate the first type of gesture; 
 a fifth appearance that indicates the viewer is preparing to initiate the second type of gesture; or 
 a sixth appearance that indicates the viewer is preparing to initiate the unknown type of gesture; 
   in response to the second viewer input indicating the viewer is preparing to initiate the second type of gesture or the unknown type of gesture, displaying, by the head-mounted display, a second indicator that the first type of gesture is anticipated; and   iteratively detecting the second viewer input and displaying the second indicator until the second viewer input indicates the viewer is preparing to initiate the first type of gesture.   
     
     
         8 . The method of  claim 1 , further comprising:
 associating, by the processing device, the first virtual object with the second graphical cursor and the second type of gesture, wherein the second type of gesture is another interaction with the first virtual object;   displaying, by the head-mounted display, the first virtual object with the second graphical cursor;   detecting, by the sensor, a second viewer input from the viewer of the head-mounted display;   determining whether a form of the second viewer input detected by the sensor is:
 a fourth appearance that indicates the viewer is preparing to initiate the first type of gesture; 
 a fifth appearance that indicates the viewer is preparing to initiate the second type of gesture; or 
 a sixth appearance that indicates the viewer is preparing to initiate the unknown type of gesture; 
   in response to indicating the viewer is preparing to initiate the first type of gesture or the unknown type of gesture, displaying, by the head-mounted display, a second indicator that the second type of gesture is anticipated; and   iteratively detecting the second viewer input and displaying the second indicator until the second viewer input indicates the viewer is preparing to initiate the second type of gesture.   
     
     
         9 . The method of  claim 1 , wherein the first type of gesture or the second type of gesture is defined by:
 a degree of finger separation between fingers of a hand of the viewer;   a bend in a knuckle of the hand;   a curve in a motion path of the hand or a finger of the hand; or   a direction of the motion path.   
     
     
         10 . The method of  claim 9 , wherein the unknown type of gesture is defined by:
 the degree of finger separation between the fingers of the hand of the viewer exceeding a defined threshold for the degree of finger separation;   the bend in the knuckle of the hand exceeding a defined threshold for the bend in the knuckle;   the curve in the motion path of the hand or the finger of the hand being different from a defined curve for the motion path of the hand or the finger; or   the direction of the motion path being different than a defined direction for the motion path.   
     
     
         11 . The method of  claim 1 , wherein the first graphical cursor or the second graphical cursor comprises:
 a hollow circle with dashed crosshair marks disposed around a periphery thereof;   a hollow circle with contracted dashed crosshair marks disposed around a periphery thereof;   a filled circle;   a filled circle with at least one concentric circle thereabout;   a hollow circle with arrow marks disposed around a periphery thereof;   a filled circle with arrow marks disposed around a periphery thereof;   a filled circle with arrow marks disposed around a periphery thereof with at least one of the arrow marks comprising at least two arrows;   a hollow dashed circle with arrow marks disposed around a periphery thereof and pointing outward therefrom;   a filled dashed circle with arrow marks disposed around a periphery thereof and pointing outward therefrom with each of the arrow marks comprising at least two arrows;   a hollow dashed circle with arrow marks disposed within a periphery thereof and pointing inward therefrom; or   a filled dashed circle with arrow marks disposed within a periphery thereof and pointing inward therefrom with each of the arrow marks comprising at least two arrows.   
     
     
         12 . The method of  claim 1 , wherein:
 the first graphical cursor comprises a first base form and a first non-base form; or   the second graphical cursor comprises a second base form and a second non-base form.   
     
     
         13 . The method of  claim 1 , wherein the first type of gesture is a single finger gesture and the second type of gesture is a multiple finger gesture. 
     
     
         14 . The method of  claim 1 , wherein the first graphical cursor or the second graphical cursor comprises a hover form or a base form. 
     
     
         15 . A device, comprising:
 a wearable display configured to display a virtual object;   a sensor configured to detect a viewer input from a viewer of the wearable display;   a processing device coupled to the wearable display and the sensor, wherein the processing device is configured to:
 establish a first type of gesture for interacting with objects in a virtual reality environment; 
 establish a second type of gesture for interacting with the objects in the virtual reality environment; 
 associate a first virtual object with the first type of gesture, wherein the first type of gesture is an interaction with the first virtual object; 
 display, by the wearable display, the first virtual object; 
 display a base form of a graphical cursor, the graphical cursor associated with the first type of gesture; 
 detect, by the sensor, a first viewer input from a viewer of the wearable display; 
 determine whether a form of the first viewer input detected by the sensor is:
 a first appearance that indicates the viewer is preparing to initiate the first type of gesture; 
 a second appearance that indicates the viewer is preparing to initiate the second type of gesture; or 
 a third appearance that indicates the viewer is preparing to initiate an unknown type of gesture; and 
 
 in response to the first viewer input indicating the viewer is preparing to initiate the first type of gesture, display a confirmation that the first viewer input corresponds to the anticipated first type of gesture, wherein the confirmation comprises an engaged form of the graphical cursor. 
   
     
     
         16 . The device of  claim 15 , wherein the processing device is further configured to, in response to the first viewer input indicating the viewer is preparing to initiate the second type of gesture or the unknown type of gesture, display, by the wearable display, an indicator that the first type of gesture is anticipated. 
     
     
         17 . The device of  claim 16 , wherein the processing device is further configured to iteratively detect the first viewer input and display the indicator until the first type of gesture is detected by the sensor.