Head-Mounted Display That Displays A Visual Representation Of Physical Interaction With An Input Interface Located Outside Of The Field Of View

Abstract

A head-mounted display (HMD) displays a visual representation of a physical interaction with an input interface that is located outside of the field of view. In one embodiment, the visual representation includes symbols that indicate when close proximity or physical contact is made with the input interface. In another embodiment, the visual representation is a simulation of the physical interaction with the input interface. The visual representation displayed by the HMD can beneficially enable the wearer to interact with the input interface more efficiently.

1 . A head-mountable display (HMD), comprising:
 a display, wherein the display is configured to provide generated images;   an input interface mounted on the HMD such that, when the HMD is worn by a wearer, the input interface is located outside of the a field of view of the wearer; and   a controller, wherein the controller is configured to:
 (a) receive an image of a manipulator and a record of a physical interaction between the manipulator and the input interface, wherein the physical interaction is outside of the field of view; 
 (b) generate, based at least in part on the image, a visual representation of the physical interaction between the manipulator and the input interface; and 
 (c) cause the display to show, inside of the field of view, the visual representation of the physical interaction between the manipulator and the input interface. 
   
     
     
         2 . The HMD of  claim 1 , wherein the display comprises at least one lens configured to define the field of view, and wherein the input interface comprises a touch pad mounted on at least one side portion extending from the at least one lens. 
     
     
         3 . The HMD of  claim 1 , wherein the record further comprises a visual record captured by a camera, and wherein the visual record comprises at least one of: a still image, a video, or both a still image and a video. 
     
     
         4 . The HMD of  claim 1 , wherein the physical interaction between the manipulator and the input interface comprises the manipulator interacting with the input interface, and wherein an indication of interacting with the input interface is provided in the visual representation. 
     
     
         5 . The HMD of  claim 1 , wherein the visual representation of the physical interaction between the manipulator and the input interface comprises a first graphic representing a realistic simulation of a hand and a second graphic representing the input interface. 
     
     
         6 . The HMD of  claim 5 , wherein the visual representation of the physical interaction between the manipulator and the input interface shows the first graphic moving relative to the second graphic to indicate movement of the manipulator relative to the input interface, and wherein the visual representation provides an indication when input data is received by the input interface. 
     
     
         7 . The HMD of  claim 5 , wherein the first graphic further comprises one or more colored dots overlaying the second graphic for representing positions of the manipulator relative to the input interface. 
     
     
         8 . A method performed using a head-mountable display (HMD), wherein the HMD includes a graphic display configured to provide generated images, the method comprising:
 (a) receiving an image of a manipulator and a record of a physical interaction between the manipulator and a touch pad, wherein, when the HMD is worn by a wearer, the physical interaction is outside of a field of view of the wearer;   (b) generating, based at least in part of the image, a visual representation of the physical interaction between the manipulator and the touch pad; and   (c) displaying, inside of the field of view, the visual representation of the physical interaction between the manipulator and the touch pad.   
     
     
         9 . The method of  claim 8 , wherein the graphic display comprises at least one lens configured to define the field of view, and wherein the touch pad is mounted on at least one side portion extending from the at least one lens. 
     
     
         10 . The method of  claim 8 , wherein the record further comprises a visual record captured by a camera, and wherein the visual record comprises at least one of: a still image, a video, or both a still image and a video. 
     
     
         11 . The method of  claim 8 , wherein the physical interaction between the manipulator and the touch pad comprises the manipulator touching the touch pad, and wherein an indication of touching the touch pad is provided in the visual representation. 
     
     
         12 . The method of  claim 8 , wherein the visual representation of the physical interaction between the manipulator and the touch pad comprises a first graphic representing a realistic simulation of a hand and a second graphic representing the touch pad. 
     
     
         13 . The method of  claim 12 , wherein the visual representation of the physical interaction between the manipulator and the touch pad shows the first graphic moving relative to the second graphic to indicate movement of the manipulator relative to the touch pad, and wherein the visual representation provides an indication when input data is received by the touch pad. 
     
     
         14 . The method of  claim 12 , wherein the first graphic further comprises one or more colored dots overlaying the second graphic for representing positions of the manipulator relative to the touch pad. 
     
     
         15 . A computing device, comprising:
 a semi-transparent display, wherein the semi-transparent display is configured to display generated images;   a touch pad mounted on the computing device such that, when the computing device is worn by a wearer, the touch pad is located outside of a field of view of the wearer;   at least one processor;   a non-transitory computer-readable medium; and   program instructions stored in the non-transitory computer-readable medium and executable by the at least one processor to cause the computing device to:
 (a) receive an image of a manipulator and a record of a physical interaction between the manipulator and the touch pad, wherein the physical interaction is outside of the field of view; 
 (b) generate, based at least in part on the image, a visual representation of the physical interaction between the manipulator and the touch pad; and 
 (c) cause the semi-transparent display to show, inside of the field of view, the visual representation of the physical interaction between the manipulator and the touch pad. 
   
     
     
         16 . The computing device of  claim 15 , wherein the semi-transparent display comprises at least one lens configured to define the field of view, and wherein the touch pad is mounted on at least one side portion extending from the at least one lens. 
     
     
         17 . The computing device of  claim 15 , wherein the record comprises a visual record captured by a camera, and wherein the visual record comprises at least one of: a still image, a video, or both a still image and a video. 
     
     
         18 . The computing device of  claim 15 , wherein the physical interaction between the manipulator and the touch pad comprises the manipulator touching the touch pad, and wherein an indication of touching the touch pad is provided in the visual representation. 
     
     
         19 . The computing device of  claim 15 , wherein the visual representation of the physical interaction between the manipulator and the touch pad comprises a first graphic representing a realistic simulation of a hand and a second graphic representing the touch pad. 
     
     
         20 . The computing device of  claim 19 , wherein the visual representation of the physical interaction between the manipulator and the touch pad shows the first graphic moving relative to the second graphic to indicate movement of the manipulator relative to the touch pad, and wherein the visual representation provides when input data is received by the touch pad.