Position Capture Input Apparatus, System and Method Therefor

Abstract

According to various embodiments, a position capture input system uses a camera to capture an image of a displayed graphical user interface that may be partially obstructed by an object, such as a user's hand or other body part. The position capture input system also includes a software component that causes a computing device to compare the captured image with a displayed image to determine which portion, if any, of the graphical user interface is obstructed. The computing device can then identify any user interface elements with which the user is attempting to interact. The position capture input system may also include an accelerometer or accelerometers for detecting gestures performed by the user to, for example, select or otherwise interact with a user interface element. The position capture input system may also include a haptic feedback module to provide confirmation, for example, that a user interface element has been selected.

1 . A position capture input system comprising:
 a first camera configured to capture an image of a graphical user interface from a perspective of a user, the graphical user interface comprising a plurality of user interface elements; and   a computing device in communication with the first camera and configured to
 receive the captured image; 
 compare the captured image to a displayed image generated by the computing device; 
 based on the comparison between the captured image and the displayed image, determine a position of an obstruction relative to the graphical user interface; and 
 based on the position of the obstruction relative to the graphical user interface, determine whether a user is interacting with a user interface element of the graphical user interface. 
   
     
     
         2 . The position capture input system of  claim 1 , further comprising an additional camera separated from the first camera and in communication with the computing device and configured to capture an additional image of the graphical user interface from the perspective of the user and to provide the additional image to the computing device. 
     
     
         3 . The position capture input system of  claim 1 , further comprising a plurality of lenses adapted to cause the displayed image to have a three-dimensional appearance. 
     
     
         4 . The position capture input system of  claim 1 , further comprising at least one accelerometer in communication with the computing device and configured to detect motion of the obstruction and to provide at least one signal to the computing device for sensing motion of the obstruction in at least one dimension, wherein the computing device is configured to determine whether the user is interacting with a user interface element of the graphical user interface based in part on the at least one signal. 
     
     
         5 . The position capture input system of  claim 1 , further comprising a haptic feedback module in communication with the computing device and configured to provide haptic feedback to the user. 
     
     
         6 . The position capture input system of  claim 1 , further comprising a calibration control configured to initiate a calibration process. 
     
     
         7 - 20 . (canceled) 
     
     
         21 . The position capture input system of  claim 1 , wherein the first camera is in a headworn device supportable on a head of a human user, the headworn device comprising a plurality of lenses which incorporate LCD elements. 
     
     
         22 . A position-capture input apparatus comprising:
 a first camera configured to capture an image of a field of view from a perspective of a human user, wherein the image captured by first camera includes at least part of what the user is viewing; and   a computing device in communication with the first camera, wherein the computing device includes:
 a calibration unit configured to output a first image on a display screen, and to receive, from the first camera, a captured image of the first image of the display screen in the field of view from the perspective of the user, wherein the calibration unit is configured to perform calibrations on an ongoing basis to accommodate for changes in the user's perspective; 
 a graphical-user-interface (GUI) unit configured to output a GUI image that includes plurality of user-interface icons and to receive, from the first camera, a captured image of the GUI image from the perspective of the user; 
 a compare unit configured to compare the captured image to the GUI image to detect an obstruction in the captured image of the GUI image; 
 a map unit configured to map a position of the detected obstruction to a location in the GUI image using results obtained from the calibration unit; and 
 an interaction unit configured to determine whether a user is interacting with one of the plurality of a user interface icons of the GUI image based on the mapped position of the obstruction. 
   
     
     
         23 . A position-capture input method comprising:
 capturing an image of a field of view from a perspective of a human user, wherein the captured image includes at least part of what the user is viewing from the perspective of the user; and   outputting a first image on a display screen;   receiving a captured image of the first image of the display screen in the field of view from the perspective of the user;   performing calibrations on an ongoing basis to accommodate for changes in the user's perspective;   outputting a GUI image that includes plurality of user-interface icons;   receiving a captured image of the GUI image from the perspective of the user;   comparing the captured image to the GUI image to detect an obstruction in the captured image of the GUI image;   mapping a position of the detected obstruction to a location in the GUI image using results obtained from the calibration unit; and   determining whether a user is interacting with one of the plurality of a user interface icons of the GUI image based on the mapped position of the obstruction.   
     
     
         24 . The method of  claim 23 , where the calibration screen can be viewed by at least one camera, but not by the user. 
     
     
         25 . The method of  claim 23 , wherein the computing device is configured to recognize an obstruction as a hand. 
     
     
         26 . The method of  claim 23 , wherein the computing device is configured to recognize portions of the obstruction as various parts of the hand. 
     
     
         27 . The method of  claim 23 , wherein the computing device is configured to recognize the behavior of the obstruction, or portions of the obstruction, as hand or finger gestures. 
     
     
         28 . The method of  claim 23 , wherein the mapped point or points of portions of the obstruction, and action being performed on the interface, are determined based on the gesture being used. 
     
     
         29 . The method of  claim 23 , wherein hand gestures are recognized using 3d information produced by two or more cameras or by other methods or sensors used to provide 3d information to the computing device.