Systems, devices, and methods for interacting with content displayed on head-mounted displays

Abstract

Systems, devices, and methods that enable sophisticated and inconspicuous interactions with content displayed on a head-mounted display are described. A head-mounted display includes an eye-tracker and the user also carries/wears a wireless portable interface device elsewhere on their body, such as a ring. The wireless nature of the portable interface device enables a small and unobtrusive form factor. The portable interface device includes an actuator that, when activated by the user, causes the portable interface device to wirelessly transmit a signal (e.g., a radio frequency signal or a sonic signal). A selection operation performed by the user is defined as the user activating the actuator of the portable interface device while the user is substantially concurrently gazing at a displayed object (as detected by the eye-tracker). In response to the selection operation, the head-mounted display displays a visual effect to the user.

1 . A system that enables interaction with content displayed on a head-mounted display, the system comprising:
 a head-mounted display including:
 at least one display positioned in a field of view of at least one eye of a user when the head-mounted display is worn on a head of the user; 
 a processor communicatively coupled to the at least one display; 
 a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable instructions and/or data that, when executed by the processor, cause the at least one display to display at least one object that is responsive to a selection operation performed by the user; and 
 a wireless receiver to wirelessly receive signals, the wireless receiver communicatively coupled to the processor; 
   an eye-tracker communicatively coupled to the processor, the eye-tracker to detect that the user is gazing at the at least one object displayed by the at least one display; and   a wireless portable interface device with a form factor to be carried by or on the user, the portable interface device physically separate from the head-mounted display, wherein the portable interface device includes at least one actuator that, when activated by the user, causes the portable interface device to wirelessly transmit a signal.   
     
     
         2 . The system of  claim 1  wherein:
 the selection operation performed by the user comprises a substantially concurrent combination of gazing at the least one object displayed by the at least one display and activating the at least actuator of the portable interface device; and 
 in response to the selection operation performed by the user, a visual effect is displayed on the at least one display of the head-mounted display. 
 
     
     
         3 . The system of  claim 2  wherein the non-transitory processor-readable storage medium further stores processor-executable instructions and/or data that, when executed by the processor in response to the wireless receiver wirelessly receiving the signal from the portable interface device, cause the processor to:
 request current gaze direction data from the eye-tracker; 
 identify a particular object at which the user is gazing based on the current gaze direction data received from the eye-tracker, the particular object identified among the at least one object displayed by the at least one display; and 
 cause the at least one display to display the visual effect on the particular object. 
 
     
     
         4 . The system of  claim 1  wherein the portable interface device is batteryless and includes a piezoelectric element communicatively coupled to the actuator and an antenna communicatively coupled to the piezoelectric element, and wherein the wireless receiver of the head-mounted display includes a radio frequency receiver. 
     
     
         5 . The system of  claim 4  wherein:
 when activated by the user, the actuator mechanically actuates the piezoelectric element, 
 in response to the mechanical actuation, the piezoelectric element generates an electric signal, and 
 in response to the electric signal generated by the piezoelectric element, the antenna transmits a radio frequency signal. 
 
     
     
         6 . The system of  claim 1  wherein the portable interface device is batteryless and includes a mechanical resonator physically coupled to the actuator, and wherein the wireless receiver of the head-mounted display includes at least one of: a microphone and/or a piezoelectric element tuned to be responsive to a sonic signal. 
     
     
         7 . The system of  claim 6  wherein:
 when activated by the user, the actuator mechanically actuates the mechanical resonator, and 
 in response to the mechanical actuation, the mechanical resonator generates the sonic signal. 
 
     
     
         8 . The system of  claim 1  wherein the portable interface device includes an on-board power source and a radio frequency transmitter to transmit radio signals having a frequency in a frequency range of 10 MHz to 10 GHz, and wherein the wireless receiver of the head-mounted display includes a radio frequency receiver tuned to receive the radio signals. 
     
     
         9 . The system of  claim 7  wherein the sonic signal includes an ultrasonic signal. 
     
     
         10 . The system of  claim 1  wherein the object includes at least one object selected from a group consisting of: a menu item, a graphical button, a keyboard key, a notification, one of multiple objects displayed by the at least one display of the head-mounted display, a file, a folder, and an alphanumeric character. 
     
     
         11 . The system of  claim 1  wherein the portable interface device is a wearable device selected from a group consisting of: a ring, a wristband, and an armband. 
     
     
         12 . The system of  claim 1  wherein the actuator of the portable interface device includes a button. 
     
     
         13 . The system of  claim 1  wherein the eye-tracker is carried by and physically coupled to the head-mounted display. 
     
     
         14 . A method of operating a system, wherein the system comprises a head-mounted display, an eye-tracker, and a wireless portable interface device, the portable interface device physically separate from the head-mounted display, the method comprising:
 displaying an object within a field of view of at least one eye of a user by at least one display of the head-mounted display;   receiving a selection operation from the user by the system, wherein receiving a selection operation from the user by the system includes:
 detecting, by the eye tracker, that the user is gazing at the object; 
 while detecting, by the eye tracker, that the user is gazing at the object, receiving, by the portable interface device, an activation from the user of an actuator of the portable interface device; 
 in response to receiving, by the portable interface device, the activation from the user of the actuator of the portable interface device, wirelessly transmitting a signal by the portable interface device; and 
 wirelessly receiving the signal by a wireless receiver of the head-mounted display; and 
   in response to receiving the selection operation from the user by the system, displaying a visual effect by the at least one display of the head-mounted display.   
     
     
         15 . The method of  claim 14  wherein the head-mounted display includes a processor and a non-transitory processor-readable storage medium communicatively coupled to the processor, and wherein the non-transitory processor-readable storage medium stores processor-executable instructions and/or data and the method comprises executing, by the processor, the processor-executable instructions and/or data stored in the non-transitory processor-readable storage medium to cause:
 the at least one display of the head-mounted display to display the object within the field of view of at least one eye of the user; and 
 the at least one display of the head-mounted display to display the visual effect in response to the system receiving the selection operation from the user. 
 
     
     
         16 . The method of  claim 14  wherein:
 the portable interface device is batteryless and includes a piezoelectric element communicatively coupled to the actuator and an antenna communicatively coupled to the piezoelectric element, 
 the wireless receiver of the head-mounted display includes a radio receiver, 
 receiving, by the portable interface device, an activation from the user of an actuator of the portable interface device includes receiving, by the portable interface device, a mechanical actuation of the piezoelectric element from the user, 
 in response to the mechanical actuation, the piezoelectric element generates an electric signal, 
 wirelessly transmitting a signal by the portable interface device includes wirelessly transmitting a radio signal by the antenna of the portable interface device in response to the electric signal generated by the piezoelectric element, and 
 wirelessly receiving the signal by a wireless receiver of the head-mounted display includes wirelessly receiving the radio signal by the radio receiver of the head-mounted display. 
 
     
     
         17 . The method of  claim 14  wherein:
 the portable interface device is batteryless and includes a mechanical resonator physically coupled to the actuator, 
 receiving, by the portable interface device, an activation from the user of an actuator of the portable interface device includes receiving, by the portable interface device, a mechanical actuation of the mechanical resonator from the user, 
 in response to the mechanical actuation, the mechanical resonator generates a sonic signal, 
 wirelessly transmitting a signal by the portable interface device includes wirelessly transmitting the sonic signal by mechanical resonator of the portable interface device, and 
 wirelessly receiving the signal by a wireless receiver of the head-mounted display includes wirelessly receiving the sonic signal by the wireless receiver of the head-mounted display, and wherein the wireless receiver of the head-mounted display includes at least one of: a microphone and/or a piezoelectric element tuned to be responsive to the sonic signal. 
 
     
     
         18 . The method of  claim 17  wherein the sonic signal generated by the mechanical resonator of the portable interface device includes an ultrasonic signal. 
     
     
         19 . The method of  claim 14  wherein:
 the portable interface device includes an on-board power source and a radio frequency transmitter; 
 wirelessly transmitting a signal by the portable interface device includes wirelessly transmitting a radio signal by the radio frequency transmitter of the portable interface device, the radio signal having a frequency in a range of 10 MHz to 10 GHz, and 
 wirelessly receiving the signal by a wireless receiver of the head-mounted display includes wirelessly receiving the radio signal by the wireless receiver of the head-mounted display, and wherein the wireless receiver of the head-mounted display includes a radio frequency receiver. 
 
     
     
         20 . The method of  claim 14  wherein the object includes at least one object selected from a group consisting of: a menu item, a graphical button, a keyboard key, a notification, one of multiple objects displayed by the at least one display of the head-mounted display, a file, a folder, and an alphanumeric character. 
     
     
         21 . The method of  claim 14  wherein the portable interface device is a wearable device selected from a group consisting of: a ring, a wristband, and an armband. 
     
     
         22 . A completely wearable human-electronics interface comprising:
 a wearable heads-up display including a wireless receiver;   an eye-tracker carried by the wearable heads-up display; and   a wearable actuator including a wireless transmitter to transmit wireless signals to the wireless receiver of the wearable heads-up display.   
     
     
         23 . The completely wearable human-electronics interface of  claim 22  wherein the wearable heads-up display further includes a processor that is communicatively coupled to both the eye-tracker and the wireless receiver, the processor to effect interactions with content displayed by the wearable heads-up display in response to concurrent inputs from both the eye-tracker and the wearable actuator.