Methods and systems of extended reality environment interaction based on eye motions

Abstract

Systems and methods are described for extended reality environment interaction. An extended reality environment including an object is generated for display, and a table of eyelid motion identifiers and corresponding actions performable on the object in the extended reality environment is stored in memory. An eyelid motion is detected by using a sensor, and the detected eyelid motion is matched to one of the stored eyelid motion identifiers. An updated version of the extended reality environment, based on the action that corresponds to the matched eyelid motion, is generated for display, in response to matching the detected eyelid motion to one of the stored eyelid motion identifiers.

1 - 2 . (canceled) 
     
     
         3 . A method for extended reality (XR) environment interaction, comprising:
 generating for display an XR environment comprising an object;   detecting, using one or more sensors, a first eye motion directed towards the object, wherein the first eye motion is associated with a first period of time;   comparing the first period of time to a threshold period of time;   based at least in part on determining that the first period of time is less than the threshold period of time, refraining from performing one or more actions associated with the object;   detecting, using the one or more sensors, a second eye motion directed towards the object, wherein the second eye motion is associated with a second period of time, and wherein the second eye motion is detected subsequent to the detection of the first eye motion;   comparing the second period of time to the threshold period of time; and   based at least in part on determining that the second period of time is equal to or greater than the threshold period of time, performing the one or more actions associated with the object.   
     
     
         4 . The method of  claim 3 , wherein the object is an identifier for a media asset, and wherein the one or more actions associated with the object comprises selecting the media asset for playing. 
     
     
         5 . The method of  claim 3 , wherein:
 the XR environment is generated for display at a head-mounted device (HMD); and   detecting the first eye motion comprises determining that one or more eyes of a user wearing the HMD were closed for less than the threshold period of time, the method further comprising:
 determining that the first eye motion is an involuntary blink of the user, based at least in part on determining that the one or more eyes of the user were closed for less than the threshold period of time. 
   
     
     
         6 . The method of  claim 3 , wherein:
 the XR environment is generated for display at a head-mounted device (HMD); and   detecting the second eye motion comprises determining that one or more eyes of a user wearing the HMD were closed for at least the threshold period of time, the method further comprising:
 determining that the second eye motion is a voluntary blink of the user wearing the HMD, based at least in part on determining that the one or more eyes of the user were closed for at least the threshold period of time. 
   
     
     
         7 . The method of  claim 3 , wherein:
 the XR environment is generated for display at a head-mounted device (HMD); and   detecting the first eye motion comprises determining a blink pattern of one or more eyes of a user wearing the HMD over the first period of time, the method further comprising:
 based at least in part on determining that the one or more eyes of the user were closed for less than the threshold period of time during the first period of time, determining that the first eye motion corresponds to involuntary blinks of the user over the first period of time. 
   
     
     
         8 . The method of  claim 3 , wherein:
 the XR environment is generated for display at a head-mounted device (HMD); and   detecting the first eye motion comprises determining a blink pattern of one or more eyes of a user wearing the HMD over the second period of time, the method further comprising:
 based at least in part on determining that the one or more eyes of the user were closed for at least the threshold period of time during the second period of time, determining that the second eye motion corresponds to voluntary blinks of the user over the second period of time. 
   
     
     
         9 . The method of  claim 3 , wherein the one or more actions associated with the object comprises regenerating for display the object with a modified level of detail by presenting the object in a higher resolution. 
     
     
         10 . The method of  claim 3 , further comprising:
 determining a type associated with the object; and   performing the one or more actions based at least in part on the type associated with the object.   
     
     
         11 . The method of  claim 3 , wherein the object is associated with an entity, and wherein the one or more actions comprises transmitting a request for media content related to the entity. 
     
     
         12 . The method of  claim 3 , further comprising determining the second period of time based at least in part on:
 determining that the object is in a field of view of a user; and   determining that the object is continuously in the field of view of the user for the second period of time during an XR session.   
     
     
         13 . The method of  claim 3 , wherein the threshold period of time is a predetermined period of time. 
     
     
         14 . A system for extended reality (XR) environment interaction, comprising:
 a memory; and   a control circuitry configured to:
 generate for display an XR environment comprising an object; 
 detect, using one or more sensors, a first eye motion directed towards the object, wherein the first eye motion is associated with a first period of time; 
 compare the first period of time to a threshold period of time; 
 based at least in part on determining that the first period of time is less than the threshold period of time, refrain from performing one or more actions associated with the object, wherein the one or more actions associated with the object are stored in the memory; 
 detect, using the one or more sensors, a second eye motion directed towards the object, wherein the second eye motion is associated with a second period of time, and wherein the second eye motion is detected subsequent to the detection of the first eye motion; 
 compare the second period of time to the threshold period of time; and 
 based at least in part on determining that the second period of time is equal to or greater than the threshold period of time, perform the one or more actions associated with the object. 
   
     
     
         15 . The system of  claim 14 , wherein the object is an identifier for a media asset, and wherein the one or more actions associated with the object comprises selecting the media asset for playing. 
     
     
         16 . The system of  claim 14 , wherein the control circuitry is configured to:
 generate the XR environment for display at a head-mounted device (HMD); and   detect the first eye motion by determining that one or more eyes of a user wearing the HMD were closed for less than the threshold period of time; and
 wherein the control circuitry is further configured to: 
   determine that the first eye motion is an involuntary blink of the user, based at least in part on determining that the one or more eyes of the user were closed for less than the threshold period of time.   
     
     
         17 . The system of  claim 14 , wherein the control circuitry is configured to:
 generate the XR environment for display at a head-mounted device (HMD); and   detect the second eye motion by determining that one or more eyes of a user wearing the HMD were closed for at least the threshold period of time; and   wherein the control circuitry is further configured to:   determine that the second eye motion is a voluntary blink of the user wearing the HMD, based at least in part on determining that the one or more eyes of the user were closed for at least the threshold period of time.   
     
     
         18 . The system of  claim 14 , wherein the control circuitry is configured to:
 generate the XR environment for display at a head-mounted device (HMD); and   detect the first eye motion by determining a blink pattern of one or more eyes of a user wearing the HMD over the first period of time; and   wherein the control circuitry is further configured to:   based at least in part on determining that the one or more eyes of the user were closed for less than the threshold period of time during the first period of time, determine that the first eye motion corresponds to involuntary blinks of the user over the first period of time.   
     
     
         19 . The system of  claim 14 , wherein the control circuitry is configured to:
 generate the XR environment for display at a head-mounted device (HMD); and   detect the first eye motion by determining a blink pattern of one or more eyes of a user wearing the HMD over the second period of time; and   wherein the control circuitry is further configured to:   based at least in part on determining that the one or more eyes of the user were closed for at least the threshold period of time during the second period of time, determine that the second eye motion corresponds to voluntary blinks of the user over the second period of time.   
     
     
         20 . The system of  claim 14 , wherein the one or more actions associated with the object comprises regenerating for display the object with a modified level of detail by presenting the object in a higher resolution. 
     
     
         21 . The system of  claim 14 , wherein the control circuitry is further configured to:
 determine a type associated with the object; and   perform the one or more actions based at least in part on the type associated with the object.   
     
     
         22 . The system of  claim 14 , wherein the control circuitry is further configured to:
 regenerate for display the object with a modified level of detail different from an initial level of detail;   based at least in part on matching the second eye motion to a particular eyelid motion identifier of a plurality of eyelid motion identifiers stored in a table, cause performance of an action indicated in the table as corresponding to the particular eyelid motion identifier;   detect that a gaze has shifted from a first portion of the XR environment to a second portion of the XR environment, the object being excluded from the first portion of the XR environment and included in the second portion of the XR environment;   based at least in part on detecting the gaze shift, generate for display within the XR environment an indicator of the gaze shift;   detect a voice command while the indicator is in a vicinity of the object and executing an action corresponding to the voice command;   provide for display, within the XR environment, a plurality of opacity-based indicators in the vicinity of the first portion of the XR environment;   vary an opacity of at least one of the plurality of opacity-based indicators; and   based at least in part on determining that the object is in a field of view for at least the threshold period of time, generate for display in the XR environment one or more items related to the object.