Calibration free, motion tolerent eye-gaze direction detector with contextually aware computer interaction and communication methods

Abstract

Eye tracking systems and methods include such exemplary features as a display device, at least one image capture device and a processing device. The display device displays a user interface including one or more interface elements to a user. The at least one image capture device detects a user's gaze location relative to the display device. The processing device electronically analyzes the location of user elements within the user interface relative to the user's gaze location and dynamically determine whether to initiate the display of a zoom window. The dynamic determination of whether to initiate display of the zoom window may further include analysis of the number, size and density of user elements within the user interface relative to the user's gaze location, the application type associated with the user interface or at the user's gaze location, and/or the structure of eye movements relative to the user interface.

1 . An eye gaze detection system, comprising:
 a display device configured to display a user interface to a user, said user interface comprising one or more interface elements;   at least one image capture device for detecting a user's gaze location relative to said display device; and   a processing device configured to electronically analyze the location of user elements within the user interface relative to the user's gaze location and dynamically determine whether to initiate the display of a zoom window;   wherein said processing device is further configured to electronically analyze one or more of the number, size and density of user elements within the user interface relative to the user's gaze location as part of the dynamic determination of whether to initiate the display of a zoom window.   
     
     
         2 . (canceled) 
     
     
         3 . The eye gaze detection system of  claim 1 , wherein said processing device is further configured to electronically analyze the application type associated with said user interface or at the user's gaze location as part of the dynamic determination of whether to initiate the display of a zoom window. 
     
     
         4 . The eye gaze detection system of  claim 1 , wherein said processing device is further configured to determine the level of magnification of said zoom window relative to said user interface based on one or more of the size, number, location and density of interface elements relative to the user's gaze location. 
     
     
         5 . The eye gaze detection system of  claim 1 , wherein said processing device is configured to implement the display of a focus region on the display device around the user's gaze location on the user interface or within the zoom window. 
     
     
         6 . The eye gaze detection system of  claim 5 , wherein said focus region is configured to match the size of one or more interface elements within the user interface or zoom window. 
     
     
         7 . The eye gaze detection system of  claim 5 , wherein said processing device is configured to implement the display of a visual feedback element at a designated location within the focus region while a user's gaze location remains anywhere within the focus region. 
     
     
         8 . The eye gaze detection system of  claim 1 , wherein said processing device is further configured to electronically analyze the structure of eye movements detected by said at least one image capture device as part of the dynamic determination of whether to initiate the display of a zoom window. 
     
     
         9 . A method for automatically initiating user interface magnification within an electronic device, said method comprising the steps of:
 electronically detecting the presence of one or more interface elements in a user interface relative a user's gaze point on the user interface;   electronically determining the density of interface elements around the user's gaze point; and   automatically initiating the display of a zoom window if the electronically determined density of interface elements exceeds a predetermined density threshold level, said zoom window comprising a magnified view of a portion of said user interface.   
     
     
         10 . The method of  claim 9 , further comprising receiving electronic input from a user to customize the predetermined density threshold level for initiating the zoom window. 
     
     
         11 . The method of  claim 9 , wherein the degree of magnification within the zoom window is determined by the density and size of selected interface elements in the user interface. 
     
     
         12 . The method of  claim 9 , further comprising displaying a focus region around the user's gaze location on the user interface or zoom window. 
     
     
         13 . The method of  claim 12 , wherein said focus region is configured to match the size of one or more interface elements within the user interface or zoom window. 
     
     
         14 . The method of  claim 12 , further comprising displaying a visual feedback element at a designated location within the focus region while a user's gaze point remains anywhere within the focus region. 
     
     
         15 . The method of  claim 12 , wherein the timing of said step of automatically initiating the display of a zoom window is based in part on the length of time a user's gaze location remains within said focus region. 
     
     
         16 . The method of  claim 9 , further comprising implementing an electronic reaction associated with a given interface element provided in the zoom window after detection of a predetermined action relative to the given interface element. 
     
     
         17 . The method of  claim 16 , wherein said predetermined action comprises one or more of blinking, fixating of user gaze for a predetermined dwell time, actuating a button or switch, and speaking a command. 
     
     
         18 . The method of  claim 9 , further comprising a step of initiating further magnification of said user interface within said zoom window. 
     
     
         19 . A computer readable medium comprising computer readable and executable instructions configured to control a processing device to implement the method of  claim 9 . 
     
     
         20 . An eye gaze detection system, comprising:
 a display device configured to display a user interface to a user, said user interface comprising one or more interface elements;   at least one image capture device for detecting a user's gaze location relative to said display device; and   a processing device configured to detect user interface elements within the user interface relative to the user's gaze location and dynamically determine whether to initiate the display of one or more visual feedback elements on the user interface at or near the user's gaze location, wherein such dynamic determination is made based on whether the user's gaze location is at or within a predetermined distance of an interface element.   
     
     
         21 . The eye gaze detection system of  claim 20 , wherein said processing device is further configured to snap the one or more visual feedback elements on the user interface to one or more interface elements that are determined to be closest to the user's gaze location. 
     
     
         22 . The eye gaze detection system of  claim 20 , wherein said processing device is further configured to initiate an electronic action associated with the interface element within the user interface that is at or closest to the user's gaze location. 
     
     
         23 . The eye gaze detection system of  claim 20 , wherein said one or more visual feedback elements comprise a pointer placed on the user's gaze location. 
     
     
         24 . The eye gaze detection system of  claim 20 , wherein said one or more visual feedback elements comprise an overlying image having its center of origin substantially corresponding to the user's gaze location. 
     
     
         25 . The eye gaze detection system of  claim 20 , wherein said one or more visual feedback elements comprise one or more highlighted regions corresponding to one or more interface elements near a user's gaze location. 
     
     
         26 . The eye gaze detection system of  claim 20 , wherein said processing device is configured to detect user interface elements by determining whether any interface elements present data for initiating commands to an operating system running on said processing device. 
     
     
         27 . The eye gaze detection system of  claim 20 , wherein said processing device is configured to detect user interface elements by applying pattern recognition techniques to identify user interface elements having one or more predefined shapes. 
     
     
         28 . The eye gaze detection system of  claim 20 , wherein said processing device is further configured to use detected interface elements as input to possible scanning choices for selection by a user employing a scanning access method for the eye gaze detection system. 
     
     
         29 . The eye gaze detection system of  claim 20 , wherein the dynamic determination made by said processing device regarding whether to initiate the display of one or more visual feedback elements on the user interface is further dependent on the structure of eye movements detected by said at least one image capture device. 
     
     
         30 . The eye gaze detection system of  claim 20 , wherein the dynamic determination made by said processing device regarding whether to initiate the display of one or more visual feedback elements on the user interface is further dependent on the type of action associated with one or more interface elements near the user's gaze location, and wherein different feedback elements can be displayed for different types of actions. 
     
     
         31 . A method for displaying and updating visual feedback elements in an eye tracking system, said method comprising:
 electronically detecting a user's gaze location corresponding to where a user is looking relative to a user interface;   electronically determining whether any reactable interface elements are pointed at or within a predetermined distance from the user's gaze location; and   electronically displaying one or more visual feedback elements on the user interface at or near the user's gaze location if one or more reactable interface elements are found at or within a predetermined distance from the user's gaze location.   
     
     
         32 . The method of  claim 31 , further comprising snapping the one or more visual feedback elements on the user interface to one or more reactable interface elements that are determined to be closest to the user's gaze location. 
     
     
         33 . The method of  claim 31 , further comprising electronically initiating an electronic action associated with the reactable interface element within the user interface that is at or closest to the user's gaze location. 
     
     
         34 . The method of  claim 31 , wherein said one or more user interface elements comprise a pointer placed on the user's gaze location, an overlying image having its center of origin substantially corresponding to the user's gaze location, or one or more highlighted regions corresponding to one or more interface elements near a user's gaze location. 
     
     
         35 . The method of  claim 31 , wherein electronically determining whether any interface elements are pointed at or within a predetermined distance from the user's gaze location comprises determining whether any interface elements present data for initiating commands to an operating system running on said processing device. 
     
     
         36 . The method of  claim 31 , wherein electronically determining whether any interface elements are pointed at or within a predetermined distance from the user's gaze location comprises applying pattern recognition techniques to identify user interface elements having one or more predefined shapes. 
     
     
         37 . The method of  claim 31 , further comprising electronically displaying a focus region around the user's gaze location, wherein said focus region is configured to match the size of a reactable interface element within the user interface that is at or closest to the user's gaze location. 
     
     
         38 . The method of  claim 31 , further comprising a step of electronically implementing an additional action relative to a reactable interface element within the user interface that is at or closest to the user's gaze location. 
     
     
         39 . The method of  claim 31 , further comprising using reactable interface elements as input to possible scanning choices for selection by a user employing a scanning access method for the eye tracking system. 
     
     
         40 . The method of  claim 31 , wherein electronically displaying a visual feedback element on the user interface at the user's gaze location occurs based on additional analysis of the structure of eye movements determined by detecting the user's gaze location. 
     
     
         41 . A computer readable medium comprising computer readable and executable instructions configured to control a processing device to implement the method of  claim 31 . 
     
     
         42 - 105 . (canceled)