Eye closure detection using structured illumination

Abstract

A monitoring system and method for monitoring and/or predicting drowsiness of a driver of a vehicle or a machinery operator are provided. A set of infrared, IR, or near infrared, NIR, light sources, is arranged such that an amount of the light emitted from the light source incidents on an eye of the driver or operator. The light that impinges on the eye of the driver or operator forms a virtual image of the signal sources on the eye including the sclera and/or cornea. An image sensor obtains consecutive images capturing the reflected light, wherein each image will contain glints from at least a subset or all of the light sources. A drowsiness index can be determined based on the extracted information of the glints of the sequence of images, wherein the drowsiness index indicates a degree of drowsiness of the driver or operator.

That which is claimed is: 
     
         1 . A monitoring system for monitoring or predicting drowsiness of a driver of a vehicle or a machinery operator, comprising:
 a set of infrared or near infrared light sources, each light source arranged to direct light towards an eye of a driver or operator, wherein the emitted light from the set of light sources forms a glint pattern on the eye;   an image sensor arranged such that light reflected from the eye can be captured in an image;   an image processing unit adapted to extract information of the light reflected from the eye; and   a determining unit adapted to:
 determine multiple glints in the glint pattern of the light reflected from the eye, 
 determine a number of the multiple glints visible at a point in time, 
 determine an amplitude measure, the amplitude measure indicating an amplitude of an eye opening or eye closure, the amplitude measure being based on the number of the multiple glints visible from the glint pattern over time, and 
 determine a drowsiness index value based on the amplitude measure and the glint pattern. 
   
     
     
         2 . The monitoring system according to  claim 1 , wherein the set of light sources are synchronized. 
     
     
         3 . The monitoring system according to  claim 1 , wherein the glint pattern includes one or more of light emitted from one of the set of light sources, or light emitted from one of the set of light sources that is reflected from a surface. 
     
     
         4 . The monitoring system according to  claim 1 , wherein said determining unit is adapted to alert the driver or operator when the drowsiness index value exceeds a predetermined drowsiness index threshold. 
     
     
         5 . The monitoring system according to  claim 4 , further including an alert system, and wherein said determining unit is adapted to instruct said alert system to notify the driver or operator when the drowsiness index value has exceeded the predetermined drowsiness index threshold, with the notification being delivered by an alert signal comprising at least one of: an alert tone, an alert sound, an alert light signal, or a vibration signal. 
     
     
         6 . The monitoring system according to  claim 1 , wherein the amplitude measure corresponds to one or more of:
 a spatial measure indicating a distance between a first glint in the glint pattern and an additional glint in the glint pattern;   a frequency measure based on a spectral width of the glint pattern; or   a measure of overlap based on a wavelet base function.   
     
     
         7 . The monitoring system according to  claim 1 , wherein:
 the image sensor is further arranged to capture a sequence of images, each image in the sequence including a glint image of the glint pattern; and   the determining unit is further adapted to determine the amplitude measure based on the sequence of glint images.   
     
     
         8 . The monitoring system according to  claim 1 , wherein the set of light sources are configured to emit infrared or near infrared light at a specific wavelength. 
     
     
         9 . The monitoring system according to  claim 8 , wherein the specific wavelength is one of 750 nm, 850 nm and 950 nm. 
     
     
         10 . A method for monitoring and predicting drowsiness of a driver of a vehicle or a machinery operator, comprising:
 emitting light using a set of infrared or near infrared light sources, each light source being arranged to direct light towards an eye of a driver or operator, wherein the emitted light from the set of light sources forms a glint pattern on the eye;   capturing light reflected from the eye in an image using an image sensor;   processing the images to extract information of the light reflected from the eye;   determining multiple glints in the glint pattern of the light reflected from the eye;   determining a number of the multiple glints visible at a point in time;   determining an amplitude measure, the amplitude measure indicating an amplitude of an eye opening or eye closure, the amplitude measure being based on the number of the multiple glints visible from the glint pattern over time; and   determining a drowsiness index value based on the amplitude measure and the glint pattern.   
     
     
         11 . The method according to  claim 10 , wherein the set of light sources are synchronized. 
     
     
         12 . The method according to  claim 10 , wherein the glint pattern includes one or more of light emitted from one of the set of light sources, or light emitted from one of the set of light sources that is reflected from a surface. 
     
     
         13 . The method according to  claim 10 , further comprising alerting the driver or operator when the drowsiness index value exceeds a predetermined drowsiness index threshold. 
     
     
         14 . The method according to  claim 13 , further comprising instructing an alert system to notify the driver or operator when the drowsiness index value has exceeded the predetermined drowsiness index threshold, with the notification being delivered by an alert signal comprising at least one of: an alert tone, an alert sound, an alert light signal, or a vibration signal. 
     
     
         15 . The method according to  claim 10 , wherein the amplitude measure corresponds to one or more of:
 a spatial measure indicating a distance between a first glint in the glint pattern and an additional glint in the glint pattern;   a frequency measure based on a spectral width of the glint pattern; or   a measure of overlap based on a wavelet base function.   
     
     
         16 . The method according to  claim 10 , wherein:
 the image sensor is arranged to capture a sequence of images, each image in the sequence including a glint image of the glint pattern; and   the method further comprises determining the amplitude measure based on the sequence of glint images.   
     
     
         17 . The method according to  claim 10 , wherein the set of light sources are configured to emit infrared or near infrared light at a specific wavelength. 
     
     
         18 . The method according to  claim 17 , wherein the specific wavelength is one of 750 nm, 850 nm and 950 nm. 
     
     
         19 . A non-transitory computer-readable storage medium storing computer-executable instructions for performing operations for monitoring or predicting drowsiness of a driver of a vehicle or a machinery operator, the operations comprising:
 emitting light using a set of infrared or near infrared light sources, each light source being arranged to direct light towards an eye of a driver or operator, wherein the emitted light from the set of light sources forms a glint pattern on the eye;   capturing light reflected from the eye in an image using an image sensor;   processing the images to extract information of the light reflected from the eye;   determining multiple glints in the glint pattern of the light reflected from the eye;   determining a number of the multiple glints visible at a point in time;   determining an amplitude measure, the amplitude measure indicating an amplitude of an eye opening or eye closure, the amplitude measure being based on the number of the multiple glints visible from the glint pattern over time; and   determining a drowsiness index value based on the amplitude measure and the glint pattern.   
     
     
         20 . The non-transitory computer-readable storage medium according to  claim 19 , wherein:
 the image sensor is arranged to capture a sequence of images, each image in the sequence including a glint image of the glint pattern; and   the operations further comprise determining the amplitude measure based on the sequence of glint images.