Systems And Methods For Detecting And Dynamically Mitigating Driver Fatigue

Abstract

This technology relates to dynamically detecting, managing and mitigating driver fatigue in autonomous systems. For instance, interactions of a driver in a vehicle may be monitored to determine a distance or time when primary tasks associated with operation of the vehicle or secondary tasks issued by the vehicle computing were last performed. If primary tasks or secondary tasks are not performed within given distance thresholds or time limits, then one or more secondary tasks are initiated by the computing device of the vehicle. In another instance, potential driver fatigue, driver distraction or overreliance on an automated driving system is detected based on gaze direction or pattern of a driver. For example, a detected gaze direction or pattern may be compared to an expected gaze direction or pattern given the surrounding environment in a vicinity of the vehicle

1 . A method for managing driver inattention, the method comprising:
 acquiring, by one or more processors, images of one or both of a driver's eyes to determine an actual gaze pattern of the driver of a vehicle;   comparing, by the one or more processors, the actual gaze pattern to an expected gaze pattern of the driver;   determining, by the one or more processors based at least in part on the comparison, a fatigue state of the driver; and   in response to the determined fatigue state of the driver, initiating, by the one or more processors, an action to be taken with the driver.   
     
     
         2 . The method of  claim 1 , further comprising:
 determining, by the one or more processors, the expected gaze pattern by using a normative model based on environmental statistics and a planned driving path.   
     
     
         3 . The method of  claim 1 , further comprising:
 determining, by the one or more processors, the expected gaze pattern by using a normative model based on human visual behavior statistics.   
     
     
         4 . The method of  claim 1 , further comprising:
 determining, by the one or more processors, the expected gaze pattern of the driver based on a planned route of the vehicle.   
     
     
         5 . The method of  claim 5 , wherein the determining of the fatigue state is based on whether the driver's visual scanning of an external environment of the vehicle deviates from the expected gaze pattern. 
     
     
         6 . The method of  claim 1 , wherein the determining of the fatigue state further comprises:
 determining, by the one or more processors based on the comparison, whether the driver's visual scanning of the external environment deviates from the expected gaze pattern, wherein the fatigue state is determined based on deviation of the driver's visual scanning of the external environment from the expected gaze pattern.   
     
     
         7 . The method of  claim 1 , further comprising:
 determining, by the one or more processors, the expected gaze pattern of the driver based on objects in an external environment of the vehicle   
     
     
         8 . The method of  claim 1 , wherein the acquiring of images comprises monitoring one or both of the driver's eyes for eye closure. 
     
     
         9 . The method of  claim 1 , wherein the images are acquired by the one or more processors through use of cameras within a compartment of the vehicle. 
     
     
         10 . The method of  claim 1 , further comprising using a scanning model to determine where the driver should be looking based on visual stimuli in a field of view. 
     
     
         11 . The method of  claim 10 , wherein the scanning model provides a benchmark against which a measured visual scanning pattern of the driver can be evaluated. 
     
     
         12 . A vehicle comprising:
 a driving system including a steering subsystem, an acceleration subsystem and a deceleration subsystem to control driving of the vehicle;   a perception system including one or more sensors configured to detect objects in an environment external to the vehicle;   a positioning system configured to determine a current position of the vehicle; and   a control system including one or more processors, the control system operatively coupled to the driving system, the perception system and the positioning system, the control system being configured to:
 acquire, from the perception system, images of one or both of a driver's eyes to determine an actual gaze pattern of the driver; 
 compare the actual gaze pattern to an expected gaze pattern; 
 determine, based at least in part on the comparison, a fatigue state of the driver; and 
 in response to the determined fatigue state of the driver, initiate an action to be taken with the driver. 
   
     
     
         13 . The vehicle of  claim 12 , wherein the control system is further configured to determine the expected gaze pattern by using a normative model based on environmental statistics and a planned driving path. 
     
     
         14 . The vehicle of  claim 12 , wherein the control system is further configured to determine the expected gaze pattern by using a normative model based on human visual behavior statistics. 
     
     
         15 . The vehicle of  claim 12 , wherein the control system is further configured to determine the expected gaze pattern of the driver based on a planned route of the vehicle. 
     
     
         16 . The vehicle of  claim 15 , wherein the control system is further configured to determine the fatigue state based on whether the driver's visual scanning of an external environment of the vehicle deviates from the expected gaze pattern. 
     
     
         17 . The vehicle of  claim 12 , wherein the control system is further configured to determine the fatigue state by determining, based on the comparison, whether the driver's visual scanning of the external environment deviates from the expected gaze pattern, wherein the fatigue state is determined based on deviation of the driver's visual scanning of the external environment from the expected gaze pattern. 
     
     
         18 . The vehicle of  claim 12 , wherein the control system is further configured to determine the expected gaze pattern of the driver based on objects in an external environment of the vehicle 
     
     
         19 . The vehicle of  claim 12 , wherein the control system acquires the images by monitoring one or both of the driver's eyes for eye closure. 
     
     
         20 . The vehicle of  claim 12 , further comprising one or more cameras within a compartment of the vehicle, the one or more cameras being configured to acquire the images