Driving state monitoring methods and apparatuses, driver monitoring systems, and vehicles

Abstract

Embodiments of the present application disclose driving state monitoring methods and apparatuses, driver monitoring systems, and vehicles. The driving state monitoring method includes: performing driver state detection on a driver image; and performing at least one of: outputting a driving state monitoring result of a driver or performing intelligent driving control based on a result of the driver state detection. The embodiments of the present application can implement real-time monitoring of the driving state of a driver, so as to take corresponding measures in time when the driving state of the driver is poor, to ensure safe driving and avoid road traffic accidents.

1 . A driving state monitoring method, comprising:
 detecting, based on a driver image, a scheduled distraction action for a driver;   determining a distraction degree of the scheduled distraction action, in response to detecting the scheduled distraction action; and   performing an intelligent driving control and/or outputting a driving state monitoring result of a driver, based on the determined distraction degree,   wherein the detecting comprises:
 detecting a target object corresponding to the scheduled distraction action on the driver image to obtain a detection frame of the target object; and 
 detecting the scheduled distraction action based on the obtained detection frame. 
   
     
     
         2 . The method according to  claim 1 , wherein the determining comprises:
 obtaining a parameter value of an index for representing the distraction degree;   determining a detection result of the scheduled distraction action, based on the obtained parameter value; and   determining the distraction degree of the scheduled distraction action corresponding to the detection result.   
     
     
         3 . The method according to  claim 1 , wherein the scheduled distraction action comprises at least one of: a smoking action, a drinking action, an eating action, a phone call action, or an entertainment action. 
     
     
         4 . The method according to  claim 1 , wherein the parameter value comprises at least one of: a number of occurrences of the scheduled distraction action, duration of the scheduled distraction action, or a frequency of the scheduled distraction action. 
     
     
         5 . The method according to  claim 1 , wherein the target object comprises at least one of hands, mouth, eyes of a driver, or a target item. 
     
     
         6 . The method according to  claim 1 , wherein detecting the scheduled distraction action based on the obtained detection frame comprise:
 performing the target object detection corresponding to the eating action, drinking action, phone call action or entertainment action on the driver image via a neural network to obtain a detection frame for the target object; and   determining a detection result of the scheduled distraction action based on the detection frame for the target object; the detection result of the scheduled distraction action comprising one of: no occurrence of eating action, drinking action, phone call action or entertainment action, an occurrence of eating action, the drinking action, the phone call action, or the entertainment action.   
     
     
         7 . The method according to  claim 6 , wherein detecting the scheduled distraction action based on the obtained detection frame comprises:
 detecting a detection frame for the hands, a detection frame for the mouth, a detection frame for the eyes of the driver, and a detection frame for the target item, and   determining the scheduled distraction action based on whether the detection frame for the hand and the detection frame for the target object overlap, whether the distance between the detection frame for the target object and the detection frame for the mouth or the detection frame for the eyes meets a preset condition, and the type of the target object.   
     
     
         8 . The method according to  claim 7 , wherein determining the scheduled distraction action comprises:
 determining an eating action or a drinking action occurs, in response to the detection frame for the hand overlapping with the detection frame for the target object, the type of the target object being a container or a food, and the detection frame for the target object overlapping with the detection frame for the mouth.   
     
     
         9 . The method according to  claim 7 , wherein determining the scheduled distraction action comprises:
 determining that an entertainment action or a telephone call action occurs, in response to the detection frame for the hands overlapping with the detection frame for the target item, the type of the target item being an electronic device, and the minimum distance between the detection frame for the target item and the detection frame for the mouth being less than a first preset distance, or the minimum distance between the detection frame for the target item and the detection frame for the eyes being less than a second preset distance.   
     
     
         10 . The method according to  claim 6 , further comprising:
 determining that the detection result of the scheduled distraction action is that no eating action, drinking action, phone call action and entertainment action is detected, in response to the detection frame for the hands, the detection frame for the mouth, and the detection frame for any one target item being not detected simultaneously, and the detection frame for the hands, the detection frame for the eyes, and the detection frame for any one target item being not detected simultaneously.   
     
     
         11 . The method according to  claim 6 , further comprising:
 in response to the detection frame for the hands not overlapping with the detection frame for the target item, determining that the detection result of the scheduled distraction action is that no eating action, drinking action, phone call action, and entertainment action is detected.   
     
     
         12 . The method according to  claim 6 , further comprising:
 determining that the detection result of the scheduled distraction action is that no eating action, drinking action, phone call action, and entertainment action is detected, in response to ae type of the target item being a container or food and the detection frame for the target item not overlapping with the detection frame for the mouth, or the type of the target item being an electronic device and the minimum distance between the detection frame for the target item and the detection frame for the mouth being not less than the first preset distance, or the minimum distance between the detection frame for the target item and the detection frame for the eyes being not less than the second preset distance.   
     
     
         13 . The method according to  claim 1 , wherein the outputting comprises:
 determining a driving state level according to a preset condition that the result of the scheduled distraction action detection satisfy; and   using the determined driving state level as the driving state monitoring result.   
     
     
         14 . The method according to  claim 1 , wherein
 in response to determining that the determined driving state monitoring result satisfies a predetermined prompting or warning condition, outputting one or more prompting or warning information corresponding to the predetermined prompting or warning condition; or   in response to determining that the determined driving state monitoring result satisfies a predetermined driving mode switching condition, switching a driving mode to an automatic driving mode.   
     
     
         15 . The method according to  claim 1 , further comprising:
 performing facial recognition on the driver image; and   performing authentication control based on a result of the facial recognition.   
     
     
         16 . The method according to  claim 15 , wherein the performing facial recognition on the driver image comprises:
 performing face detection on the driver image via a neural network, and performing feature extraction on the detected face to obtain a face feature;   performing face matching between the face feature and face feature templates in a database; and   in response to determining that a face feature template matching the face feature exists in the database, outputting identity information corresponding to the face feature template matching the face feature.   in response to determining that no face feature template matching the face feature exists in the database, prompting the driver to register;   in response to receiving a registration request from the driver, performing face detection on the collected driver image via a neural network, and performing feature extraction on the detected face to obtain a face feature; and   establishing user information of the driver in the database by using the face feature as the face feature template of the driver.   
     
     
         17 . The method according to  claim 16 , further comprising:
 storing the driving state monitoring result in the user information of the driver in the database.   
     
     
         18 . The method according to  claim 1 , further comprising:
 performing image collection via an infrared camera deployed in at least one location within a vehicle;   wherein the at least one location comprises at least one of: a location above or near a dashboard, a location above or near a center console, an A-pillar or nearby location, or a rear-view mirror or nearby location.   
     
     
         19 . A driving state monitoring apparatus, comprising:
 a processor; and   a memory storing instructions, the instructions when executed by the processor, cause the processor to perform operations, the operations comprising:   detecting, based on a driver image, a scheduled distraction action for a driver;   determining a distraction degree of the scheduled distraction action, in response to detecting the scheduled distraction action; and   performing an intelligent driving control and/or outputting a driving state monitoring result of a driver, based on the determined distraction degree,   wherein the detecting comprises:
 detecting a target object corresponding to the scheduled distraction action on the driver image to obtain a detection frame of the target object; and 
 detecting the scheduled distraction action based on the obtained detection frame 
   
     
     
         20 . A non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program when executed by a processor, causes the processor to perform operations, the operations comprising:
 detecting, based on a driver image, a scheduled distraction action for a driver;   determining a distraction degree of the scheduled distraction action, in response to detecting the scheduled distraction action; and   performing an intelligent driving control and/or outputting a driving state monitoring result of a driver, based on the determined distraction degree,   wherein the detecting comprises:
 detecting a target object corresponding to the scheduled distraction action on the driver image to obtain a detection frame of the target object; and 
   detecting the scheduled distraction action based on the obtained detection frame.