Method for recognizing dangerous action of personnel in vehicle, electronic device and storage medium

Abstract

A method for recognizing a dangerous action of personnel in a vehicle, an electronic device, and a storage medium are provided. The method includes: obtaining at least one video stream of the personnel in the vehicle through an image capturing device, each video stream includes information about at least one of the personnel in the vehicle; performing action recognition on the personnel in the vehicle based on the video stream; and responsive to that a result of the action recognition belongs to a predetermined dangerous action, performing at least one of: sending prompt information, or executing an operation to control the vehicle, wherein the predetermined dangerous action includes at least one of the following action representations of the personnel in the vehicle: a distraction action, a discomfort state, or a non-standard behavior.

1 . A method for recognizing a dangerous action of personnel in a vehicle, comprising:
 obtaining at least one video stream of the personnel in the vehicle through an image capturing device, each video stream comprising information about at least one of the personnel in the vehicle;   performing action recognition on the personnel in the vehicle based on the video stream; and   responsive to that a result of the action recognition belongs to a predetermined dangerous action, performing at least one of: sending prompt information, or executing an operation to control the vehicle, wherein the predetermined dangerous action comprises at least one of the following action representations of the personnel in the vehicle: a distraction action, a discomfort state, or a non-standard behavior.   
     
     
         2 . The method according to  claim 1 , wherein performing the action recognition on the personnel in the vehicle based on the video stream comprises:
 detecting at least one target area comprised by the personnel in the vehicle, in at least one frame of video image of the video stream;   capturing a target image corresponding to the target area from the at least one frame of video image of the video stream according to the target area obtained through detection; and   performing action recognition on the personnel in the vehicle according to the target image.   
     
     
         3 . The method according to  claim 2 , wherein detecting the at least one target area comprised by the personnel in the vehicle, in the at least one frame of video image of the video stream comprises:
 extracting a feature, comprised in the at least one frame of video image of the video stream, of the personnel in the vehicle; and   extracting a target area from the at least one frame of video image based on the feature, wherein the target area comprises at least one of: a face local area, an action interactive object, or a limb area.   
     
     
         4 . The method according to  claim 3 , wherein the face local area comprises at least one of: a mouth area, an ear area, or an eye area. 
     
     
         5 . The method according to  claim 3 , wherein the action interactive object comprises at least one of: a container, a cigarette, a mobile phone, food, a tool, a beverage bottle, glasses, or a mask. 
     
     
         6 . The method according to  claim 1 , wherein the distraction action comprises at least one of: calling, drinking water, putting on or taking off sunglasses, putting on or taking off a mask, or eating food;
 the discomfort state comprises at least one of: wiping sweat, rubbing an eye, or yawning;   the non-standard behavior comprises at least one of: smoking, stretching a hand out of the vehicle, bending over a steering wheel, putting both feet on the steering wheel, leaving both hands away from the steering wheel, holding an instrument with a hand, or disturbing a driver.   
     
     
         7 . The method according to  claim 1 , wherein responsive to that the result of the action recognition belongs to the predetermined dangerous action, performing the at least one of: sending the prompt information, or executing the operation to control the vehicle comprises:
 responsive to that the result of the action recognition belongs to the predetermined dangerous action;   determining a danger level of the predetermined dangerous action; and   performing at least one of: sending corresponding prompt information according to the danger level, or executing an operation corresponding to the danger level and controlling the vehicle according to the operation.   
     
     
         8 . The method according to  claim 7 , wherein the danger level comprises a primary level, an intermediate level, and a high level;
 wherein performing the at least one of: sending the corresponding prompt information according to the danger level, or executing the operation corresponding to the danger level and controlling the vehicle according to the operation comprises:   sending the prompt information responsive to that the danger level is the primary level;   executing the operation corresponding to the danger level and controlling the vehicle according to the operation, responsive to that the danger level is the intermediate level; and   executing the operation corresponding to the danger level and controlling the vehicle according to the operation while sending the prompt information, responsive to that the danger level is the high level.   
     
     
         9 . The method according to  claim 7 , wherein determining the danger level of the predetermined dangerous action comprises:
 acquiring at least one of a frequency or a duration of occurrence of the predetermined dangerous action in the video stream, and determining the danger level of the predetermined dangerous action based on the at least one of the frequency or the duration.   
     
     
         10 . The method according to  claim 1 , wherein the result of the action recognition comprises a duration of an action, and a condition of belonging to the predetermined dangerous action comprises: recognizing that the duration of the action exceeds a duration threshold. 
     
     
         11 . The method according to  claim 1 , wherein the result of the action recognition comprises a number of times for which an action is performed, and a condition of belonging to the predetermined dangerous action comprises: recognizing that the number of times exceeds a number threshold. 
     
     
         12 . The method according to  claim 1 , wherein the result of the action recognition comprises a duration of an action and a number of times for which the action is performed, and a condition of belonging to the predetermined dangerous action comprises: recognizing that the duration of the action exceeds a duration threshold, and the number of times exceeds a number threshold. 
     
     
         13 . The method according to  claim 1 , wherein the personnel in the vehicle comprises at least one of a driver or a non-driver of the vehicle. 
     
     
         14 . The method according to  claim 13 , wherein responsive to that the result of the action recognition belongs to the predetermined dangerous action, performing the at least one of: sending the prompt information, or executing the operation to control the vehicle comprises at least one of:
 responsive to that the personnel in the vehicle is the driver, performing at least one of: sending corresponding first prompt information according to the predetermined dangerous action, or controlling the vehicle to execute a corresponding first predetermined operation according to the predetermined dangerous action; or   responsive to that the personnel in the vehicle is the non-driver, performing at least one of: sending corresponding second prompt information according to the predetermined dangerous action, or executing a corresponding second predetermined operation according to the predetermined dangerous action.   
     
     
         15 . An electronic device, comprising:
 a processor; and   a memory configured to store instructions that, when executed by the processor, cause the processor to perform the following operations comprising:   obtaining at least one video stream of personnel in a vehicle through an image capturing device, each video stream comprising information about at least one of the personnel in the vehicle;   performing action recognition on the personnel in the vehicle based on the video stream; and   responsive to that a result of the action recognition belongs to a predetermined dangerous action, performing at least one of: sending prompt information, or executing an operation to control the vehicle, wherein the predetermined dangerous action comprises at least one of the following action representations of the personnel in the vehicle: a distraction action, a discomfort state, or a non-standard behavior.   
     
     
         16 . The device according to  claim 15 , wherein the processor is configured to: detect at least one target area comprised by the personnel in the vehicle in at least one frame of video image of the video stream, capture a target image corresponding to the target area from the at least one frame of video image of the video stream according to the target area obtained through detection, and perform action recognition on the personnel in the vehicle according to the target image. 
     
     
         17 . The device according to  claim 16 , wherein the processor is configured to: extract a feature, comprised in the at least one frame of video image of the video stream, of the personnel in the vehicle when detecting the at least one target area comprised by the personnel in the vehicle in the at least one frame of video image of the video stream, and extract a target area from the at least one frame of video image based on the feature, wherein the target area comprises at least one of: a face local area, an action interactive object, or a limb area. 
     
     
         18 . The device according to  claim 15 , wherein the processor is configured to:
 determine a danger level of the predetermined dangerous action responsive to that the result of the action recognition belongs to the predetermined dangerous action; and   perform at least one of: sending corresponding prompt information according to the danger level, or executing an operation corresponding to the danger level and controlling the vehicle according to the operation.   
     
     
         19 . The device according to  claim 18 , wherein the danger level comprises a primary level, an intermediate level, and a high level; and
 the processor is configured to:   send prompt information responsive to that the danger level is the primary level;   execute the operation corresponding to the danger level and control the vehicle according to the operation, responsive to that the danger level is the intermediate level; and   execute the operation corresponding to the danger level and control the vehicle according to the operation while sending the prompt information, responsive to that the danger level is the high level.   
     
     
         20 . A non-transitory computer readable storage medium configured to store computer readable instructions that, when executed by a processor of an electronic device, cause the processor to perform a method for method for recognizing a dangerous action of personnel in a vehicle, comprising:
 obtaining at least one video stream of the personnel in the vehicle through an image capturing device, each video stream comprising information about at least one of the personnel in the vehicle;   performing action recognition on the personnel in the vehicle based on the video stream; and   responsive to that a result of the action recognition belongs to a predetermined dangerous action, performing at least one of: sending prompt information, or executing an operation to control the vehicle, wherein the predetermined dangerous action comprises at least one of the following action representations of the personnel in the vehicle: a distraction action, a discomfort state, or a non-standard behavior.