System and method for detecting rainfall for an autonomous vehicle

Abstract

A system includes an autonomous vehicle and a control device associated with the autonomous vehicle. The control device obtains a plurality of sensor data captured by sensors of the autonomous vehicle. The control device determines a plurality of rainfall levels based on the sensor data. Each rainfall level is captured by a different sensor. the control device determines an aggregated rainfall level in a particular time period by combining the plurality of rainfall levels determined during the particular time period. The control device selects a particular object detection algorithm for detecting objects by at least one sensor. The particular object detection algorithm is configured to filter at least a portion of interference caused by the aggregated rainfall level in the sensor data. The control device causes the particular object detection algorithm to be implemented for the at least one sensor.

1 . A system comprising:
 a memory configured to store a plurality of sensor data that provides information about rainfall; and   at least one processor operably coupled to the memory, and configured to at least:
 obtain the plurality of sensor data from a plurality of sensors associated with an autonomous vehicle; 
 determine a plurality of rainfall levels based at least in part upon the plurality of sensor data, wherein each rainfall level from among the plurality of rainfall levels is captured by a different sensor from among the plurality of sensors, wherein to determine the plurality of rainfall levels, the at least one processor is further configured to at least:
 for at least one sensor from among the plurality of sensors:
 capturing a first sensor data when it is raining on the autonomous vehicle; 
 capturing a second sensor data when it is not raining on the autonomous vehicle; 
 comparing the first sensor data with the second sensor data; 
 determining a difference between the first sensor data and the second sensor data; 
 determining that the difference between the first sensor data and the second sensor data is due to rainfall; and 
 determining a rainfall level associated with the at least one sensor, wherein the rainfall level corresponds to the difference between the first sensor data and the second sensor data; 
 
 
 determine an aggregated rainfall level in a particular time period by combining the plurality of rainfall levels determined during the particular time period. 
   
     
     
         2 . The system of  claim 1 , wherein the at least one processor is further configured to at least update driving instructions associated with the autonomous vehicle based at least in part upon the determined aggregated rainfall level,
 wherein the updated driving instructions comprises one or more of increasing a following distance, increasing a planned stopping distance, turning on headlights, and turning on windshield wipers.   
     
     
         3 . The system of  claim 2 , wherein increasing the following distance is proportional to the determined aggregated rainfall level, and
 wherein increasing the planned stopping distance is proportional to the determined aggregated rainfall level.   
     
     
         4 . The system of  claim 1 , wherein the at least one processor is further configured to at least schedule a sensor cleaning operation based at least in part upon the determined aggregated rainfall level such that a housing of at least one sensor is scheduled to be cleaned more frequently for a higher determined aggregated rainfall level. 
     
     
         5 . The system of  claim 1 , wherein the plurality of rainfall levels is determined from one or more of at least one rain sensor, at least one light detection and ranging (LiDAR) sensor, at least one camera, at least one infrared sensor, and a weather report. 
     
     
         6 . The system of  claim 1 , wherein to combine the plurality of rainfall levels, the at least one processor is further configured to at least determine a mean of a plurality of nominal rainfall levels, and
 wherein each nominal rainfall level from among the plurality of nominal rainfall levels is mapped with a corresponding rainfall level from among the plurality of rainfall levels.   
     
     
         7 . The system of  claim 1 , wherein the at least one processor is further configured to at least:
 for the at least one sensor:
 select a particular object detection algorithm associated with the at least one sensor that is pre-mapped with the determined aggregated rainfall level, wherein the particular object detection algorithm is configured to filter at least a portion of interference caused by the determined aggregated rainfall level in sensor data captured by the at least one sensor; and 
 cause the particular object detection algorithm to be implemented for the at least one sensor. 
   
     
     
         8 . A method comprising:
 obtaining a plurality of sensor data from a plurality of sensors, wherein the plurality of sensors is associated with an autonomous vehicle, wherein each sensor from among the plurality of sensors is configured to capture sensor data, wherein the autonomous vehicle configured to travel along a road;   determining a plurality of rainfall levels based at least in part upon the plurality of sensor data, wherein each rainfall level from among the plurality of rainfall levels is captured by a different sensor from among the plurality of sensors, wherein determining the plurality of rainfall levels based at least in part upon the plurality of sensor data comprises:
 for at least one sensor from among the plurality of sensors:
 capturing a first sensor data when it is raining on the autonomous vehicle; 
 capturing a second sensor data when it is not raining on the autonomous vehicle; 
 comparing the first sensor data with the second sensor data; 
 determining a difference between the first sensor data and the second sensor data; 
 determining that the difference between the first sensor data and the second sensor data is due to rainfall; and 
 determining a rainfall level associated with the at least one sensor, wherein the rainfall level corresponds to the difference between the first sensor data and the second sensor data; 
 
   determining an aggregated rainfall level in a particular time period by combining the plurality of rainfall levels determined during the particular time period.   
     
     
         9 . The method of  claim 8 , wherein:
 the plurality of sensor data comprises at least one rain sensor;   the at least one rain sensor is configured to detect liquid levels on a housing of the at least one rain sensor; and   the method further comprises:
 for each rain sensor from among the at least one rain sensor:
 receiving a first signal from the rain sensor; 
 determining, from the first signal, a liquid level on the housing of the rain sensor while it is raining on the housing of the rain sensor; 
 comparing the liquid level with a reference liquid level on the housing of the rain sensor, wherein the reference liquid level is detected when there is no rainfall on the housing of the rain sensor; 
 determining a difference between the reference liquid level and the liquid level; and 
 determining a first rainfall level based at least in part upon the difference between the reference liquid level and the liquid level such that the first rainfall level is proportional to the difference between the reference liquid level and the liquid level. 
 
   
     
     
         10 . The method of  claim 9 , wherein determining the plurality of rainfall levels based at least in part upon the plurality of sensor data further comprises:
 for each rain sensor from among the at least one rain sensor:
 accessing a first table of calibration curve associated with the rain sensor in which each rainfall level detected by the rain sensor is mapped to a corresponding nominal rainfall level, wherein each rainfall level is mapped to the corresponding nominal rainfall level based at least in part upon a traveling speed of the autonomous vehicle; 
 identifying a first nominal rainfall level that is mapped to the first rainfall level in the first table of calibration curve; and 
 determining an average of a plurality of first nominal rainfall levels determined for the at least one rain sensor. 
   
     
     
         11 . The method of  claim 8 , wherein:
 the plurality of sensor data comprises at least one light detection and ranging (LiDAR) sensor;   the at least one LiDAR sensor is configured to propagate incident laser beams and receive reflected laser beams bounced back from objects; and   the method further comprises:
 for each LiDAR sensor from among the at least one LiDAR sensor:
 receiving a second signal from the LiDAR sensor; 
 determining, from the second signal, a laser beam power loss, wherein the laser beam power loss corresponds to a first difference between a first incident laser beam propagated by the LiDAR sensor and a first reflected laser beam received by the LiDAR sensor, wherein the laser beam power loss is determined when it is raining on the autonomous vehicle; 
 comparing the laser beam power loss with a reference laser beam power loss, wherein the reference laser beam power loss is determined when it is not raining on the autonomous vehicle; 
 determining an increase in the laser beam power loss compared to the reference laser beam power loss; and 
 determining a second rainfall level based at least in part upon the increase in the laser beam power loss such that the second rainfall level is proportional to the increase in the laser beam power loss; 
 
 wherein determining the plurality of rainfall levels based at least in part upon the plurality of sensor data further comprises:
 for each LiDAR sensor from among the at least one LiDAR sensor:
 accessing a second table of calibration curve associated with the LiDAR sensor in which each rainfall level detected by the LiDAR sensor is mapped to a corresponding nominal rainfall level, wherein each rainfall level is mapped to the corresponding nominal rainfall level based at least in part upon a traveling speed of the autonomous vehicle; 
 identifying a second nominal rainfall level that is mapped with the second rainfall level; and 
 determining an average of a plurality of second nominal rainfall levels determined for the at least one LiDAR sensor. 
 
 
   
     
     
         12 . The method of  claim 8 , wherein:
 the plurality of sensors comprises at least one camera; and   the at least one camera is configured to capture at least one image of an environment around the autonomous vehicle.   
     
     
         13 . The method of  claim 12 , wherein determining the plurality of rainfall levels based at least in part upon the plurality of sensor data further comprises:
 for each camera from among the at least one camera:
 receiving the at least one image from the camera; 
 feeding the at least one image to an image processing neural network that is trained to identify a third rainfall level from the at least one image; and 
 determining the third rainfall level from the at least one image. 
   
     
     
         14 . The method of  claim 8 , wherein:
 the plurality of sensors comprises at least one infrared camera; and   the at least one infrared camera is configured to capture at least one infrared image of an environment around the autonomous vehicle, wherein a color of an object in the at least one infrared image represents a particular temperature of the object.   
     
     
         15 . The method of  claim 14 , wherein determining the plurality of rainfall levels based at least in part upon the plurality of sensor data further comprises:
 for each infrared camera from among the at least one infrared camera:
 receiving a first infrared image from the infrared camera when it is not raining on the autonomous vehicle; 
 determining a reference temperature associated with a portion of the autonomous vehicle that is shown in the first infrared image, wherein the reference temperature is represented by a first color of the portion of the autonomous vehicle in the first infrared image; 
 receiving a second infrared image from the infrared camera when it is raining on the autonomous vehicle; 
 determining a temperature associated with the portion of the autonomous vehicle that is shown in the second infrared image wherein the temperature associated with the portion of the autonomous vehicle is represented by a second color of the portion of the autonomous vehicle in the second infrared image; 
 comparing the temperature and the reference temperature of the portion of the autonomous vehicle shown in the first infrared image and the second infrared image, respectively; and 
 determining a fourth rainfall level based at least in part upon a difference between the reference temperature and the temperature of the portion of the autonomous vehicle; and 
 determining an average of a plurality of fourth rainfall levels determined for the at least one infrared camera. 
   
     
     
         16 . A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
 obtain a plurality of sensor data from a plurality of sensors, wherein the plurality of sensors is associated with an autonomous vehicle, wherein each sensor from among the plurality of sensors is configured to capture sensor data, wherein the autonomous vehicle configured to travel along a road;   determine a plurality of rainfall levels based at least in part upon the plurality of sensor data, wherein each rainfall level from among the plurality of rainfall levels is captured by a different sensor from among the plurality of sensors, wherein to determine the plurality of rainfall levels, the at least one processor is further configured to at least:
 for at least one sensor from among the plurality of sensors:
 capture a first sensor data when it is raining on the autonomous vehicle; 
 capture a second sensor data when it is not raining on the autonomous vehicle; 
 compare the first sensor data with the second sensor data; 
 determine a difference between the first sensor data and the second sensor data; 
 determine that the difference between the first sensor data and the second sensor data is due to rainfall; and 
 determine a rainfall level associated with the at least one sensor, wherein the rainfall level corresponds to the difference between the first sensor data and the second sensor data; 
 
   determine an aggregated rainfall level in a particular time period by combining the plurality of rainfall levels determined during the particular time period.   
     
     
         17 . The non-transitory computer-readable medium of  claim 16 , wherein the instructions when executed by the at least one processor, further cause the at least one processor to at least:
 receive feedback that indicates the plurality of sensor data indicates a sixth rainfall level;   associate the plurality of sensor data to the sixth rainfall level;   feed the plurality of sensor data associated with the sixth rainfall level to a rainfall level detection model, wherein the rainfall level detection model comprises a neural network configured to detect rainfalls from sensor data; and   train the rainfall level detection model to learn to associate the plurality of sensor data with the sixth rainfall level.   
     
     
         18 . The non-transitory computer-readable medium of  claim 16 , wherein the instructions when executed by the at least one processor, further cause the at least one processor to at least determine a confidence score to the aggregated rainfall level, wherein the confidence score is determined based at least in part upon a standard deviation of a mean value of the plurality of rainfall levels such that the confidence score is inversely proportional to the standard deviation. 
     
     
         19 . The non-transitory computer-readable medium of  claim 16 , wherein the instructions when executed by the at least one processor, further cause the at least one processor to at least communicate a message to one or more autonomous vehicles traveling on the road behind the autonomous vehicle indicating the aggregated rainfall level at a location of the autonomous vehicle. 
     
     
         20 . The non-transitory computer-readable medium of  claim 16 , wherein:
 one of the plurality of rainfall levels is determined from a weather report;   the weather report is associated with a location of the autonomous vehicle; and   the instructions when executed by the at least one processor, further cause the at least one processor to:
 identify a rainfall level indicated in the weather report; 
 include the rainfall level in the plurality of rainfall levels; 
   each of the plurality of rainfall levels is assigned a corresponding weight value, wherein the corresponding weight value assigned to a rainfall level represents an accuracy level of the rainfall level, and wherein the instructions when executed by the at least one processor, further cause the at least one processor to:
 determine a location of the autonomous vehicle; 
 determine an area associated with the weather report; 
 determine that the autonomous vehicle is located within a threshold distance from the area associated with the weather report; and 
 assign a higher weight value to a particular rainfall determined from the weather report compared to other weight values assigned to other rainfall levels determined from the plurality of sensors.