Vapor detection in lidar point cloud

Abstract

Various technologies described herein pertain to detecting data in a lidar point cloud representative of vapor and controlling an autonomous vehicle based on such detection. The lidar point cloud is outputted by a lidar sensor system of the autonomous vehicle. The techniques set forth herein utilize dual return lidar data outputted by the lidar sensor system. The dual return lidar data includes two lidar returns received responsive to a light beam emitted (e.g., at a particular azimuthal angle) by the lidar sensor system into a driving environment of the autonomous vehicle. The data in the lidar point cloud detected as being caused by vapor can be removed from downstream processing; accordingly, the autonomous vehicle can be controlled such that the autonomous vehicle need not stop for or maneuver around vapor in the driving environment.

What is claimed is: 
     
       1. An autonomous vehicle, comprising:
 a vehicle propulsion system; 
 a braking system; 
 a steering system; 
 a lidar sensor system; and 
 a computing system that is in communication with the vehicle propulsion system, the braking system, the steering system, and the lidar sensor system, wherein the computing system comprises:
 a processor; and 
 memory that stores computer-executable instructions that, when executed by the processor, cause the processor to perform acts comprising:
 receiving dual return lidar data outputted by the lidar sensor system, the dual return lidar data comprising two lidar returns per channel detected responsive to a light beam emitted by the lidar sensor system into a driving environment of the autonomous vehicle; 
 assigning a label to an object captured in the dual return lidar data comprising the two lidar returns per channel, wherein the label identifies a type of the object, wherein the label is assigned based upon a probability determined for the type of the object from amongst a predefined set of types, wherein the predefined set of types comprises vapor, wherein the probability is determined for the type of the object based on the dual return lidar data comprising the two lidar returns per channel, wherein the probability is determined for the type of the object being at a first range value corresponding to a strongest lidar return in the dual return lidar data, and wherein the probability is determined based on the first range value, a first intensity value of the strongest lidar return, a second range value corresponding to a farthest lidar return in the dual return lidar data, and a second intensity value of the farthest lidar return; and 
 controlling at least one of the vehicle propulsion system, the braking system, or the steering system based on the label assigned to the object, wherein the vehicle propulsion system, the braking system, and the steering system are controlled such that the object is ignored when the object is assigned the label of vapor. 
 
 
 
     
     
       2. The autonomous vehicle of  claim 1 , wherein the two lidar returns per channel detected responsive to the light beam comprise:
 the first range value and the first intensity value of the strongest lidar return received responsive to the light beam; and 
 the second range value and the second intensity value of the farthest lidar return received responsive to the light beam. 
 
     
     
       3. The autonomous vehicle of  claim 1 , wherein the vapor comprises at least one of steam, fog, smoke, exhaust, or mist. 
     
     
       4. The autonomous vehicle of  claim 1 , wherein assigning the label to the object captured in the dual return lidar data further comprises:
 determining the probability for the type of the object from amongst the predefined set of types. 
 
     
     
       5. The autonomous vehicle of  claim 4 , wherein determining the probability for the type of the object from amongst the predefined set of types further comprises:
 generating a feature vector based upon the dual return lidar data, the feature vector comprises an identifier of the light beam, an identifier of an azimuthal angle of the light beam, the first range value of the strongest lidar return received responsive to the light beam at the azimuthal angle, the second range value of the farthest lidar return received responsive to the light beam at the azimuthal angle, the first intensity value of the strongest lidar return, and the second intensity value of the farthest lidar return; and 
 inputting the feature vector to a neural network, wherein the probability for the type of the object is outputted by the neural network responsive to the feature vector being inputted to the neural network. 
 
     
     
       6. The autonomous vehicle of  claim 5 , wherein the feature vector further comprises vapor map data indicating likely presence of vapor at a particular geographic location. 
     
     
       7. The autonomous vehicle of  claim 5 , wherein determining the probability for the type of the object from amongst the predefined set of types further comprises:
 modifying the probability for the type of the object outputted by the neural network based on vapor map data indicating likely presence of vapor at a particular geographic location. 
 
     
     
       8. The autonomous vehicle of  claim 5 , wherein determining the probability for the type of the object from amongst the predefined set of types further comprises:
 modifying the probability for the type of the object outputted by the neural network based on data indicating likely presence of vapor emission from a tracked object. 
 
     
     
       9. The autonomous vehicle of  claim 1 , further comprising:
 a differing sensor system, the differing sensor system being at least one of a camera sensor system or a radar sensor system; 
 wherein the acts further comprise:
 projecting the dual return lidar data outputted by the lidar sensor system onto data outputted by the differing sensor system; 
 determining whether one or both of the lidar returns per channel as projected correspond to the data outputted by the differing sensor system; and 
 modifying the probability for the type of the object based on whether one or both of the lidar returns per channel as projected correspond to the data outputted by the differing sensor system, wherein the label is assigned to the object based on the probability as modified. 
 
 
     
     
       10. The autonomous vehicle of  claim 1 , further comprising:
 a camera sensor system; 
 wherein the acts further comprise:
 projecting the dual return lidar data outputted by the lidar sensor system onto a camera image outputted by the camera sensor system; 
 analyzing the camera image to detect whether at least a portion of the camera image is hazy; and 
 modifying the probability for the type of the object based on whether the portion of the camera image is detected as being hazy. 
 
 
     
     
       11. A method for controlling an autonomous vehicle, comprising:
 receiving dual return lidar data outputted by a lidar sensor system of the autonomous vehicle, the dual return lidar data comprising two lidar returns per channel detected responsive to a light beam emitted by the lidar sensor system into a driving environment of the autonomous vehicle; 
 assigning a label to an object captured in the dual return lidar data comprising the two lidar returns per channel, wherein the label identifies a type of the object, wherein the label is assigned based upon a probability determined for the type of the object from amongst a predefined set of types, wherein the predefined set of types comprises vapor, wherein the probability is determined for the type of the object based on the dual return lidar data comprising the two lidar returns per channel, wherein the probability is determined for the type of the object being at a first range value corresponding to a strongest lidar return in the dual return lidar data, and wherein the probability is determined based on the first range value, a first intensity value of the strongest lidar return, a second range value corresponding to a farthest lidar return in the dual return lidar data, and a second intensity value of the farthest lidar return; and 
 controlling at least one of a vehicle propulsion system, a braking system, or a steering system of the autonomous vehicle based on the label assigned to the object, wherein the vehicle propulsion system, the braking system, and the steering system are controlled such that the object is ignored when the object is assigned the label of vapor. 
 
     
     
       12. The method of  claim 11 , wherein assigning the label to the object captured in the dual return lidar data further comprises:
 determining the probability for the type of the object from amongst the predefined set of types. 
 
     
     
       13. The method of  claim 12 , wherein determining the probability for the type of the object from amongst the predefined set of types further comprises:
 generating a feature vector based upon the dual return lidar data, the feature vector comprises an identifier of the light beam, an identifier of an azimuthal angle of the light beam, the first range value of the strongest lidar return received responsive to the light beam at the azimuthal angle, the second range value of the farthest lidar return received responsive to the light beam at the azimuthal angle, the first intensity value of the strongest lidar return, and the second intensity value of the farthest lidar return; and 
 inputting the feature vector to a neural network, wherein the probability for the type of the object is outputted by the neural network responsive to the feature vector being inputted to the neural network. 
 
     
     
       14. The method of  claim 13 , wherein the feature vector further comprises vapor map data indicating likely presence of vapor at a particular geographic location. 
     
     
       15. The method of  claim 13 , wherein determining the probability for the type of the object from amongst the predefined set of types further comprises:
 modifying the probability for the type of the object outputted by the neural network based on vapor map data indicating likely presence of vapor at a particular geographic location. 
 
     
     
       16. The method of  claim 13 , wherein determining the probability for the type of the object from amongst the predefined set of types further comprises:
 modifying the probability for the type of the object outputted by the neural network based on data indicating likely presence of vapor emission from a tracked object. 
 
     
     
       17. The method of  claim 11 , further comprising:
 projecting the dual return lidar data outputted by the lidar sensor system onto data outputted by a differing sensor system of the autonomous vehicle; 
 determining whether one or both of the lidar returns per channel as projected correspond to the data outputted by the differing sensor system; and 
 modifying the probability for the type of the object based on whether one or both of the lidar returns per channel as projected correspond to the data outputted by the differing sensor system, wherein the label is assigned to the object based on the probability as modified. 
 
     
     
       18. The method of  claim 11 , further comprising:
 projecting the dual return lidar data outputted by the lidar sensor system onto a camera image outputted by a camera sensor system of the autonomous vehicle; 
 analyzing the camera image to detect whether at least a portion of the camera image is hazy; and 
 modifying the probability for the type of the object based on whether the portion of the camera image is detected as being hazy. 
 
     
     
       19. A method of for controlling an autonomous vehicle, comprising:
 receiving dual return lidar data outputted by a lidar sensor system of the autonomous vehicle, the dual return lidar data comprising a first intensity value of a strongest lidar return and a second intensity value of a farthest lidar return per channel detected responsive to a light beam emitted by the lidar sensor system at a particular azimuthal angle into a driving environment of the autonomous vehicle, the dual return lidar data further comprising a first range value corresponding to the strongest lidar return and a second range value corresponding to the farthest lidar return; 
 determining probabilities for predefined object types based on the dual return lidar data, wherein the predefined object types comprise vapor, wherein a probability is determined for a type of an object being at the first range value, and wherein the probability is determined based on the first range value, the first intensity value of the strongest lidar return, the second range value, and the second intensity value of the farthest lidar return; 
 projecting the dual return lidar data outputted by the lidar sensor system onto data outputted by a differing sensor system of the autonomous vehicle; 
 determining whether at least one of the strongest lidar return or the farthest lidar return per channel as projected corresponds to the data outputted by the differing sensor system; 
 modifying the probabilities for the predefined object types based on the determination of whether at least one of the strongest lidar return or the farthest lidar return per channel as projected corresponds to the data outputted by the differing sensor system; and 
 controlling at least one of a vehicle propulsion system, a braking system, or a steering system of the autonomous vehicle based on the probabilities for the predefined object types as modified. 
 
     
     
       20. The method of  claim 19 , wherein determining probabilities for the predefined object types based on the dual return lidar data further comprises:
 generating a feature vector based upon the dual return lidar data, the feature vector comprises an identifier of the light beam, an identifier of an azimuthal angle of the light beam, the first range value of the strongest lidar return, the second range value of the farthest lidar return, the first intensity value of the strongest lidar return, and the second intensity value of the farthest lidar return; and 
 inputting the feature vector to a neural network, wherein the probabilities for the predefined object types are outputted by the neural network responsive to the feature vector being inputted to the neural network.