Autonomous driving with surfel maps

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate a prediction for a state of an environment. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises an uncertainty measure; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more particular surfels corresponding to respective locations of the obtained sensor data; and combining the surfel data and the sensor data to generate a respective object prediction for each of the one or more locations of the obtained sensor data.

What is claimed is: 
     
         1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method comprising:
 receiving, by an on-board prediction subsystem of an autonomous vehicle, processed sensor data for one or more locations in a driving environment of the autonomous vehicle, wherein the processed sensor data indicates for a location of the one or more locations a first likelihood value representing how likely it is that an object is present at the location in the driving environment;   obtaining, by the prediction subsystem of the autonomous vehicle, a surfel map for the driving environment, wherein the surfel map comprises a plurality of surfels for a plurality of respective locations in the driving environment, and wherein each surfel of the plurality of surfels is associated with a respective second likelihood value representing how likely it is that a surface was previously detected in sensor data obtained by one or more other vehicles that previously traveled through the driving environment; and   generating a combined object prediction for the location in the driving environment including combining the first likelihood value of the processed sensor data with the second likelihood value of a surfel in the surfel map corresponding to the location.   
     
     
         22 . The method of  claim 21 , further comprising:
 providing the combined object prediction for the location to a planning subsystem of the autonomous vehicle; and   generating, by the planning system, an updated plan based on the combined object prediction for the location; and   causing the autonomous vehicle to follow a trajectory corresponding to the updated plan based on the combined object prediction for the location.   
     
     
         23 . The method of  claim 21 , wherein the surfel map comprises a prior probability distribution over semantic labels for the location, and wherein generating the combined object prediction comprises using the processed sensor data to compute a posterior probability distribution over the semantic labels for the location. 
     
     
         24 . The method of  claim 21 , wherein each surfel of the surfel map is associated with one or more semantic labels. 
     
     
         25 . The method of  claim 24 , wherein the semantic labels indicate a respective class of detected objects for each surfel in the surfel map. 
     
     
         26 . The method of  claim 24 , wherein the semantic labels indicate a respective reflectivity value of detected objects for each surfel in the surfel map. 
     
     
         27 . The method of  claim 26 , further comprising determining, based on the reflectivity value for the location, that a transparent object is at the location. 
     
     
         28 . The method of  claim 26 , further comprising determining that an occluded object is at the location. 
     
     
         29 . A system comprising:
 one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:   receiving, by an on-board prediction subsystem of an autonomous vehicle, processed sensor data for one or more locations in a driving environment of the autonomous vehicle, wherein the processed sensor data indicates for a location of the one or more locations a first likelihood value representing how likely it is that an object is present at the location in the driving environment;   obtaining, by the prediction subsystem of the autonomous vehicle, a surfel map for the driving environment, wherein the surfel map comprises a plurality of surfels for a plurality of respective locations in the driving environment, and wherein each surfel of the plurality of surfels is associated with a respective second likelihood value representing how likely it is that a surface was previously detected in sensor data obtained by one or more other vehicles that previously traveled through the driving environment; and   generating a combined object prediction for the location in the driving environment including combining the first likelihood value of the processed sensor data with the second likelihood value of a surfel in the surfel map corresponding to the location.   
     
     
         30 . The system of  claim 29 , wherein the operations further comprise:
 providing the combined object prediction for the location to a planning subsystem of the autonomous vehicle; and   generating, by the planning system, an updated plan based on the combined object prediction for the location; and   causing the autonomous vehicle to follow a trajectory corresponding to the updated plan based on the combined object prediction for the location.   
     
     
         31 . The system of  claim 29 , wherein the surfel map comprises a prior probability distribution over semantic labels for the location, and wherein generating the combined object prediction comprises using the processed sensor data to compute a posterior probability distribution over the semantic labels for the location. 
     
     
         32 . The system of  claim 29 , wherein each surfel of the surfel map is associated with one or more semantic labels. 
     
     
         33 . The system of  claim 32 , wherein the semantic labels indicate a respective class of detected objects for each surfel in the surfel map. 
     
     
         34 . The system of  claim 32 , wherein the semantic labels indicate a respective reflectivity value of detected objects for each surfel in the surfel map. 
     
     
         35 . The system of  claim 34 , wherein the operations further comprise determining, based on the reflectivity value for the location, that a transparent object is at the location. 
     
     
         36 . The system of  claim 34 , wherein the operations further comprise determining that an occluded object is at the location. 
     
     
         37 . One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
 receiving, by an on-board prediction subsystem of an autonomous vehicle, processed sensor data for one or more locations in a driving environment of the autonomous vehicle, wherein the processed sensor data indicates for a location of the one or more locations a first likelihood value representing how likely it is that an object is present at the location in the driving environment;   obtaining, by the prediction subsystem of the autonomous vehicle, a surfel map for the driving environment, wherein the surfel map comprises a plurality of surfels for a plurality of respective locations in the driving environment, and wherein each surfel of the plurality of surfels is associated with a respective second likelihood value representing how likely it is that a surface was previously detected in sensor data obtained by one or more other vehicles that previously traveled through the driving environment; and   generating a combined object prediction for the location in the driving environment including combining the first likelihood value of the processed sensor data with the second likelihood value of a surfel in the surfel map corresponding to the location.   
     
     
         38 . The one or more computer storage media of  claim 37 , wherein the operations further comprise:
 providing the combined object prediction for the location to a planning subsystem of the autonomous vehicle; and   generating, by the planning system, an updated plan based on the combined object prediction for the location; and   causing the autonomous vehicle to follow a trajectory corresponding to the updated plan based on the combined object prediction for the location.   
     
     
         39 . The one or more computer storage media of  claim 37 , wherein the surfel map comprises a prior probability distribution over semantic labels for the location, and wherein generating the combined object prediction comprises using the processed sensor data to compute a posterior probability distribution over the semantic labels for the location. 
     
     
         40 . The one or more computer storage media of  claim 37 , wherein each surfel of the surfel map is associated with one or more semantic labels.