System and method for labelling aerial images

Abstract

A system and method for labelling aerial images. A neural network generates predicted map data. The parameters of the neural network are trained by optimizing an objective function which compensates for noise in the map images. The function compensates both omission noise and registration noise.

1 - 20 . (canceled) 
     
     
         21 . A method performed by one or more computers, the method comprising:
 training a neural network on training data, wherein the training data includes a plurality of labeled aerial images, wherein the neural network is configured to generate a respective score for each of a plurality of pixels of an input aerial image, wherein the respective scores for each of the plurality of pixels represent a probability that the pixel belongs to an object class, and   wherein training the neural network on the training data comprises:
 optimizing an objective function having a derivative that includes a posterior probability term that represents a likelihood that labels for pixels from the labeled aerial images differ from true labels for the pixels, wherein, for each labeled aerial image, the posterior probability term accounts for (i) a first predetermined likelihood representative of a label for the pixels indicating that the pixels belong to the object class given that a true label for the pixels indicates that the pixels do not belong to the object class, and (ii) a second predetermined likelihood representative of a label for the pixels indicating that the pixels do not belong to the object class given that the true label for the pixels indicates that the pixels belong to the object class, a value of the first predetermined likelihood relative to a value of the second predetermined likelihood being determined according to an asymmetric noise distribution model, 
   wherein the posterior probability term compensates for omission errors in the training data, an omission error occurring when a pixel in a labeled aerial image has been incorrectly labeled as not belonging to the object class.   
     
     
         22 . The method of  claim 21 , wherein the objective function is a negative log likelihood function. 
     
     
         23 . The method of  claim 21 , wherein the derivative takes the form of, for a pixel i in a particular labeled aerial image, p(m 1 =1|{tilde over (m)} i ,s)−{circumflex over (m)} i , wherein p(m i =1|{tilde over (m)} i ,s) is a posterior probability that a true label m i  for the pixel i is 1 given a label for the pixel i in the particular labeled aerial image {tilde over (m)} i  and a portion s of the aerial image containing the pixel i, and wherein {circumflex over (m)} i  is a probability predicted for the pixel i by the neural network. 
     
     
         24 . The method of  claim 21 , wherein:
 the object class is roads; and   the respective scores for each of the plurality of pixels represent a probability that the pixel is a part of an image of a road.   
     
     
         25 . The method of  claim 22 , wherein:
 the value of the second predetermined likelihood greater than the first predetermined likelihood and less than 0.1; and   the value of the first predetermined likelihood is greater than zero.   
     
     
         26 . The method of  claim 25 , wherein:
 the asymmetric noise distribution model is an asymmetric Bernoulli noise distribution model; and   optimizing the objective function comprises minimizing the objective function using the asymmetric Bernoulli noise distribution model.   
     
     
         27 . The method of  claim 22 , further comprising:
 selecting one or more parameters of the neural network; and   generating map data during and after the training of the neural network using the selected one or more parameters.   
     
     
         28 . A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
 training a neural network on training data, wherein the training data includes a plurality of labeled aerial images, wherein the neural network is configured to generate a respective score for each of a plurality of pixels of an input aerial image, wherein the respective scores for each of the plurality of pixels represent a probability that the pixel belongs to an object class, and   wherein training the neural network on the training data comprises:
 optimizing an objective function having a derivative that includes a posterior probability term that represents a likelihood that labels for pixels from the labeled aerial images differ from true labels for the pixels, wherein, for each labeled aerial image, the posterior probability term accounts for (i) a first predetermined likelihood representative of a label for the pixels indicating that the pixels belong to the object class given that a true label for the pixels indicate that the pixels do not belong to the object class, and (ii) a second predetermined likelihood representative of a label for the pixels indicating that the pixels do not belong to the object class given that the true label for the pixels indicates that the pixels belong to the object class, a value of the first predetermined likelihood relative to a value of the second predetermined likelihood being determined according to an asymmetric noise distribution model, 
   wherein the posterior probability term compensates for omission errors in the training data, an omission error occurring when a pixel in a labeled aerial image has been incorrectly labeled as not belonging to the object class.   
     
     
         29 . The system of  claim 28 , wherein the objective function is a negative log likelihood function. 
     
     
         30 . The system of  claim 28 , wherein the derivative takes the form of, for a pixel i in a particular labeled aerial image, p(m i =1|{tilde over (m)} i ,s)−{circumflex over (m)} i , wherein p(m i =1|{tilde over (m)} i ,s) is a posterior probability that a true label m i  for the pixel i is 1 given a label for the pixel i in the particular labeled aerial image {tilde over (m)} i  and a portion s of the aerial image containing the pixel i, and wherein {circumflex over (m)} i  is a probability predicted for the pixel i by the neural network. 
     
     
         31 . The system of  claim 28 , wherein:
 the object class is roads; and   the respective scores for each of the plurality of pixels represent a probability that the pixel is a part of an image of a road.   
     
     
         32 . The system of  claim 28 , wherein:
 the value of the second predetermined likelihood greater than the first predetermined likelihood and less than 0.1; and   the value of the first predetermined likelihood is greater than zero.   
     
     
         33 . The system of  claim 28 , wherein:
 the asymmetric noise distribution model is an asymmetric Bernoulli noise distribution model; and   optimizing the objective function comprises minimizing the objective function using the asymmetric Bernoulli noise distribution model.   
     
     
         34 . The system of  claim 28 , wherein the operations further comprise:
 selecting one or more parameters of the neural network; and   generating map data during and after the training of the neural network using the selected one or more parameters.   
     
     
         35 . One or more non-transitory computer-readable storage media comprising instructions, which, when executed by one or more computers, cause the one or more computers to perform actions comprising:
 training a neural network on training data, wherein the training data includes a plurality of labeled aerial images, wherein the neural network is configured to generate a respective score for each of a plurality of pixels of an input aerial image, wherein the respective scores for each of the plurality of pixels represent a probability that the pixel belongs to an object class, and   wherein training the neural network on the training data comprises:
 optimizing an objective function having a derivative that includes a posterior probability term that represents a likelihood that labels for pixels from the labeled aerial images differ from true labels for the pixels, wherein, for each labeled aerial image, the posterior probability term accounts for (i) a first predetermined likelihood representative of a label for the pixels indicating that the pixels belong to the object class given that a true label for the pixels indicate that the pixels do not belong to the object class, and (ii) a second predetermined likelihood representative of a label for the pixels indicating that the pixels do not belong to the object class given that the true label for the pixels indicates that the pixels belong to the object class, a value of the first predetermined likelihood relative to a value of the second predetermined likelihood being determined according to an asymmetric noise distribution model, 
   wherein the posterior probability term compensates for omission errors in the training data, an omission error occurring when a pixel in a labeled aerial image has been incorrectly labeled as not belonging to the object class.   
     
     
         36 . The one or more non-transitory computer-readable storage media of  claim 35 , wherein the objective function is a negative log likelihood function. 
     
     
         37 . The one or more non-transitory computer-readable storage media of  claim 29 , wherein the derivative takes the form of, for a pixel i in a particular labeled aerial image, p(m i =1|{tilde over (m)} i ,s)−{circumflex over (m)} i , wherein p(m i =1|{tilde over (m)} i ,s) is a posterior probability that a true label m i  for the pixel i is 1 given a label for the pixel i in the particular labeled aerial image {tilde over (m)} i  and a portion s of the aerial image containing the pixel i, and wherein {circumflex over (m)} i  is a probability predicted for the pixel i by the neural network. 
     
     
         38 . The one or more non-transitory computer-readable storage media of  claim 35 , wherein:
 the object class is roads; and   the respective scores for each of the plurality of pixels represent a probability that the pixel is a part of an image of a road.   
     
     
         39 . The one or more non-transitory computer-readable storage media of  claim 35 , wherein the operations further comprise:
 selecting one or more parameters of the neural network; and   generating map data during and after the training of the neural network using the selected one or more parameters.   
     
     
         40 . The one or more non-transitory computer-readable storage media of  claim 35 , wherein:
 the asymmetric noise distribution model is an asymmetric Bernoulli noise distribution model; and   optimizing the objective function comprises minimizing the objective function using the asymmetric Bernoulli noise distribution model.