Path detection for autonomous machines using deep neural networks

Abstract

In various examples, a deep learning solution for path detection is implemented to generate a more abstract definition of a drivable path—without reliance on explicit lane-markings—by using a detection-based approach. Using approaches of the present disclosure, the identification of drivable paths may be possible in environments where conventional approaches are unreliable, or fail—such as where lane markings do not exist or are occluded. The deep learning solution may generate outputs that represent geometries for one or more drivable paths in an environment and confidence values corresponding to path types or classes that the geometries correspond. These outputs may be directly useable by an autonomous vehicle—such as an autonomous driving software stack—with minimal post-processing.

What is claimed is: 
     
         1 . A method comprising:
 determining, using one or more neural networks and based at least on sensor data corresponding to one or more sensory fields of one or more sensors of a machine, one or more locations of one or more points associated with a predicted path;   determining based at least on the one or more locations, at least a portion of a geometry of the predicted path; and   performing, based at least on at least the portion of the geometry, one or more operations associated with the machine.   
     
     
         2 . The method of  claim 1 , wherein the determining the one or more locations of the one or more points comprises:
 determining, using the one or more neural networks and based at least on the sensor data, one or more distances between one or more anchor points and the one or more points; and   determining, based at least on the one or more distances, the one or more locations of the one or more points.   
     
     
         3 . The method of  claim 1 , further comprising determining a path class associated with the predicted path, the path class including at least one of:
 a straight path;   a right path; or   a left path.   
     
     
         4 . The method of  claim 1 , wherein:
 the one or more points comprise one or more vertices of one or more path edges associated with the predicted path; and   the determining at least the portion of the geometry of the predicted path comprises:
 determining the one or more path edges based at least on the one or more locations of the one or more vertices; and 
 determining the predicted path based at least on the one or more path edges. 
   
     
     
         5 . The method of  claim 1 , wherein:
 the one or more points comprise one or more vertices of the predicted path; and   the determining at least the portion of the geometry of the predicted path comprises determining at least the portion of the geometry of the predicted path as including a line through the one or more vertices.   
     
     
         6 . The method of  claim 1 , wherein:
 the one or more locations of the one or more points comprise one or more two-dimensional locations of the one or more vertices within one or more sensor data representations corresponding to the sensor data; and   the determining at least the portion of the geometry of the predicted path comprises:
 determining, based at least on the one or more two-dimensional locations of the one or more vertices, one or more three-dimensional locations within an environment; and 
 determining, based at least on the one or more three-dimensional locations, at least the portion of the geometry of the predicted path. 
   
     
     
         7 . The method of  claim 1 , wherein:
 the sensor data includes image data representative of an image;   the one or more locations of the one or more points comprise one or more pixel locations of the one or more points within the image; and   the determining at least the portion of the geometry of the predicted path comprises:
 determining, based at least on the one or more pixel locations of the one or more points, one or more three-dimensional locations within an environment; and 
 determining, based at least on the one or more three-dimensional locations, at least the portion of the geometry of the predicted path. 
   
     
     
         8 . The method of  claim 1 , further comprising:
 generating, based at least on at least one of the predicted path, one or more locations of one or more objects, or one or more wait conditions, a model associated with an environment for which the machine is navigating,   wherein the performing the one or more operations associated with the machine is based at least on the model.   
     
     
         9 . A system comprising:
 one or more processing units to:
 determine, using one or more neural networks and based at least on sensor data representative of one or more sensor data representations, one or more two-dimensional points corresponding to the one or more sensor data representations that are associated with a predicted path of a machine; 
 determine, based at least on the one or more two-dimensional points, one or more three-dimensional points within an environment that are associated with the predicted path of the machine; and 
 perform, based at least on the one or more three-dimensional points within the environment, one or more operations associated with the machine. 
   
     
     
         10 . The system of  claim 9 , wherein:
 the sensor data comprises image data and at least one sensor data representation of the one or more sensor data representations comprises an image;   the one or more two-dimensional points that are associated with the predicted path of the machine comprises one or more pixel locations within the image that are associated with the predicted path; and   the determination of the one or more three-dimensional points within the environment that are associated with the predicted path of the machine is based at least on the one or more pixel locations.   
     
     
         11 . The system of  claim 9 , wherein the determination of the one or more two-dimensional points comprises:
 determining, using the one or more neural networks and based at least on the sensor data, one or more distances between one or more anchor points and one or more vertices associated with the predicted path; and   determining, based at least on the one or more distances, the one or more two-dimensional points.   
     
     
         12 . The system of  claim 9 , wherein the one or more two-dimensional points comprise at least one of:
 one or more first two-dimensional points that are associated with one or more path edges associated with the predicted path; or   one or more second two-dimensional points that are associated with a center of the predicted path.   
     
     
         13 . The system of  claim 9 , wherein the one or more three-dimensional points within the environment comprise at least one of:
 one or more first three-dimensional points that are associated with one or more path edges within the environment; or   one or more second three-dimensional points that are associated with a center of the predicted path through the environment.   
     
     
         14 . The system of  claim 9 , wherein the one or more processing units are further to determine a path class associated with the predicted path, the path class including at least one of:
 a straight path;   a right path; or   a left path.   
     
     
         15 . The system of  claim 9 , wherein the one or more processing units are further to:
 determine, based at least on the one or more three-dimensional points within the environment, at least a portion of a geometry of the predicted path,   wherein the one or more operations associated with the machine are performed based at least on at least the portion of the geometry of the predicted path.   
     
     
         16 . The system of  claim 9 , wherein the one or more processing units are further to:
 determine, based at least on at least one of the one or more three-dimensional points within the environment, one or more locations of one or more objects, or one or more wait conditions, a model associated with the environment,   wherein the one or more operations associated with the machine are performed based at least on the model.   
     
     
         17 . The system of  claim 9 , wherein the system is comprised in at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system for generating synthetic data;   a system incorporating one or more virtual machines (VMs);   a system implemented at least partially in a data center; or   a system implemented at least partially using cloud computing resources.   
     
     
         18 . A processor comprising:
 one or more processing units to perform one or more operations associated with a machine based at least on a geometry of a predicted path, wherein the geometry of the predicted path is determined based at least on one or more locations of one or more vertices associated with the predicted path that are determined based at least on one or more neural networks processing sensor data generated using the machine.   
     
     
         19 . The processor of  claim 18 , wherein:
 the one or more locations of the one or more vertices comprise one or more two-dimensional locations of the one or more vertices within one or more sensor data representations corresponding to the sensor data; and   the determination of the geometry of the predicted path comprises:
 determining, based at least on the one or more two-dimensional locations of the one or more vertices, one or more three-dimensional locations within an environment; and 
 determining the geometry of the predicted path based at least on the one or more three-dimensional locations. 
   
     
     
         20 . The processor of  claim 18 , wherein the processor is comprised in at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system for generating synthetic data;   a system incorporating one or more virtual machines (VMs);   a system implemented at least partially in a data center; or   a system implemented at least partially using cloud computing resources.