Automatic high beam control for autonomous machine applications

Abstract

In various examples, high beam control for vehicles may be automated using a deep neural network (DNN) that processes sensor data received from vehicle sensors. The DNN may process the sensor data to output pixel-level semantic segmentation masks in order to differentiate actionable objects (e.g., vehicles with front or back lights lit, bicyclists, or pedestrians) from other objects (e.g., parked vehicles). Resulting segmentation masks output by the DNN(s), when combined with one or more post processing steps, may be used to generate masks for automated high beam on/off activation and/or dimming or shading—thereby providing additional illumination of an environment for the driver while controlling downstream effects of high beam glare for active vehicles.

What is claimed is: 
     
         1 . A method comprising:
 determining, using one or more neural networks (NNs) and based at least on image data representative of one or more images of an environment, one or more regions of the one or more images that correspond to one or more objects; and   determining a beam configuration for a lighting system based at least on the one or more regions that correspond to the one or more objects.   
     
     
         2 . The method of  claim 1 , wherein the determining the one or more regions includes:
 computing, using the one or more NNs, one or more segmentation masks classifying pixels of the one or more images as corresponding to the one or more objects; and   determining the one or more regions based at least on the classifying of the pixels.   
     
     
         3 . The method of  claim 1 , wherein the determining the beam configuration includes one or more of:
 adjusting the beam configuration to reduce illumination for one or more first portions of the one or more regions;   adjusting the beam configuration to increase illumination for one or more second portions of the one or more regions; or   adjusting the beam configuration to selectively pivot one or more beam lights using one or more motors based at least on the one or more regions.   
     
     
         4 . The method of  claim 1 , wherein the determining the one or more regions is based at least on generating one or more two-dimensional mappings from one or more locations of one or more sensors used to generate the image data to one or more beam locations corresponding to the beam configuration. 
     
     
         5 . The method of  claim 1 , wherein the determining the one or more regions includes determining the one or more regions as corresponding to one or more inactive objects and the beam configuration maintains or increases illumination for the one or more regions based at least on the one or more regions corresponding to the one or more inactive objects. 
     
     
         6 . The method of  claim 1 , wherein the determining the one or more regions includes determining the one or more regions as corresponding to one or more active objects and the beam configuration at least reduces illumination for the one or more regions based at least on the one or more regions corresponding to the one or more active objects. 
     
     
         7 . The method of  claim 1 , comprising:
 computing, using the one or more NNs, one or more segmentation masks classifying pixels of the one or more images as corresponding to the one or more objects; and   generating, using the one or more segmentation masks, one or more dim region masks indicating the one or more regions for adjustment of illumination, wherein the beam configuration is determined using the one or more dim region masks.   
     
     
         8 . The method of  claim 1 , wherein the determining the one or more regions includes determining, using the one or more NNs, a status of one or more lights of the one or more objects, and the beam configuration is based at least on the status of the one or more lights of the one or more objects. 
     
     
         9 . A system comprising:
 one or more circuits to perform operations including:
 computing, using one or more neural networks (NNs) and based at least on image data representative of images of an environment, one or more first confidence values for one or more first images of the images as corresponding to one or more objects and one or more second confidence values for one or more second images of the images as corresponding to the one or more objects; 
 determining one or more pixels of the one or more second images correspond to the one or more objects based at least on the one or more first confidence values and the one or more second confidence values; and 
 determining a beam configuration based at least on the one or more pixels corresponding to the one or more objects. 
   
     
     
         10 . The system of  claim 9 , wherein the determining the one or more pixels of the one or more second images correspond to the one or more objects includes:
 weighting the one or more second confidence values based at least on the first confidence values to generate one or more updated confidence values; and   determining the one or more pixels using the one or more updated confidence values.   
     
     
         11 . The system of  claim 9 , wherein the determining the one or more pixels of the one or more second images correspond to the one or more objects includes applying the one or more first confidence values and the one or more second confidence values to a recursive Bayesian filter. 
     
     
         12 . The system of  claim 9 , wherein the determining the one or more pixels of the one or more second images correspond to the one or more objects includes applying the one or more first confidence values and the one or more second confidence values to a temporal filter that adjusts the one or more first confidence values and the one or more second confidence values using hysteresis gating. 
     
     
         13 . The system of  claim 9 , wherein the determining the beam configuration includes one or more of:
 adjusting the beam configuration to reduce illumination for one or more first portions of one or more regions of the environment corresponding to the one or more pixels;   adjusting the beam configuration to increase illumination for one or more second portions of the one or more regions of the environment; or   adjusting the beam configuration to selectively pivot one or more beam lights using one or more motors based at least on the one or more regions of the environment.   
     
     
         14 . The system of  claim 9 , wherein the system is comprised in at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing light transport simulation;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system implemented at least partially in a data center; or   a system implemented at least partially using cloud computing resources.   
     
     
         15 . A processor comprising:
 one or more circuits to determine a beam configuration of a lighting system based at least on one or more regions corresponding to one or more objects, the one or more regions determined based at least on one or more neural networks (NNs) processing image data representative of one or more images of an environment.   
     
     
         16 . The processor of  claim 15 , wherein the one or more regions are determined, at least, by:
 computing, using the one or more NNs, one or more segmentation masks classifying pixels of the one or more images as corresponding to the one or more objects; and   determining the one or more regions based at least on the classifying of the pixels.   
     
     
         17 . The processor of  claim 15 , wherein the determining the beam configuration includes one or more of:
 adjusting the beam configuration to reduce illumination for one or more first portions of the one or more regions;   adjusting the beam configuration to increase illumination for one or more second portions of the one or more regions; or   adjusting the beam configuration to selectively pivot one or more beam lights using one or more motors based at least on the one or more regions.   
     
     
         18 . The processor of  claim 15 , wherein the one or more regions are determined, at least, by generating one or more two-dimensional mappings from one or more locations of one or more sensors used to generate the image data to one or more beam locations corresponding to the beam configuration. 
     
     
         19 . The processor of  claim 15 , wherein the one or more regions are determined, at least, by determining the one or more regions as corresponding to one or more inactive objects and the beam configuration maintains or increases illumination for the one or more regions based at least on the one or more regions corresponding to the one or more inactive objects. 
     
     
         20 . The processor of  claim 15 , wherein the processor is comprised in at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing light transport simulation;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system implemented at least partially in a data center; or   a system implemented at least partially using cloud computing resources.