Under vehicle reconstruction for vehicle environment visualization

Abstract

In various examples, cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area. For each time slice, a newly captured or generated image representing that time slice may be used to generate a local map of an observed portion of the ground, and the local map may be merged with a composite map that represents previously observed local maps. Accordingly, the under vehicle area for that time slice may be reconstructed by retrieving corresponding pixels from the composite map using the vehicle's ego-motion.

What is claimed is: 
     
         1 . A method comprising:
 generating, using image data generated using one or more cameras of an ego-object during a first time slice, a local map representing an observed portion of a ground plane of an environment surrounding the ego-object;   updating a representation of the ground plane based at least on the local map; and   virtually reconstructing an area of the ground plane under the ego-object during the first time slice based at least on retrieving a corresponding portion of the representation of the ground plane.   
     
     
         2 . The method of  claim 1 , wherein the generating of the local map includes orienting the local map in a direction corresponding to a direction of ego-motion of the ego-object. 
     
     
         3 . The method of  claim 1 , wherein the generating of the local map includes determining to omit, from the local map, one or more color values of one or more pixels of the image data that do not belong to a segmented navigable space. 
     
     
         4 . The method of  claim 1 , wherein the generating of the local map includes determining a dimension of the local map based at least on a speed of the ego-object. 
     
     
         5 . The method of  claim 1 , wherein the generating of the local map includes texturing the local map with color values of pixels of the image data that project onto the local map. 
     
     
         6 . The method of  claim 1 , wherein the representation of the ground plane is a composite map, and the updating of the composite map comprises merging the local map into a composite representation of local maps generated during previous time slices. 
     
     
         7 . The method of  claim 1 , wherein the representation of the ground plane is a composite map, and the updating of the composite map limits the composite map to representing local maps generated during a designated number of time slices. 
     
     
         8 . The method of  claim 1 , wherein the retrieving of the corresponding portion of the representation of the ground plane texturizes the area under the ego-object based at least on assigning a color value to at least one cell of one or more cells in a grid in the area under the ego-object, the color value being retrieved from a corresponding pixel of the representation of the ground plane. 
     
     
         9 . The method of  claim 1 , wherein the generating of the local map comprises selecting the image data based at least on a corresponding one of the one or more cameras pointing in a direction of ego-motion of the ego-object. 
     
     
         10 . The method of  claim 1 , wherein the method is performed by at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing digital twin operations;   a system for performing light transport simulation;   a system for performing collaborative content creation for 3D assets;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system for performing conversational AI operations;   a system for generating synthetic data;   a system incorporating one or more virtual machines (VMs);   a system implemented at least partially in a data center; or   a system implemented at least partially using cloud computing resources.   
     
     
         11 . A processor comprising:
 one or more circuits to:
 merge, into a representation of a ground plane representing portions of the ground plane observed by one or more sensors of an ego-object, a local map representing a portion of the ground plane observed by the one or more sensors during a first time slice; and 
 virtually reconstruct an area of the ground plane under the ego-object during the first time slice based at least on retrieving a corresponding portion of the representation of the ground plane. 
   
     
     
         12 . The processor of  claim 11 , the one or more circuits further to generate the local map based at least on orienting the local map in a direction corresponding to a direction of ego-motion of the ego-object. 
     
     
         13 . The processor of  claim 11 , the one or more circuits further to generate the local map based at least on determining to omit, from the local map, one or more color values of one or more pixels of image data that do not belong to a segmented navigable space. 
     
     
         14 . The processor of  claim 11 , the one or more circuits further to generate the local map based at least on determining a dimension of the local map corresponding to a speed of the ego-object. 
     
     
         15 . The processor of  claim 11 , wherein the processor is comprised in at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing digital twin operations;   a system for performing light transport simulation;   a system for performing collaborative content creation for 3D assets;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system for performing conversational AI operations;   a system for generating synthetic data;   a system incorporating one or more virtual machines (VMs);   a system implemented at least partially in a data center; or   a system implemented at least partially using cloud computing resources.   
     
     
         16 . A system comprising:
 one or more processing units to generate
 a local map representing an observed portion of a ground plane of an environment surrounding an ego-object based at least on projecting image data from one or more cameras of the ego-object captured during a first time slice, cache the local map in association with a representation of the ground plane, and virtually reconstruct an area of the ground plane under the ego-object during the first time slice by retrieving a corresponding portion of the representation of the ground plane. 
   
     
     
         17 . The system of  claim 16 , the one or more processing units further to limit the representation of the ground plane to representing local maps generated during a designated number of time slices. 
     
     
         18 . The system of  claim 16 , wherein the retrieving of the corresponding portion of the representation of the ground plane texturizes the area under the ego-object based at least on assigning a color value, to each of one or more cells in a grid in the area under the ego-object, retrieved from a corresponding pixel of the representation of the ground plane. 
     
     
         19 . The system of  claim 16 , wherein the one or more processing units generate the local map by selecting the image data based at least on a corresponding one of the one or more cameras pointing in a direction of ego-motion of the ego-object. 
     
     
         20 . The system of  claim 16 , wherein the system is comprised in at least one of:
 a control system for an autonomous or semi-autonomous machine;   a perception system for an autonomous or semi-autonomous machine;   a system for performing simulation operations;   a system for performing digital twin operations;   a system for performing deep learning operations;   a system implemented using an edge device;   a system implemented using a robot;   a system incorporating one or more virtual machines (VMs);   a system implemented at least partially in a data center;   a system for performing light transport simulation;   a system for performing collaborative content creation for 3D assets;   a system for generating synthetic data; or   a system implemented at least partially using cloud computing resources.