Method and device for ascertaining a depth information image from an input image

Abstract

A method for ascertaining a depth information image for an input image. The input image is processed using a convolutional neural network, which includes multiple layers that sequentially process the input image, and each converts an input feature map into an output feature map. At least one of the layers is a depth map layer, the depth information image being ascertained as a function of a depth map layer. In the depth map layer, an input feature map of the depth map layer is convoluted with multiple scaling filters to obtain respective scaling maps, the multiple scaling maps are compared pixel by pixel to generate a respective output feature map in which each pixel corresponds to a corresponding pixel from a selected one of the scaling maps.

What is claimed is: 
     
       1. A method for ascertaining a depth information image for an input image in order to control vehicle functions as a function of the depth information image, the method comprising:
 processing the input image using a convolutional neural network, the convolutional neural network including multiple layers that sequentially process the input image and which each convert an input feature map into an output feature map, at least one of the layers is a depth map layer, the depth information image being ascertained as a function of a depth map layer; 
 wherein, in the depth map layer:
 the input feature map of the depth map layer is convoluted with multiple scaling filters to obtain respective scaling maps, 
 the multiple scaling maps are compared pixel by pixel to generate a respective output feature map in which each pixel corresponds to a corresponding pixel from a selected one of the scaling maps, and 
 a scaling feature map is generated by associating each pixel of the scaling feature map with a piece of information that indicates the selected one of the scaling maps from which the pixel of the output feature map is selected; 
 
 wherein the depth information image corresponds to the scaling feature map or is determined as a function of the scaling feature map. 
 
     
     
       2. The method as recited in  claim 1 , wherein the selected one of scaling maps corresponds to a scaling map of the scaling maps that contains a largest pixel value for the pixel. 
     
     
       3. The method as recited in  claim 2 , wherein the scaling filters are determined from a filter kernel of the convolutional neural network by downsampling or upsampling. 
     
     
       4. The method as recited in  claim 1 , wherein multiple scaling feature maps are ascertained in multiple depth map layers, the depth information image being ascertained from the multiple scaling feature maps using a further neural network, the depth information image corresponding to a depth map. 
     
     
       5. The method as recited in  claim 4 , wherein the depth information image is ascertained from the multiple scaling feature maps and one or more output feature maps of one or multiple of the layers of the neural network and/or of an output image of the neural network. 
     
     
       6. The method as recited in  claim 1 , wherein the neural network generates an output image, at least one of the layers of the multiple layers generating an output feature map and/or the output image as a function of one or more of the scaling feature maps, the one or more of the scaling feature maps of the input feature map supplied to the at least one of the layers of the multiple layers. 
     
     
       7. The method as recited in  claim 6 , wherein the output image and the depth information image are processed together in a downstream additional neural network. 
     
     
       8. The method as recited in  claim 1 , further comprising:
 using the depth information image to control a vehicle function that relates to: 
 (i) a fully autonomous or semiautonomous driving operation, or (ii) a driver assistance function for warning of objects in surroundings. 
 
     
     
       9. A device for ascertaining a depth information image for an input image in order to control vehicle functions as a function of the depth information image, the device configured to:
 process the input image using a convolutional neural network, the convolutional neural network including multiple layers that sequentially process the input image and which each convert an input feature map into an output feature map, at least one of the layers being a depth map layer, the depth information image being ascertained as a function of a depth map layer; 
 wherein, for the at least one depth map layer, the device being configured to:
 convolute an input feature map of the depth map layer with multiple scaling filters to obtain respective scaling maps, 
 compare the multiple scaling maps pixel by pixel to generate a respective output feature map in which each pixel corresponds to a corresponding pixel from a selected one of the scaling maps, and 
 generate a scaling feature map by associating each pixel of the scaling feature map with a piece of information that indicates the selected one of the scaling maps from which the pixel of the output feature map is selected; 
 
 wherein the depth information image corresponds to the scaling feature map or is determined as a function of the scaling feature map. 
 
     
     
       10. A system, comprising;
 an image detection device configured to detect an input image; 
 a preprocessing device for providing a depth information image as a function of the input image, the preprocessing device configured to:
 process the input image using a convolutional neural network, the convolutional neural network including multiple layers that sequentially process the input image and which each convert an input feature map into an output feature map, at least one of the layers being a depth map layer, the depth information image being ascertained as a function of a depth map layer; 
 wherein, for the at least one depth map layer, the device being configured to:
 convolute an input feature map of the depth map layer with multiple scaling filters to obtain respective scaling maps, 
 compare the multiple scaling maps pixel by pixel to generate a respective output feature map in which each pixel corresponds to a corresponding pixel from a selected one of the scaling maps, and 
 generate a scaling feature map by associating each pixel of the scaling feature map with a piece of information that indicates the selected one of the scaling maps from which the pixel of the output feature map is selected; 
 
 wherein the depth information image corresponds to the scaling feature map or is determined as a function of the scaling feature map; and 
 
 a control unit configured to control at least one actuator of the system as a function of the depth information image. 
 
     
     
       11. An non-transitory electronic memory medium on which is stored a computer program for ascertaining a depth information image for an input image in order to control vehicle functions as a function of the depth information image, the computer program, when executed by a computer, causing the computer to perform the following:
 processing the input image using a convolutional neural network, the convolutional neural network including multiple layers that sequentially process the input image and which each convert an input feature map into an output feature map, at least one of the layers is a depth map layer, the depth information image being ascertained as a function of a depth map layer; 
 wherein, in the depth map layer:
 the input feature map of the depth map layer is convoluted with multiple scaling filters to obtain respective scaling maps, 
 the multiple scaling maps are compared pixel by pixel to generate a respective output feature map in which each pixel corresponds to a corresponding pixel from a selected one of the scaling maps, and 
 a scaling feature map is generated by associating each pixel of the scaling feature map with a piece of information that indicates the selected one of the scaling maps from which the pixel of the output feature map is selected; 
 
 wherein the depth information image corresponds to the scaling feature map or is determined as a function of the scaling feature map.