Stereo depth estimation utilizing asymmetric downsampling in different directions

Abstract

Aspects relate to stereo depth estimation utilizing asymmetric down-sampling in different directions. A device may include one or more memories configured to store a plurality of images and a plurality of cameras. The plurality of cameras may be configured to capture a left and right image, in which, each of the images includes one or more patches, each patch including plurality of pixels. The device may include one or more processors coupled to one or more memories, in which, the one or more processors are configured to: down-sample in a first direction on a first set of pixels in a first patch of a first image to generate a first down-sample; and down-sample in a second direction on a second set of pixels in a second patch of a second image to generate a second down-sample, the second down-sample including a greater number of pixels.

What is claimed is: 
     
         1 . A device comprising:
 one or more memories configured to store a plurality of images;   a plurality of cameras configured to capture a left and right image, wherein each of the images includes one or more patches, each patch including plurality of pixels; and   one or more processors coupled to the one or memories, the one or more processors are configured to:
 down-sample in a first direction on a first set of pixels in a first patch of a first image to generate a first down-sample; and 
 down-sample in a second direction on a second set of pixels in a second patch of a second image to generate a second down-sample, wherein, the second down-sample includes a greater number of pixels. 
   
     
     
         2 . The device of  claim 1 , wherein the first down-sample in the first direction is in height and the second down-sample in the second direction is in width, such that, the first and second down-sample is an asymmetric down-sample operation that includes a higher resolution in width. 
     
     
         3 . The device of  claim 2 , wherein, multiple asymmetric down-sample operations are performed in a down-sampling process, each asymmetric down-sample operation including a width-to-height aspect ratio. 
     
     
         4 . The device of  claim 3 , wherein the multiple width-to-height aspect ratios are equal or increasing or decreasing during the down-sampling process. 
     
     
         5 . The device of  claim 3 , further comprising performing depth estimation in the down-sampling process. 
     
     
         6 . The device of  claim 3 , wherein, the one or more processors are configured to perform an up-sampling process. 
     
     
         7 . The device of  claim 3 , wherein, the one or more processors are configured to:
 render the output of the down-sampling process for the left and right images; and   combine the left and right rendered images to generate a stereo image output.   
     
     
         8 . The device of  claim 7 , wherein, the down-sampling process further comprises implementing a multi-aspect ratio method for estimating stereo disparity. 
     
     
         9 . The device of  claim 8 , wherein, based upon the implementation of the multi-aspect ratio method for estimating stereo disparity in the down-sampling process, the stereo image output rendered by the down-sampling process includes stereo depth map resolution replicating original stereo depth map resolution associated with the original stereo image. 
     
     
         10 . The device of  claim 7 , further comprising a display device, wherein, the one or more processors are configured to command the display of the stereo image output on the display device. 
     
     
         11 . The device of  claim 5 , further comprising a modem configured to transmit output from the down-sampling process to another device. 
     
     
         12 . The device of  claim 5 , wherein, the one or more processors are further configured to: implement a machine learning model including down-sampling stages to implement the down-sampling process. 
     
     
         13 . The device of  claim 12 , wherein, the machine learning model is a neural network. 
     
     
         14 . The device of  claim 2 , wherein the asymmetric operations include the use of asymmetric space-to-depth operations in a disparity width dimension, wherein a smaller rate through division in the disparity width dimension is used than in other non-disparity dimensions. 
     
     
         15 . The device of  claim 2 , wherein the asymmetric operations include the use of asymmetric depth-to-space operations in a disparity width dimension, wherein a larger rate through multiplication in the disparity width dimension is used than in other non-disparity dimensions. 
     
     
         16 . The device of  claim 2 , wherein the plurality of cameras include a left camera and a right camera, wherein, the left camera is configured to capture the left image and right camera is configured to capture the right image, and the one or more processors are configured to generate both the first down-sample and the second down-sample from the left and right image, respectively. 
     
     
         17 . A method for providing a stereo image, the method comprising:
 capturing one or more images, wherein each of the images includes one or more patches, each patch including plurality of pixels;   down-sampling in a first direction on a first set of pixels in a first patch of a first image to generate a first down-sample; and   down-sampling in a second direction on a second set of pixels in a second patch of a second image to generate a second down-sample, wherein, the second down-sample includes a greater number of pixels.   
     
     
         18 . The method of  claim 17 , wherein the first down-sample in the first direction is in height and the second down-sample in the second direction is in width, such that, the first and second down-sample is an asymmetric down-sample operation that includes a higher resolution in width. 
     
     
         19 . The method of  claim 18 , wherein, multiple asymmetric down-sample operations are performed in a down-sampling process, each asymmetric down-sample operation including a width-to-height aspect ratio. 
     
     
         20 . A non-transitory computer-readable data storage medium having stored thereon instructions that, when executed, cause one or more processors to:
 capture one or more images, wherein each of the images includes one or more patches, each patch including plurality of pixels;   down-sample in a first direction on a first set of pixels in a first patch of a first image to generate a first down-sample; and   down-sample in a second direction on a second set of pixels in a second patch of a second image to generate a second down-sample, wherein, the second down-sample includes a greater number of pixels.