Depth Map Calculation in a Stereo Camera System

Abstract

A method for generating a depth map is described. The method includes obtaining a first image of a scene from a first image capture unit, the first image having a first depth-of-field (DOF), obtaining a second image of the scene from a second image capture unit, the second image having a second DOF that is different than the first DOF. Each pixel in the second image has a corresponding pixel in the first image. The method also includes generating a plurality of third images, each corresponding to a blurred version of the second image at each of a plurality of specified depths, generating a plurality of fourth images, each representing a difference between the first image and one or the plurality of third images, and generating a depth map where each pixel in the depth map is based on the pixels in one of the plurality of fourth images.

1 . A method of generating a depth map, comprising:
 capturing a first plurality of images of a scene from a first image capture unit having a first depth of field, wherein each of the first plurality of images is associated with a different depth;   determining a sharpness metric for each of the first plurality of images; and   generating a depth map comprising a plurality of pixels, wherein generating the depth map comprises selecting, for each pixel in the depth map, a depth associated with one of the first plurality of images, based on a comparison of the sharpness metrics for the first plurality of images.   
     
     
         2 . The method of  claim 1 , further comprising:
 capturing a preview image of the scene from a second image capture unit having a second depth of field.   
     
     
         3 . The method of  claim 2 , wherein:
 the capturing of the preview image is done concurrently with the capture of the first plurality of images.   
     
     
         4 . The method of  claim 2 , further comprising:
 estimating a relative blur between each image in the first plurality of images and the preview image,   wherein each pixel in the depth map is further based on the estimated relative blurs.   
     
     
         5 . The method of  claim 2 , wherein the first image capture unit comprises a telephoto lens, and wherein the second image capture unit comprises a wide-angle lens. 
     
     
         6 . The method of  claim 1 , wherein determining the sharpness metric further comprises determining a relative degree of blur between at least two consecutively captured images of the first plurality of images. 
     
     
         7 . The method of  claim 1 , wherein selecting, for each pixel in the depth map, a depth associated with one of the first plurality of images, based on a comparison of the sharpness metrics for the first plurality of images further comprises:
 selecting, for each pixel in the depth map, the depth associated with a sharpest image from the first plurality of images at the respective pixel location.   
     
     
         8 . A non-transitory computer readable medium comprising computer code for generating a depth map, the computer code executable by one or more processors to:
 obtain a first plurality of images of a scene from a first image capture unit having a first depth of field, wherein each of the first plurality of images is associated with a different depth;   determine a sharpness metric for each of the first plurality of images; and   generate a depth map comprising a plurality of pixels, wherein generating the depth map comprises selecting, for each pixel in the depth map, a depth associated with one of the first plurality of images, based on a comparison of the sharpness metrics for the first plurality of images.   
     
     
         9 . The non-transitory computer readable medium of  claim 8 , wherein the computer code further comprises computer code executable by the one or more processors to:
 obtain a preview image of the scene from a second image capture unit having a second depth of field.   
     
     
         10 . The non-transitory computer readable medium of  claim 9 , wherein:
 the preview image is captured concurrently with a capture of the first plurality of images.   
     
     
         11 . The non-transitory computer readable medium of  claim 9 , wherein the computer code further comprises computer code executable by the one or more processors to:
 estimate a relative blur between each image in the first plurality of images and the preview image,   wherein each pixel in the depth map is further based on the estimated relative blurs.   
     
     
         12 . The non-transitory computer readable medium of  claim 9 , wherein the first image capture unit comprises a telephoto lens, and wherein the second image capture unit comprises a wide-angle lens. 
     
     
         13 . The non-transitory computer readable medium of  claim 8 , wherein the computer code executable by the one or more processors to determine the sharpness metric further comprises computer code executable by the one or more processors to:
 determine a relative degree of blur between at least two consecutively captured images of the first plurality of images.   
     
     
         14 . The non-transitory computer readable medium of  claim 8 , wherein the computer code executable by the one or more processors to select, for each pixel in the depth map, a depth associated with one of the first plurality of images, based on a comparison of the sharpness metrics for the first plurality of images further comprises computer code executable by the one or more processors to:
 select, for each pixel in the depth map, the depth associated with a sharpest image from the first plurality of images at the respective pixel location.   
     
     
         15 . A system for generating a depth map, comprising:
 a first image capture unit having a first depth of field;
 a second image capture unit having a second depth of field; and 
 a memory comprising computer code configured to cause one or more processors to:
 capture a first plurality of images of a scene from the first image capture unit, wherein each of the first plurality of images is associated with a different depth; 
 determine a sharpness metric for each of the first plurality of images; and 
 generate a depth map comprising a plurality of pixels, wherein generating the depth map comprises selecting, for each pixel in the depth map, a depth associated with one of the first plurality of images, based on a comparison of the sharpness metrics for the first plurality of images. 
 
   
     
     
         16 . The system of  claim 15 , wherein the computer code further comprises computer code executable by the one or more processors to:
 capture a preview image of the scene from the second image capture unit.   
     
     
         17 . The system of  claim 16 , wherein the computer code further comprises computer code executable by the one or more processors to:
 estimate a relative blur between each image in the first plurality of images and the preview image,   wherein each pixel in the depth map is further based on the estimated relative blurs.   
     
     
         18 . The system of  claim 15 , wherein the first image capture unit comprises a telephoto lens, and wherein the second image capture unit comprises a wide-angle lens. 
     
     
         19 . The system of  claim 15 , wherein the computer code executable by the one or more processors to determine the sharpness metric further comprises computer code executable by the one or more processors to:
 determine a relative degree of blur between at least two consecutively captured images of the first plurality of images.   
     
     
         20 . The system of  claim 15 , wherein the computer code executable by the one or more processors to select, for each pixel in the depth map, a depth associated with one of the first plurality of images, based on a comparison of the sharpness metrics for the first plurality of image further comprises computer code executable by the one or more processors to:
 select, for each pixel in the depth map, the depth associated with a sharpest image from the first plurality of images at the respective pixel location.