Method and system for optimizing depth imaging

Abstract

There is provided a system and method for optimizing depth imaging. The method including: illuminating one or more scenes with illumination patterns; capturing one or more images of each of the scenes; reconstructing the scenes; estimating the reconstruction error and a gradient of the reconstruction error; iteratively performing until the reconstruction error reaches a predetermined error condition: determining a current set of control vectors and current set of reconstruction parameters; illuminating the one or more scenes with the illumination patterns governed by the current set of control vectors; capturing one or more images of each of the scenes while the scene is being illuminated with at least one of the illumination patterns; reconstructing the scenes from the one or more captured images using the current reconstruction parameters; and estimating an updated reconstruction error and gradient; and outputting at least one of control vectors and reconstruction parameters.

1 . A computer-implemented method for generating a depth image of a scene, the method comprising:
 illuminating the scene with one or more illumination patterns, each pattern comprising a plurality of discretized elements, intensity of each element governed by a code vector;   capturing one or more images of the scene while the scene is being illuminated;   for each pixel, generating an observation vector comprising at least one intensity recorded at the pixel for each of the captured images;   for each pixel, determining the code vector that best corresponds with the respective observation vector by maximizing the zero-mean normalized cross-correlation (ZNCC);   for each pixel, determining a depth value from the best-corresponding code vector; and   outputting the depth values as a depth image.   
     
     
         2 . The method of  claim 1 , wherein each observation vector incorporates intensities of neighbouring image pixels, and wherein each code vector incorporates neighbouring discretized intensities. 
     
     
         3 . The method of  claim 2 , further comprising:
 using a trained artificial neural network to transform each observation vector to a higher-dimensional feature vector; and   using a trained artificial neural network to transform each code vector to a higher-dimensional feature vector,   wherein determining the code vector that best corresponds with the respective observation vector comprises maximizing the ZNCC between the transformed respective observation vector and the transformed code vectors.   
     
     
         4 . The method of  claim 1 , wherein each illumination pattern is a discretized two-dimensional pattern that is projected onto a scene from a viewpoint that is distinct from the captured images, wherein each element in the pattern is a projected pixel, and wherein determining the depth value from the best-corresponding code vector comprises triangulation. 
     
     
         5 . The method of  claim 1 , wherein each illumination pattern comprises multiple wavelength bands, wherein the observation vector at each pixel comprises the raw or demosaiced intensities of each wavelength band for the respective pixel. 
     
     
         6 . The method of  claim 1 , wherein the discretized elements of each illumination pattern comprise a discretized time-varying pattern that modulates the intensity of a light source, each element in the pattern is associated with a time-of-flight delay and a code vector, and wherein determining the depth value from the best-corresponding code vector comprises multiplication by the speed of light.