Hybrid depth sensing pipeline

Abstract

An apparatus for a hybrid tracking and mapping is described herein. The apparatus includes logic to determine a plurality of depth sensing techniques. The apparatus also includes logic to vary the plurality of depth sensing techniques based on a camera configuration. Additionally, the apparatus includes logic to generate a hybrid tracking and mapping pipeline based on the depth sensing techniques and the camera configuration.

1 - 23 . (canceled) 
     
     
         24 . An image capture device including at least one sensor, wherein the sensor captures depth information that is processed by a hybrid tracking and mapping pipeline, and the hybrid tracking and mapping pipeline includes a plurality of depth sensing techniques, wherein in response to a plurality of sensors the depth sensing techniques comprise at least stereoscopic imaging, otherwise the depth sensing techniques comprise at least virtual stereoscopic reconstruction to produce a variable resolution depth representation. 
     
     
         25 . The image capture device of  claim 24 , wherein the depth sensing techniques include disparity estimation from stereoscopic image left/right pairs, virtual stereoscopic frames, multi-view stereoscopic (MVS) reconstruction from image sequences, and structured light scanning 
     
     
         26 . The image capture device of  claim 24 , wherein the plurality of depth sensing techniques varies based on a camera configuration. 
     
     
         27 . The image capture device of  claim 24 , wherein virtual stereoscopic reconstruction is performed by rectifying, aligning, and cropping sequential image data. 
     
     
         28 . The image capture device of  claim 24 , wherein neighboring pixels or features are used to interpolate values within a stereoscopic left/right frame pair. 
     
     
         29 . The image capture device of  claim 24 , wherein a bundle of past frames, a past point cloud, an image pyramid buffer, or any combination thereof are used in an MVS reconstruction stage to improve the depth map. 
     
     
         30 . The image capture device of  claim 24 , wherein the hybrid tracking and mapping pipeline generates a depth representation. 
     
     
         31 . A system for hybrid tracking and mapping, comprising:
 a display;   a radio;   a memory that is to store instructions and that is communicatively coupled to the display; and   a processor communicatively coupled to the radio and the memory, wherein when the processor is to execute the instructions, the processor is to:   determine a plurality of depth sensing techniques;   vary the plurality of depth sensing techniques based on a camera configuration, wherein in response to a plurality of sensors in the camera configuration the depth sensing techniques comprise at least stereoscopic imaging, otherwise the depth sensing techniques comprise at least virtual stereoscopic reconstruction; and   generate a hybrid tracking and mapping pipeline to produce a variable resolution depth representation based on the depth sensing techniques and the camera configuration.   
     
     
         32 . The system of  claim 31 , wherein the depth sensing techniques include disparity estimation from stereoscopic image left/right pairs, virtual stereoscopic frames, multi-view stereoscopic (MVS) reconstruction from image sequences, and structured light scanning 
     
     
         33 . The system of  claim 31  wherein the camera configuration includes a sensor type, an amount of power available, and a resolution of an image sensor. 
     
     
         24 . The system of  claim 31 , wherein algorithm tuning is used to generate the hybrid tracking and mapping pipeline. 
     
     
         35 . The system of  claim 31 , wherein the hybrid tracking and mapping pipeline is used to produce a variable resolution depth representation. 
     
     
         36 . A method for hybrid tracking and mapping, comprising:
 determining a plurality of depth sensing techniques;   varying the plurality of depth sensing techniques based on a camera configuration and a compute unit configuration, wherein the camera configuration is a monocular camera that produces a sequence of frames, wherein in response to an accelerometer in the compute unit configuration the accelerometer is to guide a depth algorithm selection; and   generating a hybrid tracking and mapping pipeline to produce a variable resolution depth representation based on the depth sensing techniques and the camera configuration.   
     
     
         37 . The method of  claim 36 , further comprising generating a virtual stereo pair of frames using a frame N and a frame N- 1  of the sequence of frames by:
 locating fiducial marks in the sequence of frames; 
 determining the orientation of a camera pose; and 
 performing an affine transform on the frame N and the frame N- 1 . 
 
     
     
         38 . The method of  claim 37 , wherein the virtual stereo pair of frames is used to generate depth information.