System and method for adaptive depth map reconstruction

Abstract

What is disclosed is a system and method for adaptively reconstructing a depth map of a scene. In one embodiment, upon receiving a mask identifying a region of interest (ROI), a processor changes either a spatial attribute of a pattern of source light projected on the scene by a light modulator which projects an undistorted pattern of light with known spatio-temporal attributes on the scene, or changes an operative resolution of a depth map reconstruction module. A sensing device detects the reflected pattern of light. A depth map of the scene is generated by the depth map reconstruction module by establishing correspondences between spatial attributes in the detected pattern and spatial attributes of the projected undistorted pattern and triangulating the correspondences to characterize differences therebetween. The depth map is such that a spatial resolution in the ROI is higher relative to a spatial resolution of locations not within the ROI.

What is claimed is:  
     
         1 . A method for adaptive depth map reconstruction comprising:
 receiving a mask identifying at least one region of interest (ROI) in a scene;   using a multi-camera array to acquire images of said scene, said multi-camera array comprising at least two imaging devices with overlapping fields of view such that salient features in said scene are simultaneously imaged by said at least two imaging devices; and   reconstructing, by a depth map reconstruction module, a depth map of said scene from said acquired images, said reconstructing comprising:
 establishing correspondences between said salient features; 
 changing a density of said established correspondences by adjusting a selectivity threshold that determines a degree of saliency required for a feature to be considered salient such that said density is higher at locations corresponding to said ROI, as identified by said mask; 
 determining offsets between said established correspondences; 
 triangulating said offsets to obtain depth values; and 
 aggregating said depth values to reconstruct said depth map, said reconstructed depth map being such that a spatial resolution in said ROI is higher relative to a spatial resolution of scene locations not associated with said ROI. 
   
     
     
         2 . The method of  claim 1 , wherein said ROI is identified in said scene using any of:
 pixel classification, object identification, facial recognition, pattern recognition, color, texture, spatial features, depth-based segmentation, spectral feature-based segmentation, motion detection, and a user selection.   
     
     
         3 . The method of  claim 1 , wherein said mask is updated in real-time in response to any of: a location of said ROI changing in said scene, a new ROI being identified in said scene, a ROI no longer being considered an ROI, and a user input. 
     
     
         4 . A system for adaptive depth map reconstruction comprising:
 a mask identifying at least one region of interest (ROI) in a scene;   a multi-camera array for acquiring images of said scene, said multi-camera array comprising at least two imaging devices with overlapping fields of view such that salient features in said scene are simultaneously imaged by said at least two imaging devices; and   a depth map reconstruction module comprising a processor executing machine readable program instructions for reconstructing a depth map of said scene from said acquired images, said reconstructing comprising:
 establishing correspondences between said salient features; 
 changing a density of said established correspondences by adjusting a selectivity threshold that determines a degree of saliency required for a feature to be considered salient such that said density is higher at locations corresponding to said ROI, as identified by said mask; 
 determining offsets between said established correspondences; 
 triangulating said offsets to obtain depth values; and 
 aggregating said depth values to reconstruct said depth map, said reconstructed depth map being such that a spatial resolution in said ROI is higher relative to a spatial resolution of scene locations not associated with said ROI. 
   
     
     
         5 . The system of  claim 4 , wherein said ROI is identified in said scene using any of:
 pixel classification, object identification, facial recognition, pattern recognition, color, texture, spatial features, depth-based segmentation, spectral feature-based segmentation, motion detection, and a user selection.   
     
     
         6 . The system of  claim 4 , wherein said mask is updated in real-time in response to any of: a location of said ROI changing in said scene, a new ROI being identified in said scene, a ROI no longer being considered an ROI, and a user input. 
     
     
         7 . A method for adaptive depth map reconstruction comprising:
 receiving a mask identifying at least one region of interest (ROI) in a scene;   using a time-of-flight system to acquire point-by-point range measurements of said scene; and   reconstructing, by a depth map reconstruction module, a depth map of said scene from said acquired point-by-point range measurements, said reconstructing comprising:
 establishing correspondences between said point-by-point range measurements; 
 changing a density of said established correspondences by selectively adjusting a scanning speed of said point-by-point device to be higher at locations corresponding to said ROI, as identified by said mask; 
 determining offsets between said established correspondences; 
 triangulating said offsets to obtain depth values; and 
 aggregating said depth values to reconstruct said depth map, said reconstructed depth map being such that a spatial resolution in said ROI is higher relative to a spatial resolution of scene locations not associated with said ROI. 
   
     
     
         8 . The method of  claim 7 , further comprising changing said density by any of:
 selectively de-activating any of said point-by-point devices for array locations not associated with the ROI, and selectively downsampling said point-by-point range measurements for array locations not associated with the ROI.   
     
     
         9 . The method of  claim 7 , wherein said ROI is identified in said scene using any of:
 pixel classification, object identification, facial recognition, pattern recognition, color, texture, spatial features, depth-based segmentation, spectral feature-based segmentation, motion detection, and a user selection.   
     
     
         10 . The method of  claim 7 , wherein said mask is updated in real-time in response to any of: a location of said ROI changing in said scene, a new ROI being identified in said scene, a ROI no longer being considered an ROI, and a user input. 
     
     
         11 . A system for adaptive depth map reconstruction comprising:
 a mask identifying at least one region of interest (ROI) in a scene;   a time-of-flight system for acquiring point-by-point range measurements of said scene; and   a depth map reconstruction module comprising a processor executing machine readable program instructions for reconstructing a depth map of said scene from said point-by-point range measurements, said reconstructing comprising:
 establishing correspondences between said point-by-point range measurements; 
 changing a density of said established correspondences by selectively adjusting a scanning speed of said point-by-point device to be higher at locations corresponding to said ROI, as identified by said mask; 
 determining offsets between said established correspondences; 
 triangulating said offsets to obtain depth values; and 
 aggregating said depth values to reconstruct said depth map, said reconstructed depth map being such that a spatial resolution in said ROI is higher relative to a spatial resolution of scene locations not associated with said ROI. 
   
     
     
         12 . The system of  claim 11 , further comprising changing said density by any of: selectively de-activating any of said point-by-point devices for array locations not associated with the ROI, and selectively downsampling said point-by-point range measurements for array locations not associated with the ROI. 
     
     
         13 . The system of  claim 11 , wherein said ROI is identified in said scene using any of: pixel classification, object identification, facial recognition, pattern recognition, color, texture, spatial features, depth-based segmentation, spectral feature-based segmentation, motion detection, and a user selection. 
     
     
         14 . The system of  claim 11 , wherein said mask is updated in real-time in response to any of: a location of said ROI changing in said scene, a new ROI being identified in said scene, a ROI no longer being considered an ROI, and a user input.