Method and system for unsynchronized structured lighting

Abstract

A system and method to capture the surface geometry a three-dimensional object in a scene using unsynchronized structured lighting is disclosed. The method and system includes a pattern projector configured and arranged to project a sequence of image patterns onto the scene at a pattern frame rate, a camera configured and arranged to capture a sequence of unsynchronized image patterns of the scene at an image capture rate, and a processor configured and arranged to synthesize a sequence of synchronized image frames from the unsynchronized image patterns of the scene. Each of the synchronized image frames corresponds to one image pattern of the sequence of image patterns.

1 . A system to capture the surface geometry a three-dimensional object in a scene, comprising:
 a pattern projector configured and arranged to project a sequence of image patterns onto the scene at a pattern frame rate;   a camera configured and arranged to capture a sequence of unsynchronized image patterns of the scene at an image capture rate; and   a processor configured and arranged to synthesize a sequence of synchronized image frames from the unsynchronized image patterns of the scene, each of the synchronized image frames corresponding to one image pattern of the sequence of image patterns.   
     
     
         2 . The system of  claim 1 , wherein the sequence of image patterns comprises binary patterns. 
     
     
         3 . The system of  claim 1 , wherein the number of image patterns in the sequence of image patterns is less than or equal to the number of unsynchronized image frames in the sequence of unsynchronized image frames. 
     
     
         4 . The system of  claim 1 , wherein the camera has a rolling shutter operation. 
     
     
         5 . The system of  claim 1 , wherein the camera has a global shutter operation. 
     
     
         6 . The system of  claim 1 , wherein the image capture rate of the camera is equal to the pattern frame rate of the projector. 
     
     
         7 . The system of  claim 1 , wherein the image capture rate of the camera is greater than the pattern frame rate of the projector. 
     
     
         8 . A method of capturing the surface geometry of a three-dimensional object in a scene, comprising:
 projecting a sequence of image pattern into the scene at a pattern frame rate;   capturing a sequence of unsynchronized image patterns of the scene at an image capture rate; and   synthesizing a sequence of synchronized image frames from the from the unsynchronized image patterns of the scene, each of the synchronized image frames corresponding to one image pattern of the sequence of image patterns.   
     
     
         9 . The method of  claim 8 , wherein the step of projecting a sequence of image patterns comprises projecting a sequence of binary patterns. 
     
     
         10 . The method of  claim 8 , wherein the number of image patterns projected in the sequence of image patterns is less than or equal to the number of unsynchronized image frames in the sequence of unsynchronized image frames. 
     
     
         11 . The method of  claim 8 , further comprising selecting a pattern frame rate. 
     
     
         12 . The method of  claim 8 , further comprising selecting an image capture rate. 
     
     
         13 . The method of  claim 8 , wherein the image capture rate is equal to the pattern frame rate. 
     
     
         14 . The method of  claim 8 , wherein the image capture rate is greater than the pattern frame rate. 
     
     
         15 . The method of  claim 8 , further comprising decoding the sequence of synchronized image frames. 
     
     
         16 . The method of  claim 15 , further comprising applying three-dimensional triangulation to the sequence of synchronized image frames. 
     
     
         17 . The method of  claim 16 , further comprising applying geometric processing to the sequence of synchronized image frames. 
     
     
         18 . A method to synthesize a sequence of synchronized image frames synchronized from a sequence of unsynchronized image frames; each of the synchronized image frames corresponding to one of a sequence of image patterns; the synchronized image frames and unsynchronized image frames being image frames of the same width and height; the image frames comprising a plurality of common pixels; the method comprising the steps of:
 a) partitioning the plurality of common pixels into a pixel partition; the pixel partition comprising a plurality of pixel sets; the pixel sets being disjoint; each of the plurality of common pixels being a member of one of the pixel sets;   b) selecting a pixel set;   c) building a pixel set measurement matrix;   d) estimating a pixel set projection matrix; the pixel set projection matrix projecting a measurement space vector onto the column space of the pixel set measurement matrix;   e) estimating a pixel set system matrix; the system matrix being parameterized by a set of system matrix parameters;   f) estimating a pixel set synchronized matrix as a function of the pixel set measurement matrix and the pixel set system matrix;   g) repeating steps b) to f) until all the pixels sets have been selected; and   h) constructing a sequence of synchronized image frames from the pixel set synchronized matrices.   
     
     
         19 . A method as in  claim 18 , where the pixel partition comprises a single pixel set, and the single pixel set contains all the pixels of the image frames. 
     
     
         20 . A method as in  claim 18 , where the number of pixel sets in the pixel partition is equal to the height of the image frames, and each row of the image frames is a pixel set. 
     
     
         21 . A method as in  claim 18 , where the pixel set measurement matrix is modeled as the product of the system matrix times the pixel set synchronized matrix, and the step of estimating the pixel set synchronized matrices reduces to the solution of a linear least-squares problem. 
     
     
         22 . A method as in  claim 18 , where the system matrix is parameterized by an image frame period parameter, an integration time parameter, and a first pattern delay parameter.