Electronic system including image processing unit for reconstructing 3d surfaces and iterative triangulation method

Abstract

An electronic system includes a circuitry ( 910 ) configured to obtain a sequence of frames of an object ( 990 ) under different viewing angles at consecutive time instances. For a first time instance, the circuitry ( 910 ) generates a point cloud descriptive for an external surface of the object on basis of (i) a point cloud obtained for a second time instance preceding the first time instance and (ii) disparity information concerning a frame captured at the first time instance.

1 . An electronic system, comprising
 circuitry ( 910 ) configured to   obtain a sequence of frames of an object ( 990 ) under different viewing angles; and   generate, for a first time instance, a point cloud descriptive for an external surface of the object on basis of (i) a point cloud obtained for a second time instance preceding the first time instance and (ii) disparity information concerning a frame captured at the first time instance.   
     
     
         2 . The electronic system according to  claim 1 , wherein
 the disparity information concerns the frame captured at the first time instance and a frame captured at a preceding time instance.   
     
     
         3 . The electronic system according to  claim 1 , wherein
 the frame captured at the first time instance includes one or more sub-frames captured at the first time instance from different positions.   
     
     
         4 . The electronic system according to  claim 1 , further comprising
 a display unit ( 300 ) adapted to display a picture imaging a 3D representation of the dense point cloud obtained for the first time instance.   
     
     
         5 . The electronic system according to  claim 4 , wherein
 the circuitry ( 910 ) and the display unit ( 300 ) are adapted to update the picture imaged on the display unit ( 300 ) while capturing the sequence of frames.   
     
     
         6 . The electronic system according to  claim 1 , wherein
 the circuitry ( 910 ) is further configured to identify sparse correspondences between the first frame and a preceding frame for sparse salient image points in the first and the preceding frames.   
     
     
         7 . The electronic system according to  claim 6 , wherein
 the circuitry ( 910 ) is further configured to estimate dense correspondences by an iterative method using the preceding point cloud and the actual sparse correspondences as starting value.   
     
     
         8 . The electronic system according to  claim 7 , wherein
 the circuitry ( 910 ) is further configured to calculate an actual pose of the image capturing unit ( 100 ) relative to the object ( 990 ) on the basis of the dense correspondences dc(n) for the first frame fr(n) by using iterative methods.   
     
     
         9 . The electronic system according to  claim 1 , wherein
 the circuitry ( 910 ) is further configured to update depth information contained in the preceding point cloud by using information contained in the first frame.   
     
     
         10 . The electronic system according to  claim 9 , wherein
 the circuitry ( 910 ) uses a ray accumulator configured to temporarily store a ten coefficient vector.   
     
     
         11 . The electronic system according to  claim 1 , wherein the circuitry ( 910 ) comprises,
 an image capturing unit ( 100 ) adapted to obtain a sequence of frames of an object ( 990 ) under different viewing angles; and   a processing unit ( 200 ) adapted to generate, for a first time instance, a point cloud descriptive for an external surface of the object on basis of (i) a point cloud obtained for a second time instance preceding the first time instance and (ii) disparity information concerning a frame captured at the first time instance.   
     
     
         12 . A 3D reconstruction method, comprising
 capturing a sequence of frames of an object ( 990 ) under different viewing angles at consecutive time instances; and   generating, for a first time instance, a point cloud descriptive for an external surface of the object on basis of (i) a point cloud obtained for second time instance preceding the first time instance and (ii) disparity information concerning a frame captured at the first time instance.   
     
     
         13 . The 3D reconstruction method according to  claim 12 , wherein
 the disparity information concerns the frame captured at the first time instance and a frame captured at a preceding time instance.   
     
     
         14 . The 3D reconstruction method according to  claim 12 , wherein
 the frame captured at the first time instance includes one or more sub-frames captured at the first time instance from different positions.   
     
     
         15 . The 3D reconstruction method according to  claim 12 , further comprising
 displaying a picture imaging a 3D representation of the dense point cloud obtained for the first time instance.   
     
     
         16 . The 3D reconstruction method according to  claim 15 , wherein
 the displayed pictures are imaged on the display unit ( 300 ) while capturing the sequence of frames.   
     
     
         17 . An iterative triangulation method, comprising
 converting a 3D information for image points in frames captured from different positions into a ray representation containing direction information; and   updating the ray representation with each new captured frame.   
     
     
         18 . The iterative triangulation method according to  claim 17 , wherein
 the ray representation is formally a quadric.