Method, apparatus and system for reconstructing images of 3d surface

Abstract

The present disclosure discloses a method, an apparatus and a system for reconstructing an image of a three-dimensional surface. The method comprises the following steps of: constructing a three-dimensional model of the three-dimensional surface using X-ray imaging data obtained by imaging the three-dimensional surface with X-ray and extracting three-dimensional coordinate parameters of feature points; constructing one or more two-dimensional posture images of the three-dimensional surface using visible light imaging data obtained by imaging the three-dimensional surface with visible lights, and extracting two-dimensional coordinate parameters of feature points from each of the two-dimensional posture images; establishing a mapping relationship between the two-dimensional posture image and the three-dimensional model by matching the three-dimensional coordinate parameters and the two dimensional coordinate parameters of the feature points in each of the two-dimensional posture image; and filling the one or more two-dimensional posture image onto the three-dimensional model utilizing the mapping relationship established for each of the two dimensional posture images to form a reconstructed image of the three-dimensional surface, wherein the X-ray imaging data and the visible light imaging data of the three-dimensional surface along the same orientation are simultaneously captured.

1 . A method for reconstructing an image of a three-dimensional surface, comprising:
 a1) constructing a three-dimensional model of the three-dimensional surface using X-ray imaging data obtained by imaging the three-dimensional surface with X-ray and extracting three-dimensional coordinate parameters of feature points;   a2) constructing one or more two-dimensional posture images of the three-dimensional surface using visible light imaging data obtained by imaging the three-dimensional surface with visible lights, and extracting two-dimensional coordinate parameters of feature points from each of the two-dimensional posture images;   b) establishing a mapping relationship between the two-dimensional posture image and the three-dimensional model by matching the three-dimensional coordinate parameters and the two dimensional coordinate parameters of the feature points in each of the two-dimensional posture image; and   c) filling the one or more two-dimensional posture images onto the three-dimensional model utilizing the mapping relationship established for each of the two dimensional posture images to form a reconstructed image of the three-dimensional surface,   wherein the X-ray imaging data and the visible light imaging data of the three-dimensional surface along a same orientation are simultaneously captured.   
     
     
         2 . The method according to  claim 1 , wherein the constructing a three-dimensional model of the three-dimensional surface in the step of al) comprises:
 constructing a voxel model of the three-dimensional surface using the X-ray imaging data;   extracting a profile of the voxel model layer by layer to obtain a three-dimensional surface point cloud; and   constructing the three-dimensional model of the three-dimensional surface by establishing a connection relationship of the three-dimensional surface point cloud, wherein the three-dimensional model is a three-dimensional grid model.   
     
     
         3 . The method according to  claim 1 , wherein the visible light imaging data comprises a series of two-dimensional preliminary images generated in different orientations of the three-dimensional surface, and the constructing one or more two-dimensional posture images of the three-dimensional surface in the step of a2) comprises:
 determining postures corresponding to each of the two-dimensional preliminary image by extracting a relative position of the preliminary feature points; and   selecting the one or more two-dimensional posture images from the series of two-dimensional preliminary images based on the postures corresponding to each of the two-dimensional preliminary images.   
     
     
         4 . The method according to  claim 3 , wherein the preliminary feature point is selected from a set of feature points. 
     
     
         5 . The method according to  claim 1 , wherein in the step of b), determining the mapping matrix T between the two-dimensional posture image and the three-dimensional model by the following equation: 
       
         
           
             
               
                 
                   [ 
                   
                     
                       
                         
                           u 
                           1 
                         
                       
                       
                         
                           u 
                           2 
                         
                       
                       
                         
                             
                         
                       
                       
                         
                           u 
                           n 
                         
                       
                     
                     
                       
                         
                           v 
                           1 
                         
                       
                       
                         
                           v 
                           2 
                         
                       
                       
                         … 
                       
                       
                         
                           v 
                           n 
                         
                       
                     
                     
                       
                         1 
                       
                       
                         1 
                       
                       
                         
                             
                         
                       
                       
                         1 
                       
                     
                   
                   ] 
                 
                 = 
                 
                   T 
                    
                   
                     [ 
                     
                       
                         
                           
                             x 
                             1 
                           
                         
                         
                           
                             x 
                             2 
                           
                         
                         
                           
                               
                           
                         
                         
                           
                             x 
                             n 
                           
                         
                       
                       
                         
                           
                             y 
                             1 
                           
                         
                         
                           
                             y 
                             2 
                           
                         
                         
                           
                               
                           
                         
                         
                           
                             y 
                             n 
                           
                         
                       
                       
                         
                           
                             z 
                             1 
                           
                         
                         
                           
                             z 
                             2 
                           
                         
                         
                           … 
                         
                         
                           
                             z 
                             n 
                           
                         
                       
                       
                         
                           1 
                         
                         
                           1 
                         
                         
                           
                               
                           
                         
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
               , 
             
           
         
         wherein (u i , v i ) and (x i , y i , z i ) represent the two-dimensional coordinate parameter and the three-dimensional coordinate parameter of the i-th feature point among n feature points of the two-dimensional image, respectively, and i=1,2, . . . n. 
       
     
     
         6 . The method according to  claim 5 , wherein the mapping matrix is solved by least squares or singular value decomposition. 
     
     
         7 . The method according to  claim 1 , wherein the filling the one or more two-dimensional posture images onto the three-dimensional model in the step of c) comprises:
 dividing the three-dimensional model into corresponding one or more partitions according to the one or more two-dimensional posture images; and   filling the one or more two-dimensional posture images onto the corresponding one or more partitions to form the reconstructed three-dimensional surface image.   
     
     
         8 . An apparatus for reconstructing an image of a three-dimensional surface, comprising:
 a three-dimensional module constructing unit configured for constructing a three-dimensional model of the three-dimensional surface using X-ray imaging data obtained by imaging the three-dimensional surface with X-ray and extracting three-dimensional coordinate parameters of feature points;   a two-dimensional posture image constructing unit configured for constructing one or more two-dimensional posture images of the three-dimensional surface using visible light imaging data obtained by imaging the three-dimensional surface with visible lights, and extracting two-dimensional coordinate parameters of feature points from each of the two-dimensional posture images;   a mapping establishing unit configured for establishing a mapping relationship between the two-dimensional posture image and the three-dimensional model by matching the three-dimensional coordinate parameters and the two dimensional coordinate parameters of the feature points in each of the two-dimensional posture image; and   a reconstructing unit configured for filling the one or more two-dimensional posture images onto the three-dimensional model utilizing the mapping relationship established for each of the two dimensional posture images to form a reconstructed image of the three-dimensional surface,   wherein the X-ray imaging data and the visible light imaging data of the three-dimensional surface along a same orientation are simultaneously captured.   
     
     
         9 . The apparatus according to  claim 8 , wherein the three-dimensional module constructing unit is configured for:
 constructing a voxel model of the three-dimensional surface using the X-ray imaging data;   extracting a profile of the voxel model layer by layer to obtain a three-dimensional surface point cloud; and   constructing the three-dimensional model of the three-dimensional surface by establishing a connection relationship of the three-dimensional surface point cloud, wherein the three-dimensional model is a three-dimensional grid model.   
     
     
         10 . The apparatus according to  claim 8 , wherein the visible light imaging data comprises a series of two-dimensional preliminary images generated in different orientations of the three-dimensional surface, and the two-dimensional posture image constructing unit is configured for:
 determining postures corresponding to each of the two-dimensional preliminary image by extracting a relative position of the preliminary feature points; and   selecting the one or more two-dimensional posture images from the series of two-dimensional preliminary images based on the postures corresponding to each of the two-dimensional preliminary images.   
     
     
         11 . The apparatus according to  claim 10 , wherein the preliminary feature point is selected from a set of feature points. 
     
     
         12 . The apparatus according to  claim 8 , wherein the mapping establishing unit is configured for determining the mapping matrix T between the two-dimensional posture image and the three-dimensional model by the following equation: 
       
         
           
             
               
                 
                   [ 
                   
                     
                       
                         
                           u 
                           1 
                         
                       
                       
                         
                           u 
                           2 
                         
                       
                       
                         
                             
                         
                       
                       
                         
                           u 
                           n 
                         
                       
                     
                     
                       
                         
                           v 
                           1 
                         
                       
                       
                         
                           v 
                           2 
                         
                       
                       
                         … 
                       
                       
                         
                           v 
                           n 
                         
                       
                     
                     
                       
                         1 
                       
                       
                         1 
                       
                       
                         
                             
                         
                       
                       
                         1 
                       
                     
                   
                   ] 
                 
                 = 
                 
                   T 
                    
                   
                     [ 
                     
                       
                         
                           
                             x 
                             1 
                           
                         
                         
                           
                             x 
                             2 
                           
                         
                         
                           
                               
                           
                         
                         
                           
                             x 
                             n 
                           
                         
                       
                       
                         
                           
                             y 
                             1 
                           
                         
                         
                           
                             y 
                             2 
                           
                         
                         
                           
                               
                           
                         
                         
                           
                             y 
                             n 
                           
                         
                       
                       
                         
                           
                             z 
                             1 
                           
                         
                         
                           
                             z 
                             2 
                           
                         
                         
                           … 
                         
                         
                           
                             z 
                             n 
                           
                         
                       
                       
                         
                           1 
                         
                         
                           1 
                         
                         
                           
                               
                           
                         
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
               , 
             
           
         
         wherein (u i , v i ) and (x i , y i , z i ) represent the two-dimensional coordinate parameter and the three-dimensional coordinate parameter of the i-th feature point among n feature points of the two-dimensional image, respectively, and i=1,2, . . . n. 
       
     
     
         13 . The apparatus according to  claim 12 , wherein the mapping establishing unit is configured for solving the mapping matrix by least squares or singular value decomposition. 
     
     
         14 . The apparatus according to  claim 8 , wherein the reconstructing unit is configured for:
 dividing the three-dimensional model into corresponding one or more partitions according to the one or more two-dimensional posture images; and   filling the one or more two-dimensional posture images onto the corresponding one or more partitions to form the reconstructed three-dimensional surface image.   
     
     
         15 . A system for reconstructing an image of a three-dimensional surface, comprising:
 an X-ray imaging device configured to move around the three-dimensional surface for irradiating the three-dimensional surface with the X-ray to generate an X-ray image data;   a visible light imaging device located in the same orientation as the X-ray imaging device on the three-dimensional surface, and configured to move around the three-dimensional surface synchronously with the X-ray imaging device to generate a visible light imaging data; and   an apparatus for reconstructing an image of a three-dimensional surface according to  claim 8 .