Generating and fusing reconstructions using overlapping map segments

Abstract

Examples disclosed herein may involve a computing system that is operable to (i) receive one or more images related to a global map having a plurality of overlapping map segments, wherein each of the plurality of overlapping map segments overlaps with one or more neighboring map segments, (ii) based on a preliminary location determination for the one or more images, identify at least a first overlapping map segment of the plurality of overlapping map segments that corresponds to the one or more images, (iii) generate a reconstruction of the first identified overlapping map segment based on the one or more images, and (iv) fuse the generated reconstruction of the first identified overlapping map segment together with the first identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the generated reconstruction and the first identified overlapping map segment's one or more neighboring map segments.

We claim: 
     
       1. A computer-implemented method comprising:
 receiving one or more images related to a global map having a plurality of overlapping map segments, wherein each of the plurality of overlapping map segments corresponds to an area of the global map, and wherein each of the plurality of overlapping map segments overlaps with one or more neighboring map segments; 
 based on a preliminary location determination for the one or more images, identifying at least a first overlapping map segment of the plurality of overlapping map segments that corresponds to the one or more images; 
 generating a reconstruction of the first identified overlapping map segment based on the one or more images as well as visual features and pose data generated from the global map; and 
 fusing the generated reconstruction of the first identified overlapping map segment together with the first identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the generated reconstruction and the first identified overlapping map segment's one or more neighboring map segments. 
 
     
     
       2. The computer-implemented method of  claim 1 , further comprising:
 determining that the generated reconstruction meets a quality threshold for the global map; and 
 based on the determining, updating the global map with the reconstruction. 
 
     
     
       3. The computer-implemented method of  claim 2 , wherein the quality threshold for the global map is determined based on an existing quality of the global map. 
     
     
       4. The computer-implemented method of  claim 1 , wherein generating the reconstruction of the first identified overlapping map segment based on the one or more images as well as visual features and pose data generated from the global map comprises:
 performing optimization of the pose data using one or more reconstruction constraints comprising one or more of (i) geographical radius, (ii) image size, (iii) number of images in each segment, or (iv) number of common data points. 
 
     
     
       5. The computer-implemented method of  claim 4 , wherein the common data points comprise one or more of (i) one or more landmarks or (ii) one or more extracted visual features. 
     
     
       6. The computer-implemented method of  claim 4 , wherein fusing the generated reconstruction together with the first identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the generated reconstruction and the first identified overlapping map segment's one or more neighboring map segments comprises:
 aligning one or more common data points in the first identified overlapping map segment. 
 
     
     
       7. The computer-implemented method of  claim 1 , wherein the global map is generated using one or more images together with one or more of (i) global positioning system (GPS) data, (ii) inertial measurement unit sensor data, or (iii) accelerometer data. 
     
     
       8. The computer-implemented method of  claim 1 , wherein the one or more images comprise two or more images that are gathered at various times and in varying environmental conditions. 
     
     
       9. The computer-implemented method of  claim 1 , wherein the reconstruction of the first identified overlapping map segment comprises a visualization of the area in a 3D space. 
     
     
       10. The computer-implemented method of  claim 1 , wherein the first identified overlapping map segment has at least a 50% overlap with the first identified overlapping map segment's one or more neighboring map segments. 
     
     
       11. A computing system comprising:
 at least one processor; 
 a non-transitory computer-readable medium; and 
 program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is configured to perform functions comprising:
 receiving one or more images related to a global map having a plurality of overlapping map segments, wherein each of the plurality of overlapping map segments corresponds to an area of the global map, and wherein each of the plurality of overlapping map segments overlaps with one or more neighboring map segments; 
 based on visual data of the one or more images, identifying at least a first overlapping map segment of the plurality of overlapping map segments that corresponds to the one or more images; 
 generating a reconstruction of the first identified overlapping map segment based on the one or more images as well as visual features and pose data generated from the global map; and 
 fusing the generated reconstruction of the first identified overlapping map segment together with the first identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the generated reconstruction and the first identified overlapping map segment's one or more neighboring map segments. 
 
 
     
     
       12. The computing system of  claim 11 , wherein generating a reconstruction of the first identified overlapping map segment based on the one or more images as well as visual features and pose data generated from the global map comprises:
 performing optimization of a pose data using one or more reconstruction constraints comprising one or more of (i) geographical radius, (ii) image size, (iii) number of images in each segment, (iv) or number of common data points. 
 
     
     
       13. The computing system of  claim 12 , wherein the common data points comprise one or more of (i) one or more landmarks or (ii) one or more extracted visual features. 
     
     
       14. The computing system of  claim 11 , wherein the first identified overlapping map segment has at least a 50% overlap with the first identified overlapping map segment's one or more neighboring map segments. 
     
     
       15. A non-transitory computer-readable medium comprising program instructions stored thereon that, when executed by at least one processor of a computing system, cause the computing system to perform functions comprising:
 receiving one or more images related to a global map having a plurality of overlapping map segments, wherein each of the plurality of overlapping map segments corresponds to an area of the global map, and wherein each of the plurality of overlapping map segments overlaps with one or more neighboring map segments; 
 based on visual data of the one or more images, identifying at least a first overlapping map segment of the plurality of overlapping map segments that corresponds to the one or more images; 
 generating a reconstruction of the first identified overlapping map segment based on the one or more images as well as visual features and pose data generated from the global map; and 
 fusing the generated reconstruction of the first identified overlapping map segments together with the first identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the generated reconstruction and the first identified overlapping map segment's one or more neighboring map segments. 
 
     
     
       16. The non-transitory computer-readable medium of  claim 15 , wherein generating a reconstruction the first identified overlapping map segment based on the one or more images as well as visual features and pose data generated from the global map comprises:
 performing optimization of a pose data using one or more reconstruction constraints comprising one or more of (i) geographical radius, (ii) image size, (iii) number of images in each segment, or (iv) number of common data points. 
 
     
     
       17. The non-transitory computer-readable medium of  claim 15 , wherein the first identified overlapping map segment has at least a 50% overlap with the first identified overlapping map segment's one or more neighboring map segments. 
     
     
       18. The computer-implemented method of  claim 1 , wherein the generated reconstruction of the first identified overlapping map segment comprises a first generated reconstruction, the method further comprising:
 generating a second reconstruction of a second identified overlapping map segment of the plurality of overlapping map segments; and 
 fusing the second reconstruction of the second identified overlapping map segment together with the second identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the second generated reconstruction and the second identified overlapping map segment's one or more neighboring map segments. 
 
     
     
       19. The computer-implemented method of  claim 1 , wherein at least a subset of the plurality of overlapping map segments are processed using a parallel processing arrangement. 
     
     
       20. The computer-implemented method of  claim 1 , wherein at least a subset of the plurality of overlapping map segments are processed substantially simultaneously. 
     
     
       21. The computer-implemented method of  claim 1 , further comprising:
 pre-determining one or more requirements for the reconstruction of the first identified overlapping map segment.