Mixed reality data collaboration

Abstract

Embodiments that relate to sharing mixed reality experiences among multiple display devices are disclosed. In one embodiment, a method includes receiving current versions of a plurality of data subtypes geo-located at a keyframe location. A world map data structure is updated to include the current versions, and a neighborhood request including the keyframe location is received from a display device. Based on the keyframe location, an identifier and current version indicator for each data subtype is provided to the device. A data request from the device for two or more of the data subtypes is received, and the two or more data subtypes are prioritized based on a priority hierarchy. Based on the prioritization, current versions of the data subtypes are sequentially provided to the device for augmenting an appearance of a mixed reality environment.

1 . A data collaborator for two or more display devices, the data collaborator comprising:
 a processor configured to:
 receive a plurality of data subtypes from a first display device, each of the data subtypes having a keyframe location and a current version; 
 update a data structure to include the current versions of the data subtypes at the keyframe location; 
 receive a neighborhood request including the keyframe location from a second display device; 
 in response to the neighborhood request and based on the keyframe location, provide to the second display device an identifier for each of the plurality of data subtypes and a current version indicator for each of the plurality of data subtypes; 
 receive a data request from the second display device for two or more of the plurality of data subtypes; 
 prioritize the two or more data subtypes based on a priority hierarchy; and 
 based on the prioritization, provide current versions of the two or more data subtypes to the second display device. 
   
     
     
         2 . The data collaborator of  claim 1 , wherein the data request includes the identifier for each of the two or more data subtypes and a non-current version indicator for each of the two or more data subtypes. 
     
     
         3 . The data collaborator of  claim 2 , wherein the processor is further configured to:
 compare the non-current version indicators to the current version indicators for the corresponding two or more data subtypes; and   based on the comparison, select the current versions of the two or more data subtypes for provision to the second display device.   
     
     
         4 . The data collaborator of  claim 1 , wherein the plurality of data subtypes are prioritized by the first display device based on the priority hierarchy, and the plurality of data subtypes are sequentially received by the processor from the first display device based on the priority hierarchy. 
     
     
         5 . The data collaborator of  claim 1 , wherein the plurality of data subtypes are prioritized by the second display device based on the priority hierarchy, and the data request from the second display device includes a prioritization of the two or more data subtypes based on the priority hierarchy. 
     
     
         6 . The data collaborator of  claim 1 , wherein prior to receiving the neighborhood request from the second display device and in response to receiving the plurality of data subtypes from the first display device, the processor is further configured to provide to the second display device the identifiers that identify each of the plurality of data subtypes and the current version of each of the plurality of data subtypes received from the first display device. 
     
     
         7 . The data collaborator of  claim 1 , wherein the processor is further configured to:
 determine that a previous version of one of the requested two or more data subtypes was previously received from the second display device; and   based on this determination, refrain from providing the previous version of the one of the requested data subtypes to the second display device.   
     
     
         8 . The data collaborator of  claim 1 , wherein the data subtypes are selected from the group consisting of pose link transforms, anchor transforms, tracking images, scene images, correspondences between two-dimensional image coordinates and three-dimensional locations, keyframe metadata, keyframe descriptors, three-dimensional keyframe relocalization points, quantizations of keyframe descriptors, and rig-to-rig transforms. 
     
     
         9 . The data collaborator of  claim 1 , wherein the priority hierarchy comprises a first priority that includes anchor transforms, a second priority that includes scene images, and a third priority that includes three-dimensional keyframe relocalization points. 
     
     
         10 . The data collaborator of  claim 1 , wherein the second display device is a head-mounted display device. 
     
     
         11 . A method, comprising:
 receiving a plurality of data subtypes from a first display device, each of the data subtypes having a keyframe location and a current version;   updating a data structure to include the current versions of the data subtypes at the keyframe location;   receiving a neighborhood request including the keyframe location from a second display device;   in response to the neighborhood request and based on the keyframe location, providing to the second display device an identifier for each of the plurality of data subtypes and a current version indicator for each of the plurality of data subtypes;   receiving a data request from the second display device for two or more of the plurality of data subtypes;   prioritizing the two or more data subtypes based on a priority hierarchy; and   based on the prioritization, providing current versions of the two or more data subtypes to the second display device.   
     
     
         12 . The method of  claim 11 , wherein the data request includes the identifier for each of the two or more data subtypes and a non-current version indicator for each of the two or more data subtypes. 
     
     
         13 . The method of  claim 12 , further comprising:
 comparing the non-current version indicators to the current version indicators for the corresponding two or more data subtypes; and   based on the comparison, selecting the current versions of the two or more data subtypes for provision to the second display device.   
     
     
         14 . The method of  claim 11 , wherein the plurality of data subtypes are prioritized by the first display device based on the priority hierarchy, and the method further comprises sequentially receiving the plurality of data subtypes from the first display device based on the priority hierarchy. 
     
     
         15 . The method of  claim 11 , wherein the plurality of data subtypes are prioritized by the second display device based on the priority hierarchy, and the data request from the second display device includes a prioritization of the two or more data subtypes based on the priority hierarchy. 
     
     
         16 . The method of  claim 11 , further comprising, prior to receiving the neighborhood request from the second display device and in response to receiving the plurality of data subtypes from the first display device, providing to the second display device the identifiers that identify each of the plurality of data subtypes and the current version of each of the plurality of data subtypes received from the first display device. 
     
     
         17 . The method of  claim 11 , further comprising:
 determining that a previous version of one of the requested two or more data subtypes was previously received from the second display device; and   based on this determination, refraining from providing the previous version of the one of the requested data subtypes to the second display device.   
     
     
         18 . The method of  claim 11 , wherein the data subtypes are selected from the group consisting of pose link transforms, anchor transforms, tracking images, scene images, correspondences between two-dimensional image coordinates and three-dimensional locations, keyframe metadata, keyframe descriptors, three-dimensional keyframe relocalization points, quantizations of keyframe descriptors, and rig-to-rig transforms. 
     
     
         19 . The method of  claim 11 , wherein the priority hierarchy comprises a first priority that includes anchor transforms, a second priority that includes scene images, and a third priority that includes three-dimensional keyframe relocalization points. 
     
     
         20 . A method, comprising:
 receiving a plurality of data subtypes from a first head-mounted display device, each of the data subtypes having a keyframe location and a current version;   updating a data structure to include the current versions of the data subtypes at the keyframe location;   receiving a neighborhood request including the keyframe location from a second head-mounted display device;   in response to the neighborhood request and based on the keyframe location, providing to the second display head-mounted device an identifier for each of the plurality of data subtypes and a current version indicator for each of the plurality of data subtypes;   receiving a data request from the second head-mounted display device for two or more of the plurality of data subtypes, the data request including an identifier for each of the two or more data subtypes and a non-current version indicator for each of the two or more data subtypes;   comparing the non-current version indicators to the current version indicators for the corresponding data subtypes;   based on the comparison, selecting the current versions of the two or more data subtypes for provision to the second head-mounted display device;   prioritizing the two or more data subtypes based on a priority hierarchy; and   based on the prioritization, providing the current versions of the two or more data subtypes to the second head-mounted display device.