Systems and methods for atomic publication of distributed writes to a distributed data warehouse

Abstract

Systems and methods for retrieving and using data stored in a data warehouse. The methods comprise performing the following operations by a computing device: receiving a query for information directed to a plurality of fact tables (each fact table comprising sets of source data that are arranged so as to be respectively associated with a plurality of first publication identifiers); obtaining second publication identifiers from a publication table, in response to the query; and obtaining at least one of the sets of source data from each fact table that is associated with a first publication identifier of the plurality of first publication identifiers which matches one of the second publication identifiers.

What is claimed is: 
     
       1. A method for retrieving and using data stored in a data warehouse, comprising:
 receiving, at a computing device, a query for information directed to a plurality of fact tables, wherein each said fact table comprises sets of source data that are arranged so as to be respectively associated with a plurality of first publication identifiers, and the source data is associated with one or more unique resource names that are different than the plurality of first publication identifiers; 
 obtaining, by the computing device, second publication identifiers from a publication table in response to the query, wherein the publication table is separate from the plurality of fact tables and the second publication identifiers are arranged in the publication table so as to be respectively associated with the unique resource names and version values, wherein each said version value references a version of at least a portion of the source data at a respective last modified time; and 
 obtaining, by the computing device, at least one of the sets of source data from each said fact table of the plurality of fact tables that is associated with a first publication identifier of the plurality of first publication identifiers which matches one of the second publication identifiers. 
 
     
     
       2. The method according to  claim 1 , further comprising performing a computation as per the query using the source data. 
     
     
       3. The method according to  claim 1 , further comprising communicating, to an external device from which the query was received, a query result comprising at least one of (i) the sets of source data obtained from said fact tables and (ii) results from processing the sets of source data obtained from said fact tables. 
     
     
       4. The method according to  claim 3 , further comprising performing, by the external device or another external device, an assessment of given data or software using the query result. 
     
     
       5. The method according to  claim 4 , further comprising updating the given data or software to address issues identified during the assessment. 
     
     
       6. The method according to  claim 4 , further comprising using the given data or software to control an autonomous or semi-autonomous vehicle based on results of assessment. 
     
     
       7. The method according to  claim 1 , further comprising updating the publication table to include the second publication identifiers and version values so as to be associated with the one or more unique resource names. 
     
     
       8. The method according to  claim 7 , further comprising determining whether each said version value is greater than a current version value for the respective source data that is stored in the publication table. 
     
     
       9. The method according to  claim 8 , wherein the publication table is updated to include the version value when a determination is made that the version value is greater than the current version value. 
     
     
       10. A system, comprising:
 a processor; 
 a non-transitory computer-readable storage medium comprising programming instructions that are configured to cause the processor to implement a method for retrieving and using data, wherein the programming instructions comprise instructions to:
 receive a query for information directed to a plurality of fact tables, wherein each said fact table comprises sets of source data that are arranged so as to be respectively associated with a plurality of first publication identifiers, and the source data is associated with one or more unique resource names that are different than the plurality of first publication identifiers; 
 obtain second publication identifiers from a publication table in response to the query, wherein the publication table is separate from the plurality of fact tables and the second publication identifiers are arranged in the publication table so as to be respectively associated with the unique resource names and version values, each said version value referencing a version of at least a portion of the source data at a respective last modified time; and 
 
 obtain at least one of the sets of source data from each said fact table of the plurality of fact tables that is associated with a first publication identifier of the plurality of first publication identifiers which matches one of the second publication identifiers. 
 
     
     
       11. The system according to  claim 10 , wherein the programming instructions further comprise instructions to perform a computation as per the query using the source data. 
     
     
       12. The system according to  claim 10 , wherein the programming instructions further comprise instructions to communicate, to an external device from which the query was received, a query result comprising at least one of (i) the sets of source data obtained from said fact tables and (ii) results from processing the sets of source data obtained from said fact tables. 
     
     
       13. The system according to  claim 12 , wherein the programming instructions further comprise instructions to perform an assessment of given data or software using the query result. 
     
     
       14. The system according to  claim 13 , wherein the programming instructions further comprise instructions to update the given data or software to address issues identified during the assessment. 
     
     
       15. The system according to  claim 13 , wherein the programming instructions further comprise instructions to use the given data or software to control an autonomous or semi-autonomous vehicle based on results of assessment. 
     
     
       16. The system according to  claim 10 , wherein the programming instructions further comprise instructions to update the publication table to include the second publication identifiers and version values so as to be associated with the one or more unique resource names. 
     
     
       17. The system according to  claim 16 , wherein the programming instructions further comprise instructions to determine whether each said version value is greater than a current version value for the respective source data that is stored in the publication table. 
     
     
       18. The system according to  claim 17 , wherein the publication table is updated to include the version value when a determination is made that the version value is greater than the current version value. 
     
     
       19. A non-transitory computer-readable medium that stores instructions that, when executed by at least one computing device, will cause the at least one computing device to perform operations comprising:
 receiving a query for information directed to a plurality of fact tables stored in a data warehouse, wherein each said fact table comprises sets of source data that are arranged so as to be respectively associated with a plurality of first publication identifiers, the source data is associated with one or more unique resource names that are different than the plurality of first publication identifiers; 
 obtaining, by the computing device, second publication identifiers from a publication table in response to the query, wherein the publication table is separate from the plurality of fact tables and the second publication identifiers are arranged in the publication table so as to be respectively associated with the unique resource names and version values, each said version value referencing a version of at least a portion of the source data at a respective last modified time; and 
 obtaining, by the computing device, at least one of the sets of source data from each said fact table of the plurality of fact tables that is associated with a first publication identifier of the plurality of first publication identifiers which matches one of the second publication identifiers.