System and method of data transformation

Abstract

One or more processing devices receive data from a plurality of data sources, wherein the plurality of data sources have different data formats. The plurality of data sources are transformed into a universal data format, and using a second data source, the transformed data is enriched by applying one or more rules against the transformed data and the second data source, resulting in at least one of a first value substituted for a missing value, or the first value substituted for a second value, wherein the first value and the second value are obtained from different ones of the plurality of data sources and the second data source.

What is claimed is: 
     
         1 . A non-transitory machine-readable medium having executable instructions to cause one or more processing devices to:
 receive data from a plurality of data sources, wherein the plurality of data sources have different data formats;   transform the plurality of data sources into a universal data format; and   using a second data source, enrich the transformed data by applying one or more rules against the transformed data and the second data source, resulting in at least one of:
 a first value substituted for a missing value; or 
 the first value substituted for a second value, wherein the first value and the second value are obtained from different ones of the plurality of data sources and the second data source. 
   
     
     
         2 . The non-transitory machine-readable medium of  claim 1 , wherein the instructions to enrich the transformed data compute a confidence value for the substitution. 
     
     
         3 . The non-transitory machine-readable medium of  claim 2 , wherein the first value is determined to have a higher confidence level than the second value. 
     
     
         4 . The non-transitory machine-readable medium of  claim 2 , wherein a confidence level of the first value is associated with the confidence level of another value. 
     
     
         5 . The non-transitory machine-readable medium of  claim 1 , wherein the one or more rules comprise at least one of deductive, inductive, or abductive logic. 
     
     
         6 . The non-transitory machine-readable medium of  claim 1 , wherein a substitution is based at least in part on one of geographic proximity, gender proximity, network connectivity proximity, degrees of human relatedness, degrees of record relatedness, or temporal proximity. 
     
     
         7 . The non-transitory machine-readable medium of  claim 1 , wherein a mathematical rule or a relationship is used to determine the first value, and a dynamically determined rule is applied to assign a confidence level to the relationship. 
     
     
         8 . The non-transitory machine-readable medium of  claim 1 , wherein the instructions to enrich the transformed data populate a field that is missing data. 
     
     
         9 . The non-transitory machine-readable medium of  claim 1 , wherein the instructions to enrich the transformed data create another record. 
     
     
         10 . A method comprising:
 receiving data from a plurality of data sources, wherein the plurality of data sources have different data formats;   transforming the plurality of data sources into a universal data format; and   using a second data source, enriching the transformed data by applying one or more rules against the transformed data and the second data source, resulting in at least one of:
 a first value substituted for a missing value; or 
 the first value substituted for a second value, wherein the first value and the second value are obtained from different ones of the plurality of data sources and the second data source. 
   
     
     
         11 . The method of  claim 10 , wherein enriching the transformed data computes a confidence value for the substitution. 
     
     
         12 . The method of  claim 11 , wherein the first value is determined to have a higher confidence level than the second value 
     
     
         13 . The method of  claim 11 , wherein a confidence level of the first value is associated with the confidence level of another value 
     
     
         14 . The method of  claim 10 , wherein the one or more rules comprise at least one of deductive, inductive, or abductive logic. 
     
     
         15 . The method of  claim 10 , wherein substituting the first value is based at least in part on one of geographic proximity, gender proximity, network connectivity proximity, degrees of human relatedness, degrees of record relatedness, or temporal proximity. 
     
     
         16 . The method of  claim 10 , wherein a mathematical rule or a relationship is used to determine the first value, and a dynamically determined rule is applied to assign a confidence level to the relationship. 
     
     
         17 . The method of  claim 10 , wherein enriching the transformed data comprises populating a field that is missing data. 
     
     
         18 . The method of  claim 10 , wherein enriching the transformed data comprises creating another record. 
     
     
         19 . A system comprising:
 a memory; and   a processing device operatively connected to the memory, the processing device to:   receive data from a plurality of data sources, wherein the plurality of data sources have different data formats;   transform the plurality of data sources into a universal data format; and   using a second data source, enrich the transformed data by applying one or more rules against the transformed data and the second data source, resulting in at least one of:
 a first value substituted for a missing value; or 
 the first value substituted for a second value, wherein the first value and the second value are obtained from different ones of the plurality of data sources and the second data source. 
   
     
     
         20 . The system of  claim 19 , wherein to enrich the transformed data is further to compute a confidence value for the substitution.