Automatic dynamic reusable data recipes

Abstract

A data recipe may be automatically generated to provide requested information to a user. After the information is requested, one or more data sources may be interrogated to discover a plurality of data types of data stored in the data sources. The data types may be categorized to define a plurality of data recipe ingredients that are likely to be needed to provide the requested information. The data recipe ingredients may be compared with a reference data recipe. Based on the results of the comparison, a new data recipe that provides the requested information may be made by either modifying the reference data recipe or by proceeding independently of the reference data recipe. The new data recipe may, for example, calculate a key performance indicator used to measure organizational performance.

What is claimed is: 
     
         1 . A computer-implemented method for processing structured data, the method comprising:
 providing a library on a non-transitory storage medium, the library defining a plurality of reference data recipes, each reference data recipe comprising a data process configured to produce quantitative output data by use of one or more reference data components;   deriving live data components from data elements managed by a data source, the deriving comprising:
 determining attributes of respective data elements of a plurality of data elements managed by the data source, and 
 applying pre-determined categorization rules to the attributes determined for the respective data elements, wherein applying the pre-determined categorization rules to a live data component corresponding to a specified data element comprises categorizing the live data component as one of a measurement component and a dimension component; 
   selecting a reference data recipe from the library in response to a request, the selecting comprising:
 comparing reference data components of respective reference data recipes to live data components derived from the data elements managed by the one or more data sources, and 
 comparing data processes of the respective reference data recipes to the request; 
   producing a live data recipe, the producing comprising substituting reference data components of the selected reference data recipe with designated live data components; and   generating quantitative output data in response to the request, wherein generating the quantitative output data comprises applying a first data process to a measurement component of the live data recipe and a dimension component of the live data recipe, the measurement component comprising a first live data component corresponding to a first data element managed by the data source, and the dimension component comprising a second live data component corresponding to a second data element managed by the data source.   
     
     
         2 . The method of  claim 1 , wherein comparing the data processes of the respective reference data recipes to the request comprises comparing semantic processing metadata of the request to the respective reference data recipes. 
     
     
         3 . The method of  claim 2 , wherein the semantic processing metadata of the request comprises a key performance indicator. 
     
     
         4 . The method of  claim 1 , wherein producing the live data recipe comprises modifying a data process of the selected reference recipe to operate on the designated live data components. 
     
     
         5 . The method of  claim 4 , wherein generating the quantitative output data comprises applying the data process of the live data recipe to data elements corresponding to the designated live data components. 
     
     
         6 . The method of  claim 1 , wherein determining the attributes of the respective data elements comprises parsing a schema of the data source. 
     
     
         7 . The method of  claim 6 , wherein parsing the schema of the data source comprises parsing data elements stored within the data source into a NoSQL tree structure. 
     
     
         8 . The method of  claim 7 , wherein parsing the data elements into the NoSQL tree structure comprises selecting a schema for the NoSQL tree structure based on a structure of the data source. 
     
     
         9 . The method of  claim 1 , wherein the pre-determined categorization rules comprise:
 a first rule to categorize date data types as dimension components,   a second rule to categorize alpha-numeric data types as dimension components, and   a third rule to categorize numeric data types as measure components.   
     
     
         10 . The method of  claim 1 , further comprising categorizing the live data components by use of the pre-determined categorization rules, the categorizing further comprising inferring relationships among measure components and dimension components based on a structure of the one or more data sources. 
     
     
         11 . The method of  claim 1 , wherein comparing the reference data components of respective reference data recipes to the live data components comprises matching each of the live data components to a semantic layer. 
     
     
         12 . The method of  claim 11 , wherein matching each of the live data components to the semantic layer comprises:
 locating an identifying characteristic of the data element corresponding to each live data component;   locating, within the semantic layer, a phrase matching each identifying characteristic, wherein the identifying characteristic is selected from the group consisting of:
 a name of the data element; 
 a data type of the data element; 
 a data size of the data element; 
 a data structure of the data element; 
 metadata of the data element; 
 a sample data set related to the data element. 
   
     
     
         13 . The method of  claim 11 , wherein the semantic layer comprises a plurality of pairings, wherein each pairing comprises a phrase, a phrase mapping, and a confidence factor that indicates a likelihood that a data element is related to the phrase, wherein matching each of the live data components to the semantic layer comprises using the confidence factor of the pairing with a phrase that matches the data element of the live data component. 
     
     
         14 . The method of  claim 1 , wherein comparing reference data components of a reference data recipes to the live data components comprises using a data map to quantify relationships between the reference data components and data elements of the data source.