Systems and methods for bi-directional database application programming interface, extract transform and load system, and user computing device

Abstract

Systems and methods are provided for improved integration between database systems. Data from multiple disparate computing systems is transmitted via bi-directional communication interfaces. The data is stored in its original form from respective data sources and transformed in stream and/or batch processes into one or more predefined formats. Individual transformations are stored and/or recorded. The transformed and/or integrated data is provided to one or more computing devices via the communication interfaces.

What is claimed is: 
     
         1 . A method for transforming data in a data exchange, the method comprising:
 receiving, via an application programming interface, a first message comprising first data from a first provider entity;   determining a first data format of the first data;   determining that the first data is from the first provider entity;   causing a first transformation of the first data in the first data format to first transformed data in a first output format according to a first instruction set, wherein causing the first transformation further comprises:
 parsing the first data according to the first data format; 
 determining first transformation logic associated with the first provider entity; and 
 formatting, according to the first transformation logic, the first data into the first transformed data; 
   transmitting first outbound data associated with the first transformed data to a first destination entity;   receiving, via the application programming interface, a second message comprising second data from a second provider entity;   determining a second data format of the second data;   determining that the second data is from the second provider entity;   causing a second transformation of the second data in the second data format to second transformed data in a second output format according to a second instruction set, wherein causing the second transformation further comprises:
 parsing the second data according to the second data format; 
 determining second transformation logic associated with the second provider entity, wherein the second transformation logic is different from the first transformation logic; and 
 formatting, according to the second transformation logic, the second data into the second transformed data; and 
   transmitting second outbound data associated with the second transformed data to a second destination entity.   
     
     
         2 . The method of  claim 1 , wherein formatting the first data comprises applying a first transformation step, further comprising:
 storing, in a data store, the first transformation step.   
     
     
         3 . The method of  claim 1 , further comprising:
 storing, in a search engine data store, the first transformed data.   
     
     
         4 . The method of  claim 3 , further comprising:
 receiving, via the application programming interface, a request from a computing device, the request comprising a search parameter;   searching the search engine data store with the search parameter, wherein the search engine data store returns the first transformed data; and   transmitting the first transformed data to the computing device.   
     
     
         5 . The method of  claim 1 , further comprising:
 storing, in a data store, the first data.   
     
     
         6 . The method of  claim 5 , further comprising:
 adding a new instruction set;   generating third transformed data from the first data according to the new instruction set; and   storing, in the data store, the third transformed data.   
     
     
         7 . A system comprising:
 a data storage medium; and   one or more computer hardware processors in communication with the data storage medium, wherein the one or more computer hardware processors are configured to execute computer-executable instructions to at least:
 receive, via an application programming interface, a first message comprising first data from a first provider entity; 
 determine a first data format of the first data; 
 determine that the first data is from the first provider entity; 
 cause a first transformation of the first data in the first data format to first transformed data in a first output format according to a first instruction set, wherein causing the first transformation further comprises:
 parsing the first data according to the first data format; 
 determining first transformation logic associated with the first provider entity; and 
 formatting, according to the first transformation logic, the first data into the first transformed data; 
 
 transmit first outbound data associated with the first transformed data to a first destination entity; 
 receive, via the application programming interface, a second message comprising second data from a second provider entity; 
 determine a second data format of the second data; 
 determine that the second data is from the second provider entity; 
 cause a second transformation of the second data in the second data format to second transformed data in a second output format according to a second instruction set, wherein causing the second transformation further comprises:
 parsing the second data according to the second data format; 
 determining second transformation logic associated with the second provider entity, wherein the second transformation logic is different from the first transformation logic; and 
 formatting, according to the second transformation logic, the second data into the second transformed data; and 
 
 transmit second outbound data associated with the second transformed data to a second destination entity. 
   
     
     
         8 . The system of  claim 7 , wherein formatting the first data comprises applying a first transformation step, wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 store, in a data store, the first transformation step.   
     
     
         9 . The system of  claim 7 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 store, in a search engine data store, the first transformed data;   receive, via the application programming interface, a request from a computing device, the request comprising a search parameter;   search the search engine data store with the search parameter, wherein the search engine data store returns the first transformed data; and   transmit the first transformed data to the computing device.   
     
     
         10 . The system of  claim 7 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 store, in a data store, the first data;   add a new instruction set;   generate third transformed data from the first data according to the new instruction set; and   store, in the data store, the third transformed data.   
     
     
         11 . The system of  claim 7 , wherein formatting the first data into the first transformed data further comprises:
 omitting a portion of the first data from the first transformed data.   
     
     
         12 . The system of  claim 7 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 receive, via the application programming interface, a third message comprising third data from a third provider entity;   determine that the third data is incomplete; and   add the third data to a queue.   
     
     
         13 . The system of  claim 12 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 receive, via the application programming interface, a fourth message comprising fourth data from the third provider entity;   determine that the fourth data completes the third data from the queue;   generate, from the fourth data and the third data from the queue, fifth data according to a third instruction set; and   store the fifth data in a data store.   
     
     
         14 . A system comprising:
 a data storage medium; and   one or more computer hardware processors in communication with the data storage medium, wherein the one or more computer hardware processors are configured to execute computer-executable instructions to at least:
 receive, via an application programming interface, a first message comprising first data from a first provider entity; 
 determine a data format of the first data; 
 determine that the first data is from the first provider entity; 
 cause a transformation of the first data in the data format to first transformed data in an output format according to a first instruction set, wherein causing the transformation further comprises:
 parsing the first data according to the data format; 
 determining transformation logic associated with the first provider entity; and 
 formatting, according to the transformation logic, the first data into the first transformed data; and 
 
 transmit first outbound data associated with the first transformed data to a destination entity. 
   
     
     
         15 . The system of  claim 14 , wherein formatting the first data comprises applying a first transformation step, wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 store, in a data store, the first transformation step.   
     
     
         16 . The system of  claim 14 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 store, in a search engine data store, the first transformed data;   receive, via the application programming interface, a request from a computing device, the request comprising a search parameter;   search the search engine data store with the search parameter, wherein the search engine data store returns the first transformed data; and   transmit the first transformed data to the computing device.   
     
     
         17 . The system of  claim 14 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 store, in a data store, the first data;   add a new instruction set;   generate second transformed data from the first data according to the new instruction set; and   store, in the data store, the second transformed data.   
     
     
         18 . The system of  claim 14 , wherein formatting the first data into the first transformed data further comprises:
 omitting a portion of the first data from the first transformed data.   
     
     
         19 . The system of  claim 14 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 receive, via the application programming interface, a second message comprising second data from a second provider entity;   determine that the second data is incomplete; and   add the second data to a queue.   
     
     
         20 . The system of  claim 19 , wherein the one or more computer hardware processors are configured to execute further computer-executable instructions to at least:
 receive, via the application programming interface, a third message comprising third data from second third provider entity;   determine that the third data completes the second data from the queue;   generate, from the third data and the second data from the queue, fourth data according to a second instruction set; and   store the fourth data in a data store.