Data load system with distributed data facility technology

Abstract

A source data file can be loaded into a database by extracting data from the source data file at a computing device, validating the data from the source data file at the computing device, transforming the data from the source data file at the computing device, requesting the data to be transferred to a first database at a first message processor module, delivering the data to the first database, and responding that the data has been transferred to the first database at the first message processor module to acknowledge receipt of the data in the database at the computing device. The data may be delivered to one or more databases using, for example, a plurality of distributed data facility (DDF) threads.

What is claimed is: 
     
         1 . A method comprising:
 receiving, at one or more mid-range servers, one or more data files;   extracting, by the one or more mid-range servers, data from the one or more data files;   generating a plurality of distributed data facility (DDF) threads between the one or more mid-range servers and a first database;   sending, by the one or more mid-range servers, a first portion of the extracted data to the first database using the generated plurality of DDF threads; and   sending a second portion of the extracted data to a message queue accessible to a second database.   
     
     
         2 . The method of  claim 1 , further comprising:
 determining whether the extracted data has an error,   wherein sending the first portion of the extracted data to the first database and sending the second portion of the extracted data to the message queue are performed in response to determining that the extracted data does not have the error.   
     
     
         3 . The method of  claim 1 , wherein the first portion of the extracted data comprises a first data chunk and the second portion of the extracted data comprises a second data chunk, the method further comprising:
 dividing, by the one or more mid-range servers, the extracted data into the first data chunk and the second data chunk,   wherein sending the first portion of the extracted data comprises sending the first data chunk, and wherein sending the second portion of the extracted data comprises sending the second data chunk.   
     
     
         4 . The method of  claim 1 , further comprising:
 receiving, from the second database, a request for the second portion of the extracted data; and   in response to receiving the request, sending, via the message queue, the second portion of the extracted data to the second database.   
     
     
         5 . The method of  claim 1 , further comprising:
 transforming the extracted data into XML data,   wherein sending the first portion of the extracted data comprises sending a first portion of the XML data, and wherein sending the second portion of the extracted data comprises sending a second portion of the XML data.   
     
     
         6 . The method of  claim 1 , wherein sending the first portion of the extracted data to the first database using the generated plurality of DDF threads comprises simultaneously sending a plurality of data from the first portion of the extracted data across different DDF threads of the plurality of DDF threads. 
     
     
         7 . The method of  claim 1 , further comprising:
 prior to sending the first portion of the extracted data to the first database, verifying and validating the first portion of the extracted data.   
     
     
         8 . One or more server comprising:
 a processor; and   memory storing computer-executable instructions that, when executed by the processor, cause the one or more server to:
 receive one or more data files; 
 extract data from the one or more data files; 
 generate a plurality of distributed data facility (DDF) threads between the one or more server and a first database; 
 send a first portion of the extracted data to the first database using the generated plurality of DDF threads; and 
 send a second portion of the extracted data to a message queue accessible to a second database. 
   
     
     
         9 . The one or more server of  claim 8 , wherein the memory stores computer-executable instructions that, when executed by the processor, cause the one or more server to:
 determine whether the extracted data has an error,   wherein sending the first portion of the extracted data to the first database and sending the second portion of the extracted data to the message queue are performed in response to determining that the extracted data does not have the error.   
     
     
         10 . The one or more server of  claim 8 , wherein the first portion of the extracted data comprises a first data chunk and the second portion of the extracted data comprises a second data chunk, and wherein the memory stores computer-executable instructions that, when executed by the processor, cause the one or more server to:
 divide the extracted data into the first data chunk and the second data chunk,   wherein sending the first portion of the extracted data comprises sending the first data chunk, and wherein sending the second portion of the extracted data comprises sending the second data chunk.   
     
     
         11 . The one or more server of  claim 8 , wherein the memory stores computer-executable instructions that, when executed by the processor, cause the one or more server to:
 receive, from the second database, a request for the second portion of the extracted data; and   in response to receiving the request, send, via the message queue, the second portion of the extracted data to the second database.   
     
     
         12 . The one or more server of  claim 8 , wherein the memory stores computer-executable instructions that, when executed by the processor, cause the one or more server to:
 transform the extracted data into XML data,   wherein sending the first portion of the extracted data comprises sending a first portion of the XML data, and wherein sending the second portion of the extracted data comprises sending a second portion of the XML data.   
     
     
         13 . The one or more server of  claim 8 , wherein sending the first portion of the extracted data to the first database using the generated plurality of DDF threads comprises simultaneously sending a plurality of data from the first portion of the extracted data across different DDF threads of the plurality of DDF threads. 
     
     
         14 . The one or more server of  claim 8 , wherein the memory stores computer-executable instructions that, when executed by the processor, cause the one or more server to:
 prior to sending the first portion of the extracted data to the first database, verify and validate the first portion of the extracted data.   
     
     
         15 . A non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more server, cause the one or more server to:
 receive one or more data files;   extract data from the one or more data files;   generate a plurality of distributed data facility (DDF) threads between the one or more server and a first database;   send a first portion of the extracted data to the first database using the generated plurality of DDF threads; and   send a second portion of the extracted data to a message queue accessible to a second database.   
     
     
         16 . The non-transitory computer-readable media of  claim 15 , storing computer-readable instructions that, when executed by the one or more server, cause the one or more server to:
 determine whether the extracted data has an error,   wherein sending the first portion of the extracted data to the first database and sending the second portion of the extracted data to the message queue are performed in response to determining that the extracted data does not have the error.   
     
     
         17 . The non-transitory computer-readable media of  claim 15 , wherein the first portion of the extracted data comprises a first data chunk and the second portion of the extracted data comprises a second data chunk, and wherein the non-transitory computer-readable media stores computer-readable instructions that, when executed by the one or more server, cause the one or more server to:
 divide the extracted data into the first data chunk and the second data chunk,   wherein sending the first portion of the extracted data comprises sending the first data chunk, and wherein sending the second portion of the extracted data comprises sending the second data chunk.   
     
     
         18 . The non-transitory computer-readable media of  claim 15 , storing computer-readable instructions that, when executed by the one or more server, cause the one or more server to:
 receive, from the second database, a request for the second portion of the extracted data; and   in response to receiving the request, send, via the message queue, the second portion of the extracted data to the second database.   
     
     
         19 . The non-transitory computer-readable media of  claim 15 , storing computer-readable instructions that, when executed by the one or more server, cause the one or more server to:
 transform the extracted data into XML data,   wherein sending the first portion of the extracted data comprises sending a first portion of the XML data, and wherein sending the second portion of the extracted data comprises sending a second portion of the XML data.   
     
     
         20 . The non-transitory computer-readable media of  claim 15 , wherein sending the first portion of the extracted data to the first database using the generated plurality of DDF threads comprises simultaneously sending a plurality of data from the first portion of the extracted data across different DDF threads of the plurality of DDF threads.