Systems and methods for collection and consolidation of heterogeneous remote business data using dynamic data handling

Abstract

A method, system and computer program product for controlling collection of data from heterogeneous data sources is disclosed. Data is extracted in a data source independent manner using at least two types of control messages to hide implementation details for the heterogeneous data sources. The method receives from the heterogeneous data sources, a request for a first control message type, which defines data to be extracted from the heterogeneous data sources. Responsive to receiving the request, the method sends a first control messages type to the heterogeneous data sources. The method receives from the heterogeneous data sources, a second control message type that includes data extracted from the heterogeneous data sources that was defined in the first control message type. The method also stores, in a data store, the received data identified in the first control message type and that was extracted from the heterogeneous data sources.

What is claimed is: 
     
         1 . A method, performed by a processor, for controlling the collection of data from a plurality of heterogeneous data sources, wherein the data is extracted in a data source independent manner relative to the plurality of heterogeneous data sources using at least two types of control message to hide implementation details for each of the plurality of heterogeneous data sources, the method comprising:
 receiving from at least one of plurality of heterogeneous data sources, a request for a first type of control message, wherein the first type of control message is comprised of data collection information, wherein the data collection information is comprised of a collection definition that defines data to be extracted from the at least one of the plurality of heterogeneous data sources;   responsive to receiving the request, sending one or more of the first type of control messages to at least one of the plurality of heterogeneous data sources;   receiving from the at least one of the plurality of heterogeneous data sources, a second type of control message, wherein the second control message type is comprised of data extracted from the at least one of the plurality of heterogeneous data sources that was defined in the one or more of the first type of control messages; and   storing, in a data store, the received data that was identified in the first type of control message and that was extracted from the at least one of the plurality of heterogeneous data sources.   
     
     
         2 . The method of  claim 1 , wherein the first type of control message is further comprised of data collection scheduling information, wherein the data collection scheduling information is comprised of information indicating a schedule for collecting data from the at least one of the plurality of heterogeneous data sources, and the method further comprises:
 receiving from the at least one of the plurality of heterogeneous data sources, a second type of control message, wherein the second type of control message is comprised of data defined in the one or more of the first type of control messages that was extracted from the at least one of the plurality of heterogeneous data sources according to the data collection scheduling information.   
     
     
         3 . The method of  claim 1 , wherein the first type of control message is further comprised of updated data collection information, wherein updated data collection information is comprised of updated information defining data to be extracted from the at least one of the plurality of heterogeneous data sources. 
     
     
         4 . The method of  claim 1 , further comprising receiving from the at least one of plurality of heterogeneous data sources, a control message of a third type, wherein the control message of the third type is comprised of data collection status information, wherein the data collection status information is comprised of status information about the process of collecting data from the at least one of plurality of heterogeneous data sources; and
 storing, in a data store, the data collection status information.   
     
     
         5 . The method of  claim 4 , wherein the data collection status information is comprised of information about beginning data extraction, ending data extraction, beginning data encryption, ending data encryption, beginning encrypted data transmission or ending encrypted data transmission. 
     
     
         6 . The method of  claim 1 , further comprising:
 receiving from the at least one of plurality of heterogeneous data sources, a control message of a fourth type, wherein the control message of the fourth type is comprised of data collection error information, wherein the data collection error information is comprised of information about an error that occurred during the data collection process from the at least one of plurality of heterogeneous data sources; and   storing, in a data store, the data collection error information.   
     
     
         7 . The method of  claim 1 , wherein each of the plurality of heterogeneous data sources is comprised of data from a data source associated with one of a financial accounting application, a line of business application and a point of sale application. 
     
     
         8 . The method of  claim 7 , wherein the financial accounting application is QuickBooks. 
     
     
         9 . The method of  claim 1 , further comprising:
 consolidating the received data that was identified in the first type of control message and that was extracted from each of the plurality of heterogeneous data sources; and   storing the consolidated data in a data store; and   further transforming the consolidated data in the consolidated data store.   
     
     
         10 . The method of  claim 9 , further comprising:
 using one of a reporting system and business intelligence system to graphically display the consolidated data.   
     
     
         11 . The method of  claim 9 , further comprising:
 comparing data from one of the plurality of heterogeneous data sources with an average or peer group value generated from the consolidated data; and   displaying results of the data comparison.   
     
     
         12 . The method of  claim 1 , wherein the data is extracted via an application programming interface (API). 
     
     
         13 . A method, performed by a processor, for controlling the collection of data from a plurality of heterogeneous data sources, wherein the data is extracted in a data source independent manner relative to the plurality of heterogeneous data sources using at least two types of control message to hide implementation details for each of the plurality of heterogeneous data sources, the method comprising:
 sending, from at least one of plurality of heterogeneous data sources, a request for a first type of control message, wherein the first type of control message is comprised of data collection information, wherein the data collection information is comprised of information defining data to be collected from the at least one of the plurality of heterogeneous data sources;   receiving one or more of the first type of control messages; and   responsive to receiving the one or more first type of control messages, sending a second type of control message, wherein the second control message type is comprised of data defined in the one or more of the first type of control messages that was extracted from the at least one of the plurality of heterogeneous data sources.   
     
     
         14 . The method of  claim 13 , wherein the first type of control message is further comprised of data collection scheduling information, wherein the data collection scheduling information is comprised of information indicating a schedule for collecting data from the at least one of the plurality of heterogeneous data sources, and the method further comprises:
 responsive to receiving the one or more first type of control messages, sending a second type of control message, wherein the second control message type is comprised of data defined in the one or more of the first type of control messages that was extracted from the at least one of the plurality of heterogeneous data sources according to the data collection scheduling information.   
     
     
         15 . The method of  claim 13 , wherein the first type of control message is further comprised of updated data collection information, wherein updated data collection information is comprised of updated information defining data to be extracted from the at least one of the plurality of heterogeneous data sources. 
     
     
         16 . The method of  claim 13 , further comprising sending a control message of a third type, wherein the control message of the third type is comprised of data collection status information, wherein the data collection status information is comprised of status information about the process of collecting data from the at least one of plurality of heterogeneous data sources. 
     
     
         17 . The method of  claim 16 , wherein the data collection status information is comprised of information about beginning data extraction, ending data extraction, beginning data encryption, ending data encryption, beginning encrypted data transmission or ending encrypted data transmission. 
     
     
         18 . The method of  claim 13 , further comprising:
 sending a control message of a fourth type, wherein the control message of the fourth type is comprised of data collection error information, wherein the data collection error information is comprised of information about an error that occurred during the data collection process from the at least one of plurality of heterogeneous data sources.   
     
     
         19 . The method of  claim 13 , wherein each of the plurality of heterogeneous data sources is comprised of data from a data source associated with one of a financial accounting application, a line of business application and a point of sale application. 
     
     
         20 . The method of  claim 19 , wherein the financial accounting software application is QuickBooks.