Concurrent data processing using snapshot technology

Abstract

A method for processing data using snapshots is provided. The method comprises generating a primary snapshot of data stored in a first storage medium, wherein the primary snapshot is a copy of the data stored in the first storage medium at a point in time; generating one or more secondary snapshots of the primary snapshot, wherein the secondary snapshots are copies of the data stored on the first storage medium at the point in time; and extracting the data stored in the first storage medium at the point in time by accessing the data included in the secondary snapshots, wherein multiple processing units concurrently transform the extracted data and load the transformed data onto a second storage medium.

1 . A method for processing data using snapshots, the method comprising:
 generating a primary snapshot of data stored in a first storage medium, wherein the primary snapshot is a copy of the data stored in the first storage medium at a point in time;   generating one or more secondary snapshots of the primary snapshot, wherein the secondary snapshots are copies of the data stored on the first storage medium at the point in time; and   extracting the data stored in the first storage medium at the point in time by accessing the data included in the secondary snapshots, wherein multiple processing units concurrently transform the extracted data and load the transformed data onto a second storage medium.   
     
     
         2 . The method of  claim 1 , wherein the data stored in the first storage medium is modified after the point in time without disrupting processing of the data. 
     
     
         3 . The method of  claim 1 , wherein the primary snapshot is generated by way of a data recovery scheme. 
     
     
         4 . The method of  claim 3 , wherein the secondary snapshots are generated according to a data recovery scheme that recovers the data from the primary snapshot as if the data is being recovered after a power failure. 
     
     
         5 . The method of  claim 1 , wherein the multiple processing units extract the data from the secondary snapshots in parallel. 
     
     
         6 . The method of  claim 1 , wherein the multiple processing units transform the data in parallel. 
     
     
         7 . The method of  claim 1 , wherein the multiple processing units load the data in a second storage medium in parallel. 
     
     
         8 . The method of  claim 1 , further comprising deleting the primary and secondary snapshots after processing the data. 
     
     
         9 . The method of  claim 1 , wherein a data management system supports parallel access to the secondary snapshots. 
     
     
         10 . The method of  claim 1 , wherein the data management system communicates with an application programming interface (API) for each of the secondary snapshots to allow the data processing units extract data from the secondary snapshots in parallel, wherein the data management system distribute to each API a proportional distribution of the data that is to be processed by a processing unit corresponding to each API.