Indexing backup data generated in backup operations

Abstract

In certain embodiments, a tiered storage system is disclosed that provides for failover protection during data backup operations. The system can provide for an index, or catalog, for identifying and enabling restoration of backup data located on a storage device. The system further maintains a set of transaction logs generated by media agent modules that identify metadata with respect to individual data chunks of a backup file on the storage device. A copy of the catalog and transaction logs can be stored at a location accessible by each of the media agent modules. In this manner, in case of a failure of one media agent module during backup, the transaction logs and existing catalog can be used by a second media agent module to resume the backup operation without requiring a restart of the backup process.

What is claimed is: 
     
         1 . A non-transitory computer-readable medium that stores instructions which, when executed by a first computing device comprising one or more hardware processors, cause the first computing device to:
 receive from a second computing device one or more first transaction logs,
 wherein the one or more first transaction logs are based on one or more of:
 (a) first backup data generated by the second computing device, wherein the first backup data comprises a plurality of data chunks, and 
 (b) storage of the first backup data as the plurality of data chunks in one or more second storage devices that are communicatively coupled to the second computing device, 
 
 wherein once a given data chunk from the plurality of data chunks is stored in the one or more second storage devices, the second computing device transmits a corresponding transaction log to the first computing device, and 
 wherein the second computing device comprises one or more hardware processors; 
   after the first backup data is generated, apply the one or more first transaction logs to an index; and   wherein the index is configured to enable restoring backup data generated by at least the second computing device, including restoring data chunks from the plurality of data chunks of the first backup data.   
     
     
         2 . The non-transitory computer-readable medium of  claim 1 , wherein the index is stored at least at:
 (i) the first computing device, and   (ii) one or more first storage devices that are associated with the first computing device and that are distinct from the one or more second storage devices where the first backup data is stored.   
     
     
         3 . The non-transitory computer-readable medium of  claim 1 , wherein the instructions further cause the first computing device to, after the one or more first transaction logs are applied to the index, generate a copy of the index. 
     
     
         4 . The non-transitory computer-readable medium of  claim 1 , wherein the instructions further cause the first computing device to:
 receive second transaction logs from a third computing device that generates third backup data;   apply the second transaction logs to the index; and   wherein the index is further configured to identify within the third backup data a third backup file.   
     
     
         5 . The non-transitory computer-readable medium of  claim 1 , wherein the instructions further cause the first computing device to apply, to the index, a plurality of transaction logs received from a plurality of computing devices configured to perform backup operations, including the one or more first transaction logs received from the second computing device; and
 wherein the index is a central source of information usable to identify and restore backed up data in a storage system, including the first backup data, and wherein the storage system comprises the index.   
     
     
         6 . The non-transitory computer-readable medium of  claim 1 , wherein the one or more first transaction logs comprise information for identifying backup files within the first backup data at the one or more second storage devices. 
     
     
         7 . The non-transitory computer-readable medium of  claim 1 , wherein the index comprises information, as received in the one or more first transaction logs, for identifying first backup files within the first backup data at the one or more second storage devices. 
     
     
         8 . The non-transitory computer-readable medium of  claim 1 , wherein the index is further configured to indicate storage locations of contents of the first backup data stored on the one or more second storage devices. 
     
     
         9 . The non-transitory computer-readable medium of  claim 1 , wherein the index is further configured to indicate storage locations of data chunks of the first backup data stored on the one or more second storage devices. 
     
     
         10 . The non-transitory computer-readable medium of  claim 1 , wherein the index is further configured to identify individual data objects within the first backup data stored on the one or more second storage devices. 
     
     
         11 . A method comprising:
 by a first computing device receiving one or more first transaction logs from a second computing device,
 wherein the first computing device comprises one or more hardware processors, 
 wherein the second computing device comprises one or more hardware processors; 
   wherein the one or more first transaction logs are based on one or more of:
 (a) by the second computing device, generating first backup data as a plurality of data chunks, and 
 (b) by the second computing device, storing the first backup data as the plurality of data chunks in one or more second storage devices that are communicatively coupled to the second computing device; 
 wherein once a given data chunk, from the plurality of data chunks, is stored in the one or more second storage devices, the second computing device transmits a corresponding transaction log to the first computing device; 
   after the first backup data is generated, applying the one or more first transaction logs to an index,
 wherein the index is stored at least at:
 (i) the first computing device, and 
 (ii) one or more first storage devices that are associated with the first computing device and that are distinct from the one or more second storage devices where the first backup data is stored; and 
 
   wherein the index is configured to enable restoring backup data generated by at least the second computing device, including restoring data chunks from the first backup data.   
     
     
         12 . The method of  claim 11  further comprising: by the first computing device, after the one or more first transaction logs are applied to the index, generating a copy of the index. 
     
     
         13 . The method of  claim 11 , wherein the index is a central source of information usable to identify and restore the first backup data in a storage system, wherein the storage system comprises the index. 
     
     
         14 . The method of  claim 11 , further comprising:
 by the first computing device, receiving second transaction logs from a third computing device that generates third backup data;   by the first computing device, applying the second transaction logs to the index; and   wherein the index is further configured to identify within the third backup data a third backup file.   
     
     
         15 . The method of  claim 11 , further comprising:
 by the first computing device, applying, to the index, a plurality of transaction logs received from a plurality of computing devices configured to perform backup operations, including the one or more first transaction logs received from the second computing device.   
     
     
         16 . The method of  claim 11 , wherein the one or more first transaction logs comprise information for identifying backup files within the first backup data at the one or more second storage devices that are communicatively coupled to the second computing device. 
     
     
         17 . The method of  claim 11 , wherein the index comprises information, as received in the one or more first transaction logs, for identifying first backup files within the first backup data at the one or more second storage devices that are communicatively coupled to the second computing device. 
     
     
         18 . The method of  claim 11 , wherein the index is further configured to indicate storage locations of contents of the first backup data stored in the one or more second storage devices that are communicatively coupled to the second computing device. 
     
     
         19 . The method of  claim 11 , wherein the index is further configured to indicate storage locations of data chunks of the first backup data stored in the one or more second storage devices that are communicatively coupled to the second computing device. 
     
     
         20 . The method of  claim 11 , wherein the index is further configured to identify individual data objects within the first backup data stored on the one or more second storage devices that are communicatively coupled to the second computing device.