Transaction log index generation in an enterprise backup system

Abstract

Certain embodiments disclosed herein reduce or eliminate a communication bottleneck at the storage manager by reducing communication with the storage manager while maintaining functionality of an information management system. In some implementations, operations performed as part of a backup process may be stored in transaction logs. These transaction logs may include information about a transaction performed between the client computing system and the network storage that hosts the backup of the client computing system. The transaction logs may be provided to a secondary storage system that can be used to form a backup index. The backup index may be used to facilitate accessing the data stored at the network storage. Advantageously, generating the transaction logs and separating the generation of the backup index from the backup process can reduce resource usage during performance of the backup and speed up the backup process while further reducing interaction with the storage manager.

1 . (canceled) 
     
     
         2 . A computer-implemented method, comprising:
 as implemented by one or more hardware processors of an information management system, the one or more hardware processors configured with specific computer-executable instructions,
 accessing first metadata associated with a first data protection copy of data associated with a first client computing system of an information management system, the first data protection copy stored on one or more storage devices of a network storage system as part of a data protection operation; 
 providing a first transaction log file generated based on the first metadata to a first media agent of a plurality of media agents of the information management system, the first media agent associated with the first client computing system; and 
 replicating the first transaction log file to at least a second media agent of the plurality of media agents, wherein the second media agent is selected from the plurality of media agents based at least in part on an identity of the first client computing system or a job identifier associated with the first transaction log file. 
   
     
     
         3 . The computer-implemented method of  claim 2 , further comprising updating an index based on a set of transaction log files, the set of transaction log files including the first transaction log file. 
     
     
         4 . The computer-implemented method of  claim 3 , further comprising deleting the set of transaction log files 
     
     
         5 . The computer-implemented method of  claim 4 , further comprising, prior to said deleting the set of transaction log files, replicating the index within a distributed index storage system. 
     
     
         6 . The computer-implemented method of  claim 3 , further comprising determining occurrence of a trigger to update the index, wherein the trigger comprises one or more of: an elapse of a period of time, an indication of completion of a backup job, a number of transaction log files added to the set of transaction log files satisfying a threshold; or receipt of a command to update the index. 
     
     
         7 . The computer-implemented method of  claim 3 , wherein said updating the index comprises generating a new index. 
     
     
         8 . The computer-implemented method of  claim 2 , wherein said generating the first transaction log file is performed by a third media agent at the first client computing system that is separate from the first media agent. 
     
     
         9 . The computer-implemented method of  claim 8 , wherein a second transaction log file associated with a second data protection operation associated with a second client computing system is provided to the second media agent, and wherein the second transaction log file is replicated to the first media agent. 
     
     
         10 . The computer-implemented method of  claim 2 , wherein the first data protection operation performed using a cached job identifier associated with the first client computing system. 
     
     
         11 . A system comprising:
 one or more computing systems of an information management system, the one or more computing systems comprising one or more hardware processors configured to:
 at a first media agent of a plurality of media agents of the information management system and associated with a first client computing system of the information management system, receive a first transaction log file generated based on first metadata associated with the first client computing system, the first metadata associated with a first data protection copy of data associated with a first client computing system of the information management system, the first data protection copy stored on one or more storage devices of a network storage system as part of a data protection operation; and 
 replicate the first transaction log file to at least a second media agent of the plurality of media agents of the information management system, wherein the second media agent is selected from the plurality of media agents based at least in part on an identity of the first client computing system or a job identifier associated with the first transaction log file. 
   
     
     
         12 . The system of  claim 11 , wherein the one or more hardware processors are further configured to update an index at the network storage system based on a set of transaction log files including the first transaction log file. 
     
     
         13 . The system of  claim 12 , wherein the one or more hardware processors are further configured to determine the set of transaction log files to access based at least in part on the job identifier associated with the client computing system. 
     
     
         14 . The system of  claim 12 , wherein the one or more hardware processors are further configured to delete the set of transaction log files. 
     
     
         15 . The system of  claim 14 , wherein the one or more hardware processors are further configured to, prior to the deletion of the set of transaction log files, replicate the index within a distributed index storage system. 
     
     
         16 . The system of  claim 12 , wherein the one or more hardware processors are further configured to determine occurrence of a trigger to update the index, wherein the trigger comprises one or more of: an elapse of a period of time, an indication of completion of a backup job, a number of transaction log files added to the set of transaction log files satisfying a threshold; or receipt of a command to update the index. 
     
     
         17 . The system of  claim 12 , wherein the one or more hardware processors update the index by generating a new index. 
     
     
         18 . The system of  claim 11 , wherein the first transaction log file is generated by a third media agent at the first client computing system that is separate from the first media agent. 
     
     
         19 . The system of  claim 11 , wherein a second transaction log file associated with a second data protection operation associated with a second client computing system is provided to the second media agent, and wherein the second transaction log file is replicated to the first media agent. 
     
     
         20 . The system of  claim 11 , wherein the first data protection operation performed using a cached job identifier associated with the first client computing system.