Distributed indexing system for data storage

Abstract

A distributed indexing system spreads out the load on an index of stored data in a data storage system. Rather than maintain a single index, the distributed indexing system maintains an index in each media agent of a federated data storage system and a master index that points to the index in each media agent. In some embodiments, the distributed indexing system includes an index server (or group of servers) that handles indexing requests and forwards the requests to the appropriate distributed systems. Thus, the distributed indexing system, among other things, increases the availability and fault tolerance of a data storage index.

What is claimed is: 
     
       1. A method of creating indexes of secondary copies of data, the method comprising;
 copying primary data stored in one or more primary storage devices to one or more secondary storage devices, wherein copying the primary data to the secondary storage device creates a secondary copy on the one or more secondary storage devices; 
 indexing a first portion of the secondary copy with a first index server, wherein indexing the first portion of the secondary copy creates a first index of the first portion of secondary data; 
 indexing a second portion of the secondary copy with a second index server, wherein indexing the second portion of secondary copy creates a second index of the second portion of secondary data; 
 copying the first index to the second index server so that the first index is available at both the first and second index servers; 
 identifying that a plurality of requests for secondary data are associated with the first index; and 
 distributing the plurality of requests for the secondary data associated with the first index among the first and second index servers to lessen a load on the first index server. 
 
     
     
       2. The method of  claim 1 , further comprising:
 receiving a request for primary data; 
 determining that a secondary copy of the requested primary data exists in the one or more secondary storage devices; and 
 sending the request to at least one of first and second index servers to retrieve a portion of the secondary copy that relates to the requested primary memory. 
 
     
     
       3. The method of  claim 1  further comprising copying the second index to the first index server so that the second index is available at both the first and second index servers. 
     
     
       4. The method of  claim 1 , further comprising:
 receiving a request for primary data; 
 determining that a secondary copy of the requested primary data exists in the one or more secondary storage devices; and 
 sending the request for to both the first and second index servers to retrieve, based on the first and second indexes, a portion of the secondary copy that relates to the requested primary memory. 
 
     
     
       5. The method of  claim 1 , further comprising:
 logging new index data on a network share and associated with a network address; and 
 providing to the first and second index servers the network address. 
 
     
     
       6. The method of  claim 1 , further comprising:
 receiving a request for a document or for information about the document; 
 determining a department or office within a business organization associated with the document; and 
 forwarding the received request to one of first and second index servers associated the determined department or office. 
 
     
     
       7. The method of  claim 1  further comprising copying the first and second indexes to a third index server. 
     
     
       8. The method of  claim 1  further comprising combining the data in the first and second indexes to create a third index. 
     
     
       9. The method of  claim 8  further comprising distributing requests associated with the first index among at least one of the second and third index servers when the first index server is unavailable. 
     
     
       10. A system that creates indexes of secondary copies of data, the system comprising:
 a storage manager comprising computer hardware, the storage manager configured to copy primary data stored in one or more primary storage devices to one or more secondary storage devices, wherein copying the primary data to the secondary storage device creates a secondary copy on the one or more secondary storage devices; 
 a first index server comprising computer hardware having one or more computer processors, the first index server creates a first index by indexing a first portion of the secondary copy stored on the one or more secondary storage devices; 
 a second index server comprising computer hardware having one or more computer processors, the second index server creates a second index by indexing index a second portion of the secondary copy stored on the one or more secondary storage devices; 
 an index replication component comprising computer hardware, the index replication component configured to direct the copying the first index associated with the first index server to the second index server so that the first index is available at both the first and second index servers; 
 a third index server comprising computer hardware that is configured to identify that a plurality of requests for the secondary data are associated with the first index, the third index server further configured to distribute the plurality of requests for the secondary data associated with the first index among the first and second index servers to lessen a load on the first index server. 
 
     
     
       11. The system of  claim 10 , wherein the third index server receives a request for primary data and determines that a secondary copy of the requested primary data exists in the one or more secondary storage devices, wherein the third index server sends the request to at least one of first and second index servers to retrieve a portion of the secondary copy that relates to the requested primary memory. 
     
     
       12. The system of  claim 10  wherein the index replication component is further configured to copy the second index associated with the second index server to the first index server so that the second index is available at both the first and second index servers. 
     
     
       13. The system of  claim 10 , wherein the third index server receives a request for primary data and determines that a secondary copy of the requested primary data exists in the one or more secondary storage devices, wherein the third index server sends the request for to both the first and second index servers to retrieve, based on the first and second indexes, a portion of the secondary copy that relates to the requested primary memory. 
     
     
       14. The system of  claim 10 , wherein the storage manager is further configured to:
 log new index data on a network share and associated with a network address; and 
 provide to the first and second index servers the network address. 
 
     
     
       15. The system of  claim 10 , wherein the third index server is further configured to:
 receive a request for a document or for information about the document; 
 determine a department or office within a business organization associated with the document; and 
 forward the received request to one of first and second index servers associated the determined department or office. 
 
     
     
       16. The system of  claim 10  wherein the index replication component is further configured to copy the first and second indexes to the third index server. 
     
     
       17. The method of  claim 10  wherein the index replication component combines the data in the first and second indexes to create a third index. 
     
     
       18. The system of  claim 17  wherein the third index server is configured to distribute requests associated with the first index among the second and third index servers when the first index server is unavailable.