Dedicated client-side signature generator in a networked storage system

Abstract

A storage system according to certain embodiments includes a client-side signature repository that includes information representative of a set of data blocks stored in primary storage. During storage operations of a client, the system can generate signatures corresponding to data blocks that are being stored in primary storage. The system can store the generated signatures in the client-side signature repository along with information regarding the location of the corresponding data block within primary storage. As additional instances of the data block are stored in primary storage, the system can store the location of the additional instances in the client-side signature repository.

What is claimed is: 
     
         1 . A method of maintaining a signature repository accessible by multiple client computing devices in a data storage system, the method comprising:
 tracking storage of data units corresponding to primary data generated by one or more applications executing on a plurality of client computing devices that form a primary storage subsystem, the primary data for each of the client computing devices stored in a data store associated with the respective client computing device,   the primary storage subsystem in communication with a secondary storage subsystem that is separate from the primary storage subsystem and is configured to maintain secondary copies of at least some of the primary data;   generating, by a signature agent executing on one or more processors in the primary storage subsystem, signatures corresponding to the plurality of tracked data units; and   maintaining a signature repository including a signature block for at least each unique signature of the generated signatures, where each signature block comprises:
 the unique signature; and 
 one or data unit entries, each entry corresponding to a copy of the data unit associated with the unique signature that is stored in the primary storage subsystem, wherein each entry indicates which of the plurality of client computing devices stores the corresponding copy of the data unit, wherein at least some of the signature blocks include at least a first entry indicating that a first client computing device of the plurality of client computing devices stores a copy of the data unit and a second entry indicating that a second client computing device of the plurality of client computing devices stores a copy of the data unit. 
   
     
     
         2 . The system of  claim 1 , wherein the first entry further includes location information identifying a location where the copy of the data unit resides in the data store of the first client computing device and wherein the second entry further includes location information identifying a location where the copy of the data unit resides in the data store of the second client computing device. 
     
     
         3 . The method of  claim 1 , further comprising:
 receiving a query including a plurality of signatures;   comparing the signatures included in the query with signature blocks in the signature repository to identify a first set of signatures received in the query that correspond to data units that reside in a data store of at least one client computing device of the plurality of client computing devices; and   for at least some of the signatures in the first set of signatures, accessing the corresponding data units from the data store of the at least one client computing device.   
     
     
         4 . The method of  claim 3 , further comprising, for the signatures included in the query that are not included in first set of signatures, accessing the corresponding data units from the secondary storage subsystem. 
     
     
         5 . The method of  claim 4 , wherein the plurality of signatures included in the query correspond to a set of data units which represent a backed up version of a set of the primary data that is to be restored to a particular client computing device of the plurality of client computing devices, and wherein at least some of the data units corresponding to the signatures in the first set of signatures are accessed from client computing devices other than the particular client computing device. 
     
     
         6 . The method of  claim 1 , further comprising:
 in response to receipt of instructions to backup at least a subset of the primary data of a particular client computing device of the plurality of client computing devices, comparing a set of signatures corresponding to data units in the subset of the primary data with entries in the signature repository;   based on the comparison, identifying a first set of signatures corresponding to data units in the subset of the primary data which reside in at least one other client computing device of the plurality of client computing devices other than the particular client computing device; and   for at least some of the signatures in the first set of signatures, accessing the corresponding data units from the at least one other client computing device.   
     
     
         7 . The method of  claim 6 , further comprising:
 for the signatures not in the first set of signatures, accessing the corresponding data units from the particular client computing device;   associating the data units accessed from the at least one other client computing device with the data units accessed from the particular client computing device to generate a backup set corresponding to the subset of the primary data of the particular client computing device; and   communicating the backup set to the secondary storage subsystem.   
     
     
         8 . The method of  claim 1 , wherein the secondary storage subsystem comprises deduplicated data. 
     
     
         9 . The method of  claim 1 , wherein the data store of at least one the plurality of client computing devices comprises deduplicated data. 
     
     
         10 . A storage system, comprising:
 a signature repository agent executing on one or more processors in a primary storage subsystem,   the primary storage subsystem comprising:
 a plurality of client computing devices; 
 a plurality of data agents executing on the plurality of client computing devices, the plurality of data agents configured to track storage of a plurality of data units which correspond to primary data generated by one or more applications executing on the plurality of client computing devices, and 
 the primary data for each of the client computing devices is stored in a data store associated with the respective client computing device, and the primary storage subsystem is in communication with a secondary storage subsystem that is separate from the primary storage subsystem and that is configured to maintain secondary copies of at least some of the primary data; and 
   wherein the signature repository agent is configured to maintain a signature repository including a signature block for at least each unique signature generated by the one or more signature agents, each signature block comprising:
 the unique signature; and 
 one or data unit entries, each entry corresponding to a copy of the data unit associated with the unique signature that is stored in the primary storage subsystem, wherein each entry indicates which of the plurality of client computing devices stores the corresponding copy of the data unit, wherein at least some of the signature blocks include at least a first entry indicating that a first client computing device of the plurality of client computing devices stores a copy of the data unit and a second entry indicating that a second client computing device of the plurality of client computing devices stores a copy of the data unit. 
   
     
     
         11 . The system of  claim 10 , wherein the first entry further includes location information identifying a location where the copy of the data unit resides in the data store of the first client computing device and wherein the second entry further includes location information identifying a location where the copy of the data unit resides in the data store of the second client computing device. 
     
     
         12 . The system of  claim 10 , wherein at least one of the one or more signature agents resides on each client computing device. 
     
     
         13 . The system of  claim 10 , wherein the one or more signature agents execute on one or more computing devices that are separate from the plurality of client computing devices. 
     
     
         14 . The system of  claim 10 , wherein the signature repository agent is further configured to:
 receive a query from the secondary storage subsystem, the query including a plurality of signatures;   compare the plurality of signatures with signature blocks in the signature repository to identify a first set of signatures from the plurality of signatures that correspond to data units that reside in a data store of at least one client computing device of the plurality of client computing devices; and   for at least some signatures in the first set of signatures, request the corresponding data units from the at least one client computing device.   
     
     
         15 . The system of  claim 14 , wherein the signature repository agent is further configured to request from the secondary storage subsystem the data units corresponding to signatures of the plurality of signatures that are not included in first set of signatures. 
     
     
         16 . The system of  claim 15 , wherein the plurality of signatures correspond to a set of data units which represent a backed up version of a set of primary data that is to be restored to a particular client computing device of the plurality of client computing devices, and wherein at least some data units corresponding to the signatures in the first set of signatures are accessed from client computing devices other than the particular client computing device. 
     
     
         17 . The system of  claim 10 , wherein the signature repository agent is further configured to:
 in response to receipt of instructions to backup at least a subset of the primary data of a particular client computing device, compare a backup set of signatures corresponding to data units in the subset of the primary data with signature blocks in the signature repository;   based on the comparison, identify a first set of signatures corresponding to data units in the subset of the primary data which reside in at least one other client computing device of the plurality of client computing devices other than the particular client computing device; and   for at least some signatures in the first set of signatures, request the corresponding data units from the at least one other client computing device.   
     
     
         18 . The system of  claim 17 , wherein the signature repository agent is further configured to:
 for signatures in the backup set of signatures but not in the first set of signatures, request the corresponding data units from the particular client computing device;   associate the data units accessed from the at least one other client computing device with the data units accessed from the particular client computing device to generate a backup set of data corresponding to the subset of the primary data of the particular client computing device; and   communicate the backup set of data to the secondary storage subsystem.   
     
     
         19 . The system of  claim 10 , wherein the secondary storage subsystem comprises deduplicated data. 
     
     
         20 . The system of  claim 10 , wherein the data store of at least one of the plurality of client computing devices comprises deduplicated data. 
     
     
         21 . A computer-readable, non-transitory storage medium having one or more computer-executable modules for maintaining a signature repository accessible by multiple client computing devices in a data storage system, the one or more computer-executable modules comprising:
 a first module in communication with a plurality of client computing devices that form a primary storage subsystem,   the primary storage subsystem comprising:
 the plurality of client computing devices; 
 a plurality of data agents executing on the plurality of client computing devices, the plurality of data agents configured to track storage of a plurality of data units which correspond to primary data generated by one or more applications executing on the plurality of client computing devices, 
 the primary data for each of the client computing devices is stored in a data store associated with the respective client computing device, and the primary storage subsystem is in communication with a secondary storage subsystem that is separate from the primary storage subsystem and is configured to maintain secondary copies of at least some of the primary data; 
   wherein the first module is configured to:
 store in maintain a signature repository including a signature block for at least each unique signature of the generated signatures, where each signature block comprises:
 the unique signature; and 
 one or data unit entries, each entry corresponding to a copy of the data unit associated with the unique signature that is stored in the primary storage subsystem, wherein each entry indicates which of the plurality of client computing devices stores the corresponding copy of the data unit, wherein at least some of the signature blocks include at least a first entry indicating that a first client computing device of the plurality of client computing devices stores a copy of the data unit and a second entry indicating that a second client computing device of the plurality of client computing devices stores a copy of the data unit.