Transferring or migrating portions of data objects, such as block-level data migration or chunk-based data migration

Abstract

A system and method for migrating data objects based on portions of the data objects is described. The system may transfer portions of files, folders, and other data objects from primary storage to secondary storage based on certain criteria, such as time-based criteria, age-based criteria, and so on. An increment may be one or more blocks of a data object, or one or more chunks of a data object, or other segments that combine to form or store a data object. For example, the system identifies one or more blocks of a data object that satisfy a certain criteria, and migrates the identified blocks. The system may determine that a certain number of blocks of a file have not been modified or called by a file system in a certain time period, and migrate these blocks to secondary storage.

1 . (canceled) 
     
     
         2 . A system comprising:
 a non-transitory, computer-readable storage medium having computer-executable instructions stored thereon; and   one or more processors that, having executed the computer-executable instructions, configure the system to perform a plurality of operations comprising:
 receiving a first data object to be chunked, wherein:
 the first data object is stored on a primary storage device, and 
 the first data object comprises a first plurality of blocks; 
 
 chunking the first data object into a first plurality of chunks, wherein each chunk comprises a second plurality of blocks, wherein each second plurality of blocks is a proper subset of the first plurality of blocks; 
 storing at least some of the first plurality of chunks on a secondary storage device different from the primary storage device; 
 receiving a request to modify content of the first data object; 
 identifying one or more chunks from the first plurality of chunks corresponding to the content to be modified; 
 restoring data of the identified one or more chunks of the first plurality of chunks to a client for modifying the content associated with the received request; 
 presenting the requested content based on the restored data of the identified one or more chunks; 
 modifying the identified one or more chunks based on modifications to the presented content; and 
 replacing the identified one or more chunks stored on the secondary storage device with the modified one or more chunks. 
   
     
     
         3 . The system of  claim 1 , wherein:
 the first data object comprises a file, and   each chunk of the first plurality of chunks corresponds to a unique portion of the file.   
     
     
         4 . The system of  claim 1 , wherein:
 a first chunk of the first plurality of chunks comprises metadata of the first data object,   the first plurality of chunks other than the first chunk comprises payload data of the first data object, and   the identifying of the one or more chunks from the first plurality of chunks corresponding to the content to be modified is based on the first chunk of the first plurality of chunks.   
     
     
         5 . The system of  claim 1 , wherein the plurality of operations further comprises:
 maintaining a data structure that comprises information about the first plurality of chunks, wherein the identifying of the one or more chunks from the first plurality of chunks is based on the data structure; and   updating the data structure based on the modified one or more chunks.   
     
     
         6 . The system of  claim 1 , wherein the chunking of the first data object into the first plurality of chunks is based on at least one of:
 a minimum data size for a chunk,   a maximum data size for a chunk,   a time of creation for data within a chunk, and   a type of data within a chunk.   
     
     
         7 . The system of  claim 1 , wherein the plurality of operations further comprises:
 identifying a second data object having been previously transferred to the secondary storage device using data migration other than chunk-based data migration; and   chunking the second data object into a second plurality of chunks for future retrieval of content of the second data object.   
     
     
         8 . The system of  claim 1 , wherein content of the first data object is assigned to a particular chunk of the first plurality of chunks based on metadata associated with the content. 
     
     
         9 . A method comprising:
 receiving a first data object to be chunked, wherein:
 the first data object is stored on a primary storage device, and 
 the first data object comprises a first plurality of blocks; 
   chunking the first data object into a first plurality of chunks, wherein each chunk comprises a second plurality of blocks, wherein each second plurality of blocks is a proper subset of the first plurality of blocks;   storing at least some of the first plurality of chunks on a secondary storage device different from the primary storage device;   receiving a request to modify content of the first data object;   identifying one or more chunks from the first plurality of chunks corresponding to the content to be modified;   restoring data of the identified one or more chunks of the first plurality of chunks to a client for modifying the content associated with the received request;   presenting the requested content based on the restored data of the identified one or more chunks;   modifying the identified one or more chunks based on modifications to the presented content; and   replacing the identified one or more chunks stored on the secondary storage device with the modified one or more chunks.   
     
     
         10 . The method of  claim 9 , wherein:
 the first data object comprises a file, and   each chunk of the first plurality of chunks corresponds to a unique portion of the file.   
     
     
         11 . The method of  claim 9 , wherein:
 a first chunk of the first plurality of chunks comprises metadata of the first data object,   the first plurality of chunks other than the first chunk comprises payload data of the first data object, and   the identifying of the one or more chunks from the first plurality of chunks corresponding to the content to be modified is based on the first chunk of the first plurality of chunks.   
     
     
         12 . The method of  claim 9 , further comprising:
 maintaining a data structure that comprises information about the first plurality of chunks, wherein the identifying of the one or more chunks from the first plurality of chunks is based on the data structure; and   updating the data structure based on the modified one or more chunks.   
     
     
         13 . The method of  claim 9 , wherein the chunking of the first data object into the first plurality of chunks is based on at least one of:
 a minimum data size for a chunk,   a maximum data size for a chunk,   a time of creation for data within a chunk, and   a type of data within a chunk.   
     
     
         14 . The method of  claim 9 , further comprising:
 identifying a second data object having been previously transferred to the secondary storage device using data migration other than chunk-based data migration; and   chunking the second data object into a second plurality of chunks for future retrieval of content of the second data object.   
     
     
         15 . The method of  claim 9 , wherein content of the first data object is assigned to a particular chunk of the first plurality of chunks based on metadata associated with the content. 
     
     
         16 . A non-transitory, computer-readable storage medium having computer-executable instructions stored thereon that, when executed by one or more processors, configures a system to perform a plurality of operations comprising:
 receiving a first data object to be chunked, wherein:
 the first data object is stored on a primary storage device, and 
 the first data object comprises a first plurality of blocks; 
   chunking the first data object into a first plurality of chunks, wherein each chunk comprises a second plurality of blocks, wherein each second plurality of blocks is a proper subset of the first plurality of blocks;   storing at least some of the first plurality of chunks on a secondary storage device different from the primary storage device;   receiving a request to modify content of the first data object;   identifying one or more chunks from the first plurality of chunks corresponding to the content to be modified;   restoring data of the identified one or more chunks of the first plurality of chunks to a client for modifying the content associated with the received request;   presenting the requested content based on the restored data of the identified one or more chunks;   modifying the identified one or more chunks based on modifications to the presented content; and   replacing the identified one or more chunks stored on the secondary storage device with the modified one or more chunks.   
     
     
         17 . The non-transitory, computer-readable storage medium of  claim 16 , wherein:
 the first data object comprises a file, and   each chunk of the first plurality of chunks corresponds to a unique portion of the file.   
     
     
         18 . The non-transitory, computer-readable storage medium of  claim 16 , wherein:
 a first chunk of the first plurality of chunks comprises metadata of the first data object,   the first plurality of chunks other than the first chunk comprises payload data of the first data object, and   the identifying of the one or more chunks from the first plurality of chunks corresponding to the content to be modified is based on the first chunk of the first plurality of chunks.   
     
     
         19 . The non-transitory, computer-readable storage medium of  claim 16 , wherein the plurality of operations further comprises:
 maintaining a data structure that comprises information about the first plurality of chunks, wherein the identifying of the one or more chunks from the first plurality of chunks is based on the data structure; and   updating the data structure based on the modified one or more chunks.   
     
     
         20 . The non-transitory, computer-readable storage medium of  claim 16 , wherein the chunking of the first data object into the first plurality of chunks is based on at least one of:
 a minimum data size for a chunk,   a maximum data size for a chunk,   a time of creation for data within a chunk, and   a type of data within a chunk.   
     
     
         21 . The non-transitory, computer-readable storage medium of  claim 16 , wherein the plurality of operations further comprises:
 identifying a second data object having been previously transferred to the secondary storage device using data migration other than chunk-based data migration; and   chunking the second data object into a second plurality of chunks for future retrieval of content of the second data object.