Partial file restore in a data storage system

Abstract

The data storage system according to certain aspects can implement partial file restore, where only a portion of the secondary copy of a file is restored. Such portion may be designated by one or more application offsets for the file. The system may provide an in-chunk index that includes mapping information between the application offsets and the secondary copy offsets. Chunks may refer to logical data units in which secondary copies are stored, and the in-chunk index for a chunk may be stored in secondary storage with the chunk. Because the mapping information may not be provided at a fixed interval, the system can search through application offsets in the in-chunk index to locate the secondary copy offset corresponding to the portion application offset(s). In this manner, the system may restore the designated portion of the secondary copy in a fast and efficient manner by using the in-chunk index.

1 . (canceled) 
     
     
         2 . A method comprising:
 using one or more computing devices comprising computer hardware:
 copying a first portion of a first file from primary storage to a buffer for writing to secondary storage, wherein data stored on the secondary storage is stored in one or more chunks, and wherein each chunk comprises a logical data unit for storing the data in the secondary storage in one or more secondary storage devices residing in the secondary storage; 
 writing the first portion from the buffer to a location in a first chunk of the one or more chunks; 
 creating a first entry in an index for the first chunk, the index stored in association with the first chunk, the first entry corresponding to the first portion of the first file and comprising:
 a first application offset that identifies a starting position within the first file to begin restoring the first portion of the first file, and 
 a first secondary storage offset indicating the location of the first portion of the first file within the first chunk in the secondary storage; and 
 
 writing the first entry to the first chunk, 
 wherein creation of a secondary copy of the first file involves a series of transactions in which data is written to the buffer and then written from the buffer to the secondary storage, and wherein an amount of data written to the buffer varies between the transactions. 
   
     
     
         3 . The method of  claim 2 , wherein the index is contained within the first chunk. 
     
     
         4 . The method of  claim 2 , wherein each of the one or more chunks is of equal size. 
     
     
         5 . The method of  claim 2 , wherein a size of the buffer is based at least on a bandwidth of the one or more computing devices. 
     
     
         6 . The method of  claim 2 , wherein the first entry is written to the secondary storage substantially concurrently with the writing of the first portion of the first file to the first chunk. 
     
     
         7 . The method of  claim 2 , further comprising:
 copying a second portion of the first file from the primary storage to the buffer for writing to the secondary storage; and   creating a second entry in the index for the first chunk, the second entry corresponding to the second portion of the first file and comprising:
 a second application offset corresponding to the second portion, and 
 a second secondary storage offset indicating a location of the second portion within the first chunk in the secondary storage, 
   wherein a first interval between the first application offset and the second application offset is different from a second interval between the second application offset and a third application offset in a third entry in the index.   
     
     
         8 . The method of  claim 7 , wherein a third interval between the first secondary storage offset and the second secondary storage offset is different from a fourth interval between the second secondary storage offset and a third secondary storage offset in the third entry in the index. 
     
     
         9 . The method of  claim 2 , wherein said writing the first portion to the location indicated by the first secondary storage offset comprises deduplicating or compressing the first portion. 
     
     
         10 . The method of  claim 2 , wherein the first chunk comprises metadata information relating to the first file stored in association with the first file. 
     
     
         11 . The method of  claim 2 , further comprising:
 copying a second portion of the first file from the primary storage to the buffer for writing to the secondary storage;   writing the second portion to a second location in a second chunk of the one or more chunks; and   creating a second entry in a second index for the second chunk, the second index stored in association with the second chunk, the second entry corresponding to the second portion of the first file and comprising:
 a second application offset corresponding to the second portion, 
 a second secondary storage offset indicating the second location of the second portion within the second chunk in the secondary storage, and 
 corresponding byte position information relating to a byte position of the second portion within the second chunk. 
   
     
     
         12 . A data storage system comprising:
 one or more computing devices comprising computer hardware and configured to:
 copy a first portion of a first file from primary storage to a buffer for writing to secondary storage, wherein data stored on the secondary storage is stored in one or more chunks, and wherein each chunk comprises a logical data unit for storing the data in the secondary storage in one or more secondary storage devices residing in the secondary storage; 
 write the first portion from the buffer to a location in a first chunk of the one or more chunks; 
 create a first entry in an index for the first chunk, the index stored in association with the first chunk, the first entry corresponding to the first portion of the first file and comprising:
 a first application offset that identifies a starting position within the first file to begin restoring the first portion of the first file; and 
 a first secondary storage offset indicating the location of the first portion of the first file within the first chunk in the secondary storage; and 
 
 write the first entry to the first chunk, 
 wherein creation of a secondary copy of the first file involves a series of transactions in which data is written to the buffer and then written from the buffer to the secondary storage, and wherein an amount of data written to the buffer varies between the transactions. 
   
     
     
         13 . The system of  claim 12 , wherein the index is contained within the first chunk. 
     
     
         14 . The system of  claim 12 , wherein each of the one or more chunks is of equal size. 
     
     
         15 . The system of  claim 12 , wherein a size of the buffer is based at least on a bandwidth of the system. 
     
     
         16 . The system of  claim 12 , wherein the first entry is written to the secondary storage substantially concurrently with the writing of the first portion of the first file to the first chunk. 
     
     
         17 . The system of  claim 12 , wherein the one or more computing devices are further configured to:
 copy a second portion of the first file from the primary storage to the buffer for writing to the secondary storage; and   create a second entry in the index for the first chunk, the second entry corresponding to the second portion of the first file and comprising:
 a second application offset corresponding to the second portion, and 
 a second secondary storage offset indicating a location of the second portion within the first chunk in the secondary storage, 
   wherein a first interval between the first application offset and the second application offset is different from a second interval between the second application offset and a third application offset in a third entry in the index.   
     
     
         18 . The system of  claim 17 , wherein a third interval between the first secondary storage offset and the second secondary storage offset of the secondary copy is different from a fourth interval between the second secondary storage offset and a third secondary storage offset in the third entry in the index. 
     
     
         19 . The system of  claim 12 , wherein the writing of the first portion to the location indicated by the first secondary storage offset is performed at least in part by deduplicating or compressing the first portion. 
     
     
         20 . The system of  claim 12 , wherein the first chunk comprises metadata information relating to the first file stored in association with the first file. 
     
     
         21 . The system of  claim 12 , wherein the one or more computing devices are further configured to:
 copy a second portion of the first file from the primary storage to the buffer for writing to the secondary storage;   write the second portion to a second location in a second chunk of the one or more chunks; and   create a second entry in a second index for the second chunk, the second index stored in association with the second chunk, the second entry corresponding to the second portion of the first file and comprising:
 a second application offset corresponding to the second portion, 
 a second secondary storage offset indicating the second location of the second portion within the second chunk in the secondary storage, and 
 corresponding byte position information relating to a byte position of the second portion within the second chunk.