Inter-cloud shared content data management

Abstract

Methods, systems, and computer program products for multi-cloud shared data management. A first storage facility on a first cloud is established and a second storage facility on a second cloud established. The first cloud and the second cloud comprise at least respective first deduplication metadata and second deduplication metadata. In response to receipt of a request to store a content object comprising at least one candidate deduplication chunk, an inter-cloud deduplication protocol is initiated. The inter-cloud deduplication protocol determines whether or not a candidate deduplication chunk of a data item is already stored at either the first cloud or the second cloud. In the event that the candidate deduplication chunk of the data item deemed to have been already stored then, rather than storing a further copy of the bit sequence corresponding to the candidate deduplication chunk, instead, merely referring to the location of the already stored candidate deduplication chunk.

What is claimed is: 
     
         1 . A non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by a processor cause the processor to perform acts comprising:
 establishing a first storage facility on a first public cloud and establishing a second storage facility on a second public cloud, wherein the first public cloud and the second public cloud comprise at least respective first deduplication metadata and second deduplication metadata;   responsive to receipt of a request to store a content object comprising at least one candidate deduplication chunk:
 initiating an inter-cloud deduplication protocol wherein the inter-cloud deduplication protocol determines that a candidate deduplication chunk of a data item is already stored at either the first public cloud or the second public cloud; and 
 rather than storing a copy of a bit sequence corresponding to the candidate deduplication chunk, instead, referring to either a first location at the first public cloud or to a second location at the second public cloud as an already stored alternate location of the bit sequence corresponding to the candidate deduplication chunk. 
   
     
     
         2 . The non-transitory computer readable medium of  claim 1 , wherein the first storage facility and the second storage facility are blob storage facilities. 
     
     
         3 . The non-transitory computer readable medium of  claim 1 , wherein the inter-cloud deduplication protocol checks with one or more neighboring clouds that are one hop away, and wherein the one or more neighboring clouds that are one hop away check with one or more further neighboring clouds that are a further one hop away. 
     
     
         4 . The non-transitory computer readable medium of  claim 1 , further comprising instructions which, when stored in memory and executed by the processor cause the processor to perform further acts of generating a map entry in data item metadata to map a data unit to a physical storage location, wherein the data unit is a deduplicated data unit in deduplication metadata, and wherein the data item metadata and the deduplication metadata are different data structures. 
     
     
         5 . The non-transitory computer readable medium of  claim 4 , further comprising instructions which, when stored in memory and executed by the processor cause the processor to perform further acts of performing a scan of the data item metadata for the data unit, wherein an in-use state is set in the deduplication metadata for the data unit when the map entry is detected for the data unit in the data item metadata, a not-in-use state and a deletion state are set for the data unit in the deduplication metadata when the map entry is not detected in the data item metadata for the data unit, and the data unit is deleted from the physical storage location based at least in part on a result of the scan when the data unit is determined to correspond to the not-in-use state and the deletion state. 
     
     
         6 . The non-transitory computer readable medium of  claim 4 , wherein the data item metadata and the deduplication metadata are distributed over a plurality of computing nodes. 
     
     
         7 . The non-transitory computer readable medium of  claim 1 , wherein a deduplication of the candidate deduplication chunk of a data item is managed without implementing a reference count value of the candidate deduplication chunk. 
     
     
         8 . The non-transitory computer readable medium of  claim 1 , further comprising instructions which, when stored in memory and executed by the processor cause the processor to perform further acts of:
 enumerating a set of map entries corresponding to a data unit; and   setting a usage state to an in-use state when the set of map entries is a non-empty set, or setting to a not-in-use state when the set of map entries is an empty set.   
     
     
         9 . A method comprising:
 establishing a first storage facility on a first public cloud and establishing a second storage facility on a second public cloud, wherein the first public cloud and the second public cloud comprise at least respective first deduplication metadata and second deduplication metadata;   responsive to receipt of a request to store a content object comprising at least one candidate deduplication chunk:
 initiating an inter-cloud deduplication protocol wherein the inter-cloud deduplication protocol determines that a candidate deduplication chunk of a data item is already stored at either the first public cloud or the second public cloud; and 
 rather than storing a copy of a bit sequence corresponding to the candidate deduplication chunk, instead, referring to either a first location at the first public cloud or to a second location at the second public cloud as an already stored alternate location of the bit sequence corresponding to the candidate deduplication chunk. 
   
     
     
         10 . The method of  claim 9 , wherein the first storage facility and the second storage facility are blob storage facilities. 
     
     
         11 . The method of  claim 9 , wherein the inter-cloud deduplication protocol checks with one or more neighboring clouds that are one hop away, and wherein the one or more neighboring clouds that are one hop away check with one or more further neighboring clouds that are a further one hop away. 
     
     
         12 . The method of  claim 9 , further comprising generating a map entry in data item metadata to map a data unit to a physical storage location, wherein the data unit is a deduplicated data unit in deduplication metadata, and wherein the data item metadata and the deduplication metadata are different data structures. 
     
     
         13 . The method of  claim 12 , further comprising performing a scan of the data item metadata for the data unit, wherein an in-use state is set in the deduplication metadata for the data unit when the map entry is detected for the data unit in the data item metadata, a not-in-use state and a deletion state are set for the data unit in the deduplication metadata when the map entry is not detected in the data item metadata for the data unit, and the data unit is deleted from the physical storage location based at least in part on a result of the scan when the data unit is determined to correspond to the not-in-use state and the deletion state. 
     
     
         14 . The method of  claim 12 , wherein the data item metadata and the deduplication metadata are distributed over a plurality of computing nodes. 
     
     
         15 . The method of  claim 9 , wherein a deduplication of the candidate deduplication chunk of a data item is managed without implementing a reference count value of the candidate deduplication chunk. 
     
     
         16 . The method of  claim 9 , further comprising:
 enumerating a set of map entries corresponding to a data unit; and   setting a usage state to an in-use state when the set of map entries is a non-empty set, or setting to a not-in-use state when the set of map entries is an empty set.   
     
     
         17 . A system comprising:
 a storage medium having stored thereon a sequence of instructions; and   a processor that executes the sequence of instructions to cause the processor to perform acts comprising,
 establishing a first storage facility on a first public cloud and establishing a second storage facility on a second public cloud, wherein the first public cloud and the second public cloud comprise at least respective first deduplication metadata and second deduplication metadata; 
 responsive to receipt of a request to store a content object comprising at least one candidate deduplication chunk:
 initiating an inter-cloud deduplication protocol wherein the inter-cloud deduplication protocol determines that a candidate deduplication chunk of a data item is already stored at either the first public cloud or the second public cloud; and 
 rather than storing a copy of a bit sequence corresponding to the candidate deduplication chunk, instead, referring to either a first location at the first public cloud or to a second location at the second public cloud as an already stored alternate location of the bit sequence corresponding to the candidate deduplication chunk. 
 
   
     
     
         18 . The system of  claim 17 , wherein the first storage facility and the second storage facility are blob storage facilities. 
     
     
         19 . The system of  claim 17 , wherein the inter-cloud deduplication protocol checks with one or more neighboring clouds that are one hop away, and wherein the one or more neighboring clouds that are one hop away check with one or more further neighboring clouds that are a further one hop away. 
     
     
         20 . The system of  claim 17 , further comprising instructions which, when stored in memory and executed by the processor cause the processor to perform further acts of generating a map entry in data item metadata to map a data unit to a physical storage location, wherein the data unit is a deduplicated data unit in deduplication metadata, and wherein the data item metadata and the deduplication metadata are different data structures. 
     
     
         21 . The system of  claim 20 , further comprising instructions which, when stored in memory and executed by the processor cause the processor to perform further acts of performing a scan of the data item metadata for the data unit, wherein an in-use state is set in the deduplication metadata for the data unit when the map entry is detected for the data unit in the data item metadata, a not-in-use state and a deletion state are set for the data unit in the deduplication metadata when the map entry is not detected in the data item metadata for the data unit, and the data unit is deleted from the physical storage location based at least in part on a result of the scan when the data unit is determined to correspond to the not-in-use state and the deletion state. 
     
     
         22 . The system of  claim 20 , wherein the data item metadata and the deduplication metadata are distributed over a plurality of computing nodes. 
     
     
         23 . The system of  claim 17 , wherein a deduplication of the candidate deduplication chunk of a data item is managed without implementing a reference count value of the candidate deduplication chunk. 
     
     
         24 . The system of  claim 17 , further comprising instructions which, when stored in memory and executed by the processor cause the processor to perform further acts of:
 enumerating a set of map entries corresponding to a data unit; and   setting a usage state to an in-use state when the set of map entries is a non-empty set, or setting to a not-in-use state when the set of map entries is an empty set.