Mitigating the impact of a single point of failure in an object store

Abstract

Example apparatus and methods distribute ranges or erasure codes associated with ranges to reduce or minimize the impact of a single point of failure in an object store. Erasure codes associated with related ranges to be stored in an object store may be accessed and selectively distributed to different storage devices associated with the object store. The erasure codes may be distributed according to a distribution plan so that an unavailability of one storage device will cause less than all of the related ranges to become unavailable. Example apparatus and methods may also provide a partial GET operation that will retrieve erasure codes associated with less than an entire object or with less than all possible ranges for an object. The partial GET operation may facilitate reconstructing less than an entire object, which may be valuable in, for example, weblog analytics.

What is claimed is: 
     
         1 . A non-transitory computer-readable storage device storing computer-executable instructions that when executed by a computer cause the computer to perform a method for reducing the impact of a single point of failure in an object store, the method comprising:
 accessing a plurality of super blocks associated with an object to be stored in the object store, where the object store includes a plurality of storage apparatus;   identifying a plurality of distinct subsets of storage apparatus from the plurality of storage apparatus;   establishing a redundancy policy for erasure codes associated with the object;   controlling the object store to store the erasure codes in members of the plurality of storage apparatus, based, at least in part, on the redundancy policy;   identifying how many of the plurality of super blocks can be reconstructed from the object store;   upon determining that a desired reconstruction utility can be achieved from one or more super blocks:
 selectively retrieving the one or more super blocks from the object store; 
 and reducing a redundancy level associated with the redundancy policy. 
   
     
     
         2 . The non-transitory computer-readable storage device of  claim 1 , where a member of the plurality of distinct subsets of storage apparatus will be used to store members of the plurality of super blocks associated with the object to be stored in the object store. 
     
     
         3 . The non-transitory computer-readable storage device  claim 2 , where a single member of the plurality of super blocks can be stored in a single member of the plurality of storage apparatus. 
     
     
         4 . The non-transitory computer-readable storage device of  claim 3 , where a single member of the plurality of storage apparatus becoming unavailable will cause only one of the plurality of super blocks to become unavailable. 
     
     
         5 . The non-transitory computer-readable storage device of  claim 1 , where the redundancy policy is an N/M policy, where N describes the total number of erasure codes associated with a super block, N-M describes the minimum number of the N erasure codes needed to reconstruct the super block, N being greater than M, N and M being integers. 
     
     
         6 . The non-transitory computer-readable storage device of  claim 5 , where M is one. 
     
     
         7 . The non-transitory computer-readable storage device of  claim 5 , where M is zero. 
     
     
         8 . The non-transitory computer-readable storage device of  claim 1 , where the one or more super blocks are less than all the super blocks. 
     
     
         9 . The non-transitory computer-readable storage device of  claim 1 , where identifying the plurality of distinct subsets of storage apparatus from the plurality of storage apparatus is based on a user defined rule, an automated rule, or an input from a user interface. 
     
     
         10 . The non-transitory computer-readable storage device of  claim 1 , the method further comprising:
 partitioning the object into a plurality of super blocks.   
     
     
         11 . The non-transitory computer-readable storage device of  claim 10 , where the number of super blocks into which the object is partitioned is based, at least in part, on how many storage apparatus are available in the plurality of storage apparatus. 
     
     
         12 . The non-transitory computer-readable storage device of  claim 1 , where the plurality of storage apparatus includes two or more different storage apparatus. 
     
     
         13 . The non-transitory computer-readable storage device of  claim 12 , where the two or more different storage apparatus include disk drives, solid state drives, or tape drives. 
     
     
         14 . The non-transitory computer-readable storage device of  claim 1 , the method further comprising:
 providing data describing how many of the plurality of super blocks can be reconstructed.   
     
     
         15 . The non-transitory computer-readable storage device of  claim 1 , the method further comprising:
 identifying a subset of the plurality of super blocks that cannot be reconstructed from the object store; and   providing data concerning the subset.   
     
     
         16 . The non-transitory computer-readable storage device of  claim 1 , the method further comprising:
 selectively reconstructing the object from the one or more super blocks.   
     
     
         17 . An apparatus, comprising:
 a processor;   a memory comprising a non-transitory computer-readable storage medium, where the memory stores erasure codes associated with members of two or more related ranges to be stored in an object store, where the two or more related ranges are associated with a single object;   a set of logics that control erasure code distribution to reduce the impact of a single point of failure in an object store than includes a plurality of storage apparatus; and   an interface that connects the processor, the memory, and the set of logics, the set of logics comprising:
 a first logic that identifies two or more different storage devices associated with the object store, and that identifies two or more different types of storage devices associated with the object store; 
 a second logic that establishes a distribution plan to store erasure codes on the two more different storage devices, so that one of the two or more different storage devices becoming unavailable will cause only one of the related ranges to become unavailable, where the second logic controls the object store to store a first member of the two or more related ranges on a first type of storage device and to store a second, different member of the two or more related ranges on a second, different type of storage device based, at least in part, on the distribution plan, where the distribution plan includes a redundancy policy for the erasure codes; 
 a third logic that identifies a first subset of the two or more related ranges that are retrievable from the object store, where the first subset may be less than all of the related ranges; and 
 a fourth logic that determines if the first subset meets a threshold utility measure, where the fourth logic, upon determining that the first subset meets the threshold utility measure, reduces a redundancy setting associated with the redundancy policy, retrieves selected erasure codes associated with the first subset from the object store, and selectively reconstructs the first subset from the selected erasure codes. 
   
     
     
         18 . The apparatus of  claim 17 , where the two or more different storage devices associated with the object store include disk drives, solid state drives, or tape drives, and where the two or more different types of storage devices associated with the object store include disk drives, solid state drives, or tape drives. 
     
     
         19 . The apparatus of  claim 17 , where the redundancy policy is an N/1 policy, or an N/0 policy, N being an integer greater than one. 
     
     
         20 . A system, comprising:
 means for identifying a plurality of distinct subsets of storage apparatus from a plurality of storage apparatus, where members of the plurality of distinct subsets store members of a plurality of super blocks associated with an object to be stored in an object store;   means for establishing a redundancy policy for erasure codes associated with the object, where the redundancy policy is an N/M policy, where N describes the total number of erasure codes associated with a super block, N-M describes the minimum number of the N erasure codes needed to reconstruct the super block, N being greater than M, N and M being integers;   means for controlling the object store to store the erasure codes in members of the plurality of storage apparatus as controlled by the redundancy policy;   means for identifying how many of the plurality of super blocks can be reconstructed from the object store and for providing data describing how many of the plurality of super blocks can be reconstructed;   means for selectively retrieving one or more of the super blocks from the object store upon determining that a desired reconstruction utility can be achieved from the one or more super blocks, where the one or more super blocks are less than all the super blocks; and   means for reducing a redundancy level associated with the redundancy policy.