Storing data and associated metadata in a dispersed storage network

Abstract

A dispersed storage (DS) processing unit includes a processing module generating metadata for a data object. The DS processing unit further includes a partitioner to partition the data object to produce a plurality of data segments. The DS processing unit further includes a first dispersed storage error encoder encoding the metadata and at least a first data segment using first dispersal parameters to produce a set of encoded metadata slices with a set of first data segment encoded data slices. The DS processing unit further includes a second dispersed storage error encoding a plurality of additional data segments, not including the first data segment, using second dispersal parameters to produce a plurality of additional sets of encoded data slices. The DS processing unit further includes a grouping selector facilitating storage of the set of metadata slices, the first set of encoded data slices and the plurality of additional sets of encoded data slices in a storage set.

What is claimed is: 
     
         1 . A dispersed storage processing unit comprises:
 a processing module generating metadata for a data object;   a first dispersed storage error encoder disperse storage error encoding the metadata to produce a set of metadata slices, wherein the first disperse storage error encoder utilizes first dispersal parameters to store the metadata, the first dispersal parameters including a decode threshold of 1;   a data partitioner partitioning the data object to produce a plurality of data segments;   the first disperse storage error encoder encoding at least a first data segment of the plurality of data segments to produce a first set of encoded data slices, wherein the first disperse storage error encoder utilizes the first dispersal parameters of the metadata, the first dispersal parameters including a decode threshold of 1;   a second disperse storage error encoder encoding additional data segments of the plurality of data segments to produce a plurality of sets of encoded data slices, wherein the additional data segments do not include the at least a first data segment and wherein the second disperse storage error encoder utilizes second dispersal parameters, the second dispersal parameters different from the first dispersal parameters and including a decode threshold greater than 1; and   a grouping selector facilitating storage of the set of metadata slices, the first set of encoded data slices and the plurality of sets of encoded data slices in one or more storage units of a dispersed storage network (DSN); and   wherein subsequent accessing of the metadata slices reproduces the metadata and the first data segment of the data object, making the first data segment of the data object available for use by one or more computing devices while continuing to retrieve the additional data segments from the DSN.   
     
     
         2 . The dispersed storage processing unit of  claim 1 , wherein the first dispersed storage error encoder utilizes the first dispersal parameters to store a copy of the metadata in each storage unit. 
     
     
         3 . The dispersed storage processing unit of  claim 1 , wherein the partitioning is in accordance with a data segmentation approach. 
     
     
         4 . The dispersed storage processing unit of  claim 3 , wherein the data segmentation approach includes any of: fixed size, segment size ramping-up, or segment size ramping-down. 
     
     
         5 . The dispersed storage processing unit of  claim 1  further comprises, for each storage unit of a selected storage set, issuing a write slice request to the storage unit, where the write slice request includes an associated metadata slice of the set of metadata slices and a plurality of associated encoded data slices of each set of encoded data slices. 
     
     
         6 . The dispersed storage processing unit of  claim 5 , wherein the plurality of associated encoded data slices of each set of encoded data slices includes a common pillar. 
     
     
         7 . The dispersed storage processing unit of  claim 6 , wherein the common pillar includes a set of storage units equal in number to a width of the common pillar. 
     
     
         8 . A dispersed storage processing unit of a dispersed storage network (DSN), the dispersed storage processing unit comprises:
 an interface;   a local memory; and   a processing module operably coupled to the interface and the local memory to generate metadata for a data object;   a first dispersed storage error encoder to encode the metadata to produce a set of metadata slices, wherein the first disperse storage error encoder utilizes first dispersal parameters to store metadata, the first dispersal parameters including a decode threshold of 1;   a data partitioner to partition the data object to produce a plurality of data segments;   the first dispersed storage error encoder encoding at least a first data segment of the plurality of data segments to produce a first set of encoded data slices, wherein the first dispersed storage error encoder utilizes the first dispersal parameters of the metadata, the first dispersal parameters including a decode threshold of 1;   second dispersed storage error encoder encoding additional data segments of the plurality of data segments to produce a plurality of sets of encoded data slices, wherein the additional data segments do not include the at least a first data segment and wherein the second dispersed storage error encoder utilizes second dispersal parameters, the second dispersal parameters different from the first dispersal parameters and including a decode threshold greater than 1; and   a grouping selector facilitating storage of the set of metadata slices, the first set of encoded data slices and the plurality of sets of encoded data slices in one or more storage units of the DSN; and   wherein subsequent accessing of the set of metadata slices reproduces the metadata and the first data segment of the data object, making the first data segment of the data object available for use by one or more computing devices while continuing to retrieve the additional data segments from the DSN.   
     
     
         9 . The dispersed storage processing unit of  claim 8 , wherein the first dispersed storage error encoder utilizes the first dispersal parameters to store a copy of the metadata in each storage unit. 
     
     
         10 . The dispersed storage processing unit of  claim 8 , wherein the partitioning the data object is in accordance with a data segmentation approach. 
     
     
         11 . The dispersed storage processing unit of  claim 10 , wherein the data segmentation approach includes any of: fixed size, segment size ramping-up, or segment size ramping-down. 
     
     
         12 . The dispersed storage processing unit of  claim 8  further comprises, for each storage unit of a selected storage set, issuing a write slice request to the storage unit, where the write slice request includes an associated metadata slice of the set of metadata slices and a plurality of associated encoded data slices of each set of encoded data slices. 
     
     
         13 . The dispersed storage processing unit of  claim 12 , wherein the plurality of associated encoded data slices of each set of encoded data slices includes a common pillar. 
     
     
         14 . The dispersed storage processing unit of  claim 13 , wherein the common pillar includes a set of storage units equal in number to a width of the common pillar. 
     
     
         15 . A storage unit in a dispersed storage network (DSN) comprises:
 a grouping de-selector facilitating retrieval of a set of encoded metadata slices and a plurality of sets of encoded data slices from one or more storage units of the DSN;   a first dispersed storage error decoder decoding the set of encoded metadata slices to produce metadata, wherein the first dispersed storage error decoder utilizes first dispersal parameters including a decode threshold of 1;   the first dispersed storage error decoder further decoding at least a first set of encoded data slices to produce at least a first data segment of a plurality of data segments, wherein the first disperse storage error decoder utilizes the first dispersal parameters of the set of encoded metadata slices including a decode threshold of 1;   a second dispersed storage error decoder decoding additional ones of the plurality of sets of encoded data slices data segments to produce a plurality of additional data segments, wherein the additional data segments do not include the at least a first data segment and wherein the second disperse storage error decoder utilizes second dispersal parameters, the second dispersal parameters different from the first dispersal parameters and including a decode threshold greater than 1; and   a data de-partitioner combining the at least a first data segment with the plurality of additional data segments to produce a data object; and   wherein accessing of the set of encoded metadata slices reproduces the metadata and the first data segment of the data object, making the first data segment of the data object available for use by the one or more computing devices while continuing to retrieve the additional data segments from the DSN.   
     
     
         16 . The storage unit of  claim 15 , wherein the combining is in accordance with a data de-segmentation approach. 
     
     
         17 . The storage unit of  claim 16 , wherein the data de-segmentation approach includes any of: fixed size, segment size ramping-up, or segment size ramping-down. 
     
     
         18 . The storage unit of  claim 17  further comprises, for each storage unit of a selected storage set, issuing a read slice request to the storage unit, where the read slice request includes an retrieving an associated metadata slice of the set of metadata slices and a plurality of associated encoded data slices of each set of encoded data slices. 
     
     
         19 . The storage unit of  claim 18 , wherein the plurality of associated encoded data slices of each set of encoded data slices includes a common pillar. 
     
     
         20 . The storage unit of  claim 15 , wherein the first dispersed storage error decoder utilizes the first dispersal parameters to retrieve a copy of the encoded metadata slices from each storage unit.