Selectable parallel processing of dispersed storage error encoding

Abstract

A method includes a computing device determining error coding dispersal storage function parameters to be utilized in a dispersed storage error encoding process of a data object. The method further includes the computing device dividing the dispersed storage error encoding process into a plurality of dispersed storage error encoding tasks. The method further includes allocating, by the computing device, the plurality of dispersed storage error encoding tasks to a plurality of central processing units (CPUs) available to the computing device. The method further includes dispersed storage error encoding, by the CPUs, the data segments into the plurality of sets of encoded data slices in accordance with the dispersed storage error encoding tasks. The method further includes sending, by the computing device, the plurality of sets of encoded data slices to storage units of the DSN for storage therein.

What is claimed is: 
     
         1 . A method comprises:
 determining, by a computing device of a dispersed storage network (DSN), error coding dispersal storage function parameters, wherein the error coding dispersal storage function parameters are to be utilized in a dispersed storage error encoding process of a data object;   dividing, by the computing device, the dispersed storage error encoding into a plurality of dispersed storage error encoding tasks;   allocating, by the computing device, the plurality of dispersed storage error encoding tasks to a plurality of central processing units (CPUs) available to the computing device;   dispersed storage error encoding, by the plurality of CPUs, a plurality of data segments of the data object into the plurality of sets of encoded data slices in accordance with the plurality of dispersed storage error encoding tasks; and   sending, by the computing device, the plurality of sets of encoded data slices to storage units of the DSN for storage therein.   
     
     
         2 . The method of  claim 1  further comprises:
 determining a number of central processing units available for the dispersed storage error encoding process of the data object; and 
 selecting the plurality of CPUs from the number of central processing units. 
 
     
     
         3 . The method of  claim 1 , wherein dividing the dispersed storage error encoding process into the plurality of dispersed storage error encoding tasks comprises:
 creating a dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to be the dividing of the data object into the plurality of data segments.   
     
     
         4 . The method of  claim 1 , wherein dividing the dispersed storage error encoding process into the plurality of dispersed storage error encoding tasks comprises:
 creating a dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to generate a data matrix for a data segment of the plurality of data segments.   
     
     
         5 . The method of  claim 1 , wherein dividing the dispersed storage error encoding process into the plurality of dispersed storage error encoding tasks comprises:
 creating a first dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to perform a first matrix multiplication on a first portion of an encoded matrix and a first portion of a data matrix to produce one or more encoded data slices of a set of encoded data slices of the plurality of sets of encoded data slices; and   creating a second dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to perform a second matrix multiplication on a second portion of the encoded matrix and a second portion of the data matrix to produce another one or more encoded data slices of the set of encoded data slices.   
     
     
         6 . The method of  claim 1 , wherein dividing the dispersed storage error encoding process into the plurality of encoding tasks comprises:
 creating a first plurality of encoding sub-tasks of the plurality of dispersed storage error encoding tasks to perform a plurality of component level matrix multiplications on coefficients of an encoding matrix and data blocks of a data matrix to produce a plurality of matrix multiplied components; and   creating a second plurality of encoding sub-tasks of the plurality of dispersed storage error encoding tasks to combine plurality of matrix multiplied components into a set of encoded data slices of the plurality of sets of encoded data slices.   
     
     
         7 . The method of  claim 1 , wherein the dispersed storage error encoding process includes dividing the data object into a plurality of data segments and dispersed storage error encoding the plurality of the data segments into a plurality of sets of encoded data slices in accordance with the error coding dispersal storage function parameters. 
     
     
         8 . The method of  claim 1  further comprises:
 determining a performance associated with at least some of the plurality of CPUs; 
 ranking the at least some of the plurality of CPUs based on the performance; 
 allocating, based on the ranking, first tasks of the dispersed storage error encoding tasks associated with a read threshold number of encoded data slices of a set of encoded data slices of the plurality of sets of encoded data slices to a first subset of CPUs of the plurality of CPUs; and 
 allocating, based on the ranking, second tasks of the dispersed storage error encoding tasks associated with an additional number of encoded data slices of the set of encoded data slices to a second subset of CPUs of the plurality of CPUs. 
 
     
     
         9 . A computer readable memory comprises:
 a first memory section that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to:
 determine error coding dispersal storage function parameters, wherein the error coding dispersal storage function parameters are to be utilized in a dispersed storage error encoding process of a data object; 
 divide the dispersed storage error encoding into a plurality of dispersed storage error encoding tasks; 
 allocate the plurality of dispersed storage error encoding tasks to a plurality of central processing units (CPUs) available to the computing device; 
   a second memory section that stores operational instructions that, when executed by the plurality of CPUs, causes the plurality of CPUs to:
 dispersed storage error encode a plurality of data segments of the data object into the plurality of sets of encoded data slices in accordance with the plurality of dispersed storage error encoding tasks; and 
   a third memory section that stores operational instructions that, when executed by the computing device, causes the computing device to:
 send the plurality of sets of encoded data slices to storage units of the DSN for storage therein. 
   
     
     
         10 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:
 determine a number of central processing units available for the dispersed storage error encoding process of the data object; and   select the plurality of CPUs from the number of central processing units.   
     
     
         11 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of dispersed storage error encoding tasks by:
 creating a dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to be the dividing of the data object into the plurality of data segments.   
     
     
         12 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of dispersed storage error encoding tasks by:
 creating a dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to generate a data matrix for a data segment of the plurality of data segments.   
     
     
         13 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of dispersed storage error encoding tasks by:
 creating a first dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to perform a first matrix multiplication on a first portion of an encoded matrix and a first portion of a data matrix to produce one or more encoded data slices of a set of encoded data slices of the plurality of sets of encoded data slices; and   creating a second dispersed storage error encoding task of the plurality of dispersed storage error encoding tasks to perform a second matrix multiplication on a second portion of the encoded matrix and a second portion of the data matrix to produce another one or more encoded data slices of the set of encoded data slices.   
     
     
         14 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of encoding tasks by:
 creating a first plurality of encoding sub-tasks of the plurality of dispersed storage error encoding tasks to perform a plurality of component level matrix multiplications on coefficients of an encoding matrix and data blocks of a data matrix to produce a plurality of matrix multiplied components; and   creating a second plurality of encoding sub-tasks of the plurality of dispersed storage error encoding tasks to combine plurality of matrix multiplied components into a set of encoded data slices of the plurality of sets of encoded data slices.   
     
     
         15 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to perform the dispersed storage error encoding process by:
 dividing the data object into a plurality of data segments; and   dispersed storage error encoding the plurality of the data segments into a plurality of sets of encoded data slices in accordance with the error coding dispersal storage function parameters.   
     
     
         16 . The computer readable memory of  claim 9 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:
 determine a performance associated with at least some of the plurality of CPUs;   rank the at least some of the plurality of CPUs based on the performance;   allocate, based on the ranking, first tasks of the dispersed storage error encoding tasks associated with a read threshold number of encoded data slices of a set of encoded data slices of the plurality of sets of encoded data slices to a first subset of CPUs of the plurality of CPUs; and   allocate, based on the ranking, second tasks of the dispersed storage error encoding tasks associated with an additional number of encoded data slices of the set of encoded data slices to a second subset of CPUs of the plurality of CPUs.