Desparsified convolution for sparse tensors

Abstract

Certain aspects of the present disclosure provide techniques for desparsified convolution. A weight tensor having unstructured sparsity is accessed, and a densified weight tensor is generated based on the weight tensor by directionally squeezing the weight tensor to remove sparse values, and generating a sparsity map based on the directional squeezing. The densified weight tensor and sparsity map are output for use in a convolutional neural network.

What is claimed is: 
     
         1 . A processor-implemented method, comprising:
 accessing a weight tensor having unstructured sparsity;   generating a densified weight tensor based on the weight tensor, comprising:
 directionally squeezing the weight tensor to remove zero-value elements; and 
 generating a sparsity map based on the directional squeezing; and 
   outputting the densified weight tensor and sparsity map for use in a convolutional neural network.   
     
     
         2 . The processor-implemented method of  claim 1 , wherein outputting the densified weight tensor and sparsity map comprises compiling the convolutional neural network based at least in part on the densified weight tensor. 
     
     
         3 . The processor-implemented method of  claim 2 , wherein compiling the convolutional neural network comprises setting a sparsity indicator to indicate that the compiled convolutional neural network includes the densified weight tensor. 
     
     
         4 . The processor-implemented method of  claim 1 , wherein directionally squeezing the weight tensor comprises:
 removing zero-value elements from the weight tensor; and   compressing remaining elements of the weight tensor along one dimension in the weight tensor.   
     
     
         5 . The processor-implemented method of  claim 1 , wherein directionally squeezing the weight tensor comprises:
 selecting either a vertical direction or a horizontal direction; and   squeezing the weight tensor in the selected direction.   
     
     
         6 . The processor-implemented method of  claim 1 , wherein:
 a dimensionality of the sparsity map matches a dimensionality of the densified weight tensor; and   the sparsity map indicates associations between elements in the densified weight tensor and corresponding output elements for the densified weight tensor during convolution.   
     
     
         7 . The processor-implemented method of  claim 6 , wherein the sparsity map indicates the associations using, for each element in the densified weight tensor, an absolute indication of the corresponding output element. 
     
     
         8 . The processor-implemented method of  claim 6 , wherein the sparsity map indicates the associations using, for each respective element in the densified weight tensor, a relative spacing indicating a number of output elements to skip between the respective element and a subsequent element in the densified weight tensor. 
     
     
         9 . The processor-implemented method of  claim 1 , wherein generating the densified weight tensor is performed in response to determining that the unstructured sparsity in the weight tensor satisfies one or more defined thresholds. 
     
     
         10 . A processor-implemented method, comprising:
 accessing an activation tensor for processing using a convolution operation; and   performing the convolution operation, comprising:
 retrieving a densified weight tensor for the convolution operation; 
 retrieving a sparsity map associated with the densified weight tensor; 
 generating a set of intermediate tensors by multiplying the densified weight tensor and the activation tensor; and 
 generating an output tensor for the convolution operation by accumulating the set of intermediate tensors based on the sparsity map. 
   
     
     
         11 . The processor-implemented method of  claim 10 , wherein:
 the densified weight tensor was generated by directionally squeezing a weight tensor, having unstructured sparsity, to remove zero-value elements, and   the sparsity map indicates associations between elements in the densified weight tensor and corresponding output elements in the output tensor.   
     
     
         12 . The processor-implemented method of  claim 11 , wherein the sparsity map indicates the associations using, for each element in the densified weight tensor, an absolute indication of the corresponding output element. 
     
     
         13 . The processor-implemented method of  claim 11 , wherein the sparsity map indicates the associations using, for each respective element in the densified weight tensor, a relative spacing indicating a number of output elements to skip between the respective element and a subsequent element in the output tensor. 
     
     
         14 . The processor-implemented method of  claim 10 , wherein generating the output tensor comprises, for each respective output element in the output tensor:
 identifying a respective set of elements, from the set of intermediate tensors, that correspond to the respective output element based on the sparsity map; and   accumulating the respective set of elements.   
     
     
         15 . The processor-implemented method of  claim 10 , wherein accumulating the set of intermediate tensors based on the sparsity map is performed in response to determining that a sparsity indicator associated with the convolution operation indicates that at least one densified weight tensor is used in the convolution operation. 
     
     
         16 . A system, comprising:
 a memory comprising computer-executable instructions; and   one or more processors configured to execute the computer-executable instructions and cause the system to perform an operation comprising:
 accessing a weight tensor having unstructured sparsity; 
 generating a densified weight tensor based on the weight tensor, comprising:
 directionally squeezing the weight tensor to remove zero-value elements; and 
 generating a sparsity map based on the directional squeezing; and 
 
 outputting the densified weight tensor and sparsity map for use in a convolutional neural network. 
   
     
     
         17 . The system of  claim 16 , wherein outputting the densified weight tensor and sparsity map comprises compiling the convolutional neural network based at least in part on the densified weight tensor. 
     
     
         18 . The system of  claim 17 , wherein compiling the convolutional neural network comprises setting a sparsity indicator to indicate that the compiled convolutional neural network includes the densified weight tensor. 
     
     
         19 . The system of  claim 16 , wherein directionally squeezing the weight tensor comprises:
 removing zero-value elements from the weight tensor; and   compressing remaining elements of the weight tensor along one dimension in the weight tensor.   
     
     
         20 . The system of  claim 16 , wherein directionally squeezing the weight tensor comprises:
 selecting either a vertical direction or a horizontal direction; and   squeezing the weight tensor in the selected direction.   
     
     
         21 . The system of  claim 16 , wherein:
 a dimensionality of the sparsity map matches a dimensionality of the densified weight tensor; and   the sparsity map indicates associations between elements in the densified weight tensor and corresponding output elements for the densified weight tensor during convolution.   
     
     
         22 . The system of  claim 21 , wherein the sparsity map indicates the associations using, for each element in the densified weight tensor, an absolute indication of the corresponding output element. 
     
     
         23 . The system of  claim 21 , wherein the sparsity map indicates the associations using, for each respective element in the densified weight tensor, a relative spacing indicating a number of output elements to skip between the respective element and a subsequent element in the densified weight tensor. 
     
     
         24 . The system of  claim 16 , wherein generating the densified weight tensor is performed in response to determining that the unstructured sparsity in the weight tensor satisfies one or more defined thresholds. 
     
     
         25 . One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors of a processing system, cause the processing system to perform an operation comprising:
 accessing an activation tensor for processing using a convolution operation; and   performing the convolution operation, comprising:
 retrieving a densified weight tensor for the convolution operation; 
 retrieving a sparsity map associated with the densified weight tensor; 
 generating a set of intermediate tensors by multiplying the densified weight tensor and the activation tensor; and 
   generating an output tensor for the convolution operation by accumulating the set of intermediate tensors based on the sparsity map.   
     
     
         26 . The one or more non-transitory computer-readable media of  claim 25 , wherein:
 the densified weight tensor was generated by directionally squeezing a weight tensor, having unstructured sparsity, to remove zero-value elements, and   the sparsity map indicates associations between elements in the densified weight tensor and corresponding output elements in the output tensor.   
     
     
         27 . The one or more non-transitory computer-readable media of  claim 26 , wherein the sparsity map indicates the associations using, for each element in the densified weight tensor, an absolute indication of the corresponding output element. 
     
     
         28 . The one or more non-transitory computer-readable media of  claim 26 , wherein the sparsity map indicates the associations using, for each respective element in the densified weight tensor, a relative spacing indicating a number of output elements to skip between the respective element and a subsequent element in the output tensor. 
     
     
         29 . The one or more non-transitory computer-readable media of  claim 25 , wherein generating the output tensor comprises, for each respective output element in the output tensor:
 identifying a respective set of elements, from the set of intermediate tensors, that correspond to the respective output element based on the sparsity map; and   accumulating the respective set of elements.   
     
     
         30 . The one or more non-transitory computer-readable media of  claim 25 , wherein accumulating the set of intermediate tensors based on the sparsity map is performed in response to determining that a sparsity indicator associated with the convolution operation indicates that at least one densified weight tensor is used in the convolution operation.