Accelerator for sparse-dense matrix multiplication

Abstract

Disclosed embodiments relate to an accelerator for sparse-dense matrix instructions. In one example, a processor to execute a sparse-dense matrix multiplication instruction, includes fetch circuitry to fetch the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one, decode circuitry to decode the fetched sparse-dense matrix multiplication instruction, execution circuitry to execute the decoded sparse-dense matrix multiplication instruction to, for each non-zero element at row M and column K of the specified sparse source matrix generate a product of the non-zero element and each corresponding dense element at row K and column N of the specified dense source matrix, and generate an accumulated sum of each generated product and a previous value of a corresponding output element at row M and column N of the specified dense output matrix.

What is claimed is: 
     
       1. A processor comprising:
 a cache configured to store data; 
 a plurality of cores coupled to the cache, a core of the plurality of cores comprising:
 execution circuitry configured to perform multiply-accumulate operations with a first source matrix and a second source matrix to generate a result matrix responsive to an instruction, wherein the first source matrix is a sparse matrix having non-zero data elements located at certain positions, 
 wherein the first source matrix is stored in a compressed format that identifies the positions of the non-zero data elements in the first source matrix, the execution circuitry further comprising: 
 a plurality of multiply-accumulate circuits configured to perform the multiply-accumulate operations, wherein the multiply-accumulate operations comprise a plurality of fused multiply-add operations to multiply the non-zero data elements of the first source matrix by corresponding data elements of the second source matrix identified based on the positions in the compressed format to generate a plurality of products, and to add the plurality of products to accumulated values to generate data elements of the result matrix. 
 
 
     
     
       2. The processor of  claim 1  wherein the execution circuitry is further configured to use each position of a non-zero data element in the first source matrix to identify a row and/or column in the second source matrix, wherein the plurality of multiply-accumulate circuits multiply the non-zero data element in the first source matrix by each data element in the identified row and/or column in the second matrix, respectively. 
     
     
       3. The processor of  claim 1  wherein the second source matrix comprises a dense matrix. 
     
     
       4. The processor of  claim 1  wherein the compressed sparse matrix format comprises a compressed sparse row (CSR) format or a compressed sparse column (CSC) format. 
     
     
       5. The processor of  claim 3  further comprising an instruction fetch circuit to fetch the instruction and a decoder to decode the instruction. 
     
     
       6. The processor of  claim 5  wherein the instruction comprises a plurality of fields including a first field to specify an opcode, a second field to identify the result matrix, a third field to identify the first source matrix, and a fourth field to identify the second source matrix. 
     
     
       7. The processor of  claim 1  wherein the first source matrix and the second source matrix comprise a machine learning activation matrix and/or a weight vector. 
     
     
       8. A processor to execute a sparse-dense matrix multiplication (SDMM) instruction, comprising:
 fetch circuitry to fetch, from memory, the SDMM instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements below a sparsity threshold; 
 decode circuitry to decode the fetched SDMM instruction; and 
 execution circuitry, responsive to the decoded SDMM instruction to, for each non-zero element at row M and column K of the specified sparse source matrix:
 generate a product of the non-zero element and each corresponding dense element at row K and column N of the specified dense source matrix; and 
 generate an accumulated sum of each generated product and a previous value of a corresponding output element at row M and column N of the specified dense output matrix 
 
 wherein the SDMM instruction further specifies a format of data elements of the specified sparse source matrix, the specified dense source matrix, and the specified dense output matrix, the format being one of fixed-point, single-precision floating point, double-precision floating point, extended precision floating point, and double-extended precision floating point; 
 a memory read circuit to read the specified sparse source matrix and the specified dense source matrix, and write results to a memory, wherein the specified sparse source matrix is stored in the memory in a compressed format, the compressed format including only the non-zero elements of the specified sparse source matrix, each non-zero element being represented by a data value and a matrix location, and wherein the memory read circuit reads the specified sparse source matrix in the compressed format. 
 
     
     
       9. The processor of  claim 8 , wherein the SDMM instruction further specifies a size of data elements of the specified sparse source matrix, the specified dense source matrix, and the specified dense output matrix, the size being one of 8 bits, 16 bits, 32 bits, 64 bits, and 128 bits, the size being specified either as an operand of the instruction or as part of the opcode.