Tensor Circuits And Methods For Multiplying With Sparse Weights

Abstract

A tensor circuit includes first storage circuits coupled to store first activation values from an activation matrix, second storage circuits coupled to store second activation values from the activation matrix, multiplexer circuits configurable to output a subset of the first and the second activation values stored in the first and the second storage circuits, multiplier circuits coupled to multiply weight values from a sparse weight matrix by the subset of the first and the second activation values output by the multiplexer circuits to generate products, and a summation circuit coupled to sum the products.

What is claimed is: 
     
         1 . A tensor circuit comprising:
 first storage circuits coupled to store first activation values from an activation matrix;   second storage circuits coupled to store second activation values from the activation matrix;   first multiplexer circuits configurable to output a subset of the first and the second activation values stored in the first and the second storage circuits;   multiplier circuits coupled to multiply first weight values from a sparse weight matrix by the subset of the first and the second activation values output by the first multiplexer circuits to generate products; and   a summation circuit coupled to sum the products.   
     
     
         2 . The tensor circuit of  claim 1 , wherein the first multiplexer circuits are configured based on sparsity indices that indicate the sparsity of the first weight values from the sparse weight matrix. 
     
     
         3 . The tensor circuit of  claim 1 , wherein the multiplier circuits multiply a first subset of the first weight values by a subset of the first activation values stored in the first storage circuits to generate a first subset of the products concurrently with the second activation values being loaded into the second storage circuits. 
     
     
         4 . The tensor circuit of  claim 3 , wherein the multiplier circuits multiply a second subset of the first weight values by a subset of the second activation values stored in the second storage circuits to generate a second subset of the products concurrently with third activation values being loaded into the first storage circuits. 
     
     
         5 . The tensor circuit of  claim 1  further comprising:
 second multiplexer circuits configurable to output the first activation values stored in the first storage circuits or the second activation values stored in the second storage circuits to the first multiplexer circuits. 
 
     
     
         6 . The tensor circuit of  claim 1 , wherein the first weight values are streamed to inputs of the multiplier circuits during a structured sparsity mode without being stored within the tensor circuit. 
     
     
         7 . The tensor circuit of  claim 1 , wherein the first storage circuits store second weight values from a dense weight matrix during a dense mode, the second storage circuits store third weight values from the dense matrix during the dense mode, the first multiplexer circuits are configurable to output a subset of the second and the third weight values during the dense mode, wherein third activation values are streamed into the tensor circuit, and the multiplier circuits multiply the third activation values by the subset of the second and the third weight values output by the first multiplexer circuits to generate additional products. 
     
     
         8 . The tensor circuit of  claim 1 , wherein the first storage circuits are first register circuits coupled in series, wherein the second storage circuits are second register circuits coupled in series, wherein the tensor circuit is configurable to load a first half of a single set of activations into the first storage circuits and a second half of the single set of the activations into the second storage circuits, and wherein the tensor circuit is further configurable to load the single set of the activations into both the first and the second storage circuits in an alternating sequence. 
     
     
         9 . The tensor circuit of  claim 1 , wherein the first and the second storage circuits comprise multiple sets of registers to store individual sets of activations, and wherein the tensor circuit processes a first one of the sets of the activations received from a first one of the sets of the registers unimpeded while a second one of the sets of the registers is loaded with a second one of the sets of the activations. 
     
     
         10 . A method for multiplying weight values from a sparse weight matrix by first and second activation values from an activation matrix, the method comprising:
 loading the first activation values from the activation matrix into first storage circuits;   loading the second activation values from the activation matrix into second storage circuits;   configuring first multiplexer circuits to select a subset of the first and the second activation values stored in the first and the second storage circuits;   multiplying the weight values from the sparse weight matrix by the subset of the first and the second activation values selected by the first multiplexer circuits using multiplier circuits to generate products; and   summing the products using a summation circuit.   
     
     
         11 . The method of  claim 10  further comprising:
 configuring the first multiplexer circuits using sparsity indices that indicate the sparsity of the weight values from the sparse weight matrix. 
 
     
     
         12 . The method of  claim 10  further comprising:
 configuring second multiplexer circuits to provide the first activation values stored in the first storage circuits to the first multiplexer circuits during a first period of time; and 
 configuring the second multiplexer circuits to provide the second activation values stored in the second storage circuits to the first multiplexer circuits during a second period of time after the first period of time. 
 
     
     
         13 . The method of  claim 10  further comprising:
 providing the weight values to inputs of the multiplier circuits during a structured sparsity mode without storing the weight values in response to a clock signal that clocks the first and the second storage circuits. 
 
     
     
         14 . The method of  claim 10 , wherein the multiplier circuits stops multiplying values while one set of the first and the second activation values are loaded into the first and the second storage circuits. 
     
     
         15 . The method of  claim 10 , wherein multiplying the weight values by the subset of the first and the second activation values using the multiplier circuits to generate the products further comprises multiplying the weight values by a subset of the first activation values stored in the first storage circuits to generate the products concurrently with the second activation values being loaded into the second storage circuits. 
     
     
         16 . An integrated circuit comprising:
 a first bank of first register circuits coupled in series to store first activation values from an activation matrix;   a second bank of second register circuits coupled in series to store second activation values from the activation matrix;   first multiplexer circuits configurable to select a subset of the first and the second activation values; and   multiplier circuits that multiply sparse weight values from a sparse weight matrix by the subset of the first and the second activation values selected by the first multiplexer circuits to generate products.   
     
     
         17 . The integrated circuit of  claim 16  further comprising:
 a summation circuit that sums the products received from the multiplier circuits to generate a sum. 
 
     
     
         18 . The integrated circuit of  claim 16 , wherein the first multiplexer circuits receive sparsity indices at select inputs that indicate sparsity of the sparse weight values from the sparse weight matrix. 
     
     
         19 . The integrated circuit of  claim 16  further comprising:
 second multiplexer circuits configurable to provide the first activation values stored in the first bank of the first register circuits to the first multiplexer circuits during a first period of time and the second activation values stored in the second bank of the second register circuits to the first multiplexer circuits during a second period of time after the first period of time. 
 
     
     
         20 . The integrated circuit of  claim 16 , wherein the multiplier circuits multiply the sparse weight values by a subset of the first activation values stored in the first bank of the first register circuits to generate the products concurrently with the second activation values being loaded into the second bank of the second register circuits.