Power savings for neural network architecture with zero activations during inference

Abstract

Embodiments are generally directed to providing power savings for a neural network architecture with zero activations during inference. An embodiment of an apparatus includes one or more processors including one or more processor cores; and a memory to store data for processing including neural network processing, wherein the apparatus to perform a fast clear operation to initialize activation buffers for a neural network by updating metadata to indicate zero values, the neural network including a plurality of layers, wherein the apparatus is to compare outputs for the neural network to the metadata values and to write an output to memory only if the output is non-zero.

1 - 20 . (canceled) 
     
     
         21 . An apparatus comprising:
 one or more processors including one or more processor cores; and   a memory to store data for processing including processing for a neural network, the neural network including a plurality of layers, each layer including a plurality of nodes;   wherein the apparatus is to perform calculation for the layers of the neural network, the calculation including application of activation values and weight values for the nodes of each layer;   wherein the apparatus is to identify weights or activations that have zero values for any node the one or more layers of the neural network; and   wherein the apparatus is to apply clock gating for operations of the nodes of one or more layers that include any weights or activations having zero values.   
     
     
         22 . The apparatus of  claim 21 , further comprising a systolic compute mechanism to provide the calculation for the layers of the neural network, wherein the systolic compute mechanism is to identify the weights or activations that have zero values for any node the one or more layers of the neural network. 
     
     
         23 . The apparatus of  claim 22 , wherein the systolic compute mechanism performs matrix multiplication in neural network calculation. 
     
     
         24 . The apparatus of  claim 21 , wherein clock gating includes eliminating an entire clock for a process. 
     
     
         25 . The apparatus of  claim 21 , wherein the apparatus is to apply the clock gating in neural network inference. 
     
     
         26 . The apparatus of  claim 25 , wherein the apparatus is further to apply the clock gating in neural network training. 
     
     
         27 . The apparatus of  claim 21 , wherein the apparatus is to perform normalization of blocks of nodes of the neural network during training of the neural network, the normalization to force one or more weights or activations for the nodes of the one or more layers to be zero. 
     
     
         28 . The apparatus of  claim 21 , wherein the one or more layers includes one or more hidden layers of the neural network. 
     
     
         29 . A non-transitory computer-readable storage medium having stored thereon data representing sequences of instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
 processing data for a neural network, the neural network including a plurality of layers, each layer including a plurality of nodes;   calculating values for the layers of the neural network, the calculation including application of activation values and weight values for the nodes of each layer;   identifying weights or activations that have zero values for any node the one or more layers of the neural network; and   applying clock gating for operations of the nodes of one or more layers that include any weights or activations having zero values.   
     
     
         30 . The storage medium of  claim 29 , wherein a systolic compute mechanism is to identify the weights or activations that have zero values for any node the one or more layers of the neural network. 
     
     
         31 . The storage medium of  claim 30 , wherein calculating values for the layers of the neural network includes the systolic compute mechanism performing matrix multiplication. 
     
     
         32 . The storage medium of  claim 29 , wherein applying the clock gating includes eliminating an entire clock for a process. 
     
     
         33 . The storage medium of  claim 29 , wherein applying the clock gating includes applying clock gating in neural network inference. 
     
     
         34 . The storage medium of  claim 33 , wherein applying the clock gating includes applying clock gating in neural network training. 
     
     
         35 . The storage medium of  claim 29 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
 performing normalization of blocks of nodes of the neural network during training of the neural network, the normalization to force one or more weights or activations for the nodes of the one or more layers to be zero.   
     
     
         36 . A processing system comprising:
 one or more processors including one or more graphical processing units (GPUs); and   a memory to store data for processing including processing for a neural network, the neural network including a plurality of layers, each layer including a plurality of nodes; and   wherein the one or more GPUs include a systolic compute mechanism to perform calculation for the layers of the neural network, the calculation including application of activation values and weight values for the nodes of each layer;   wherein the systolic compute mechanism is to identify weights or activations that have zero values for any node the one or more layers of the neural network; and   wherein the processing system is to apply clock gating for operations of the nodes of one or more layers that include any weights or activations having zero values.   
     
     
         37 . The processing system of  claim 36 , wherein the systolic compute mechanism performs matrix multiplication in neural network calculation. 
     
     
         38 . The processing system of  claim 36 , wherein applying clock gating includes eliminating an entire clock for a process. 
     
     
         39 . The processing system of  claim 36 , wherein applying clock gating includes applying the clock gating in neural network inference. 
     
     
         40 . The processing system of  claim 39 , wherein applying clock gating further includes applying the clock gating in neural network training.