Method and system for bit quantization of artificial neural network

Abstract

The present disclosure provides a method for bit quantization of an artificial neural network. This method may comprise: (a) a step of selecting one parameter or one parameter group to be quantized in an artificial neural network; (b) a bit quantization step of reducing the size of data representation for the selected parameter or parameter group to bits; (c) a step of determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; and (d) a step of, when the accuracy of the artificial neural network is greater than or equal to the target value, repeatedly performing said step (a) to step (c).

1 . A hardware for an artificial neural network, comprising:
 a memory configured to store element values of a weight kernel or element values of a feature map of the artificial neural network in which all parameters or all parameter groups are quantized, wherein the artificial neural network is quantized by:   selecting at least one parameter from the parameters or at least one parameter group from the parameter groups,   executing bit quantization to reduce a size of a data representation for the selected at least one parameter or the selected at least one parameter group to a unit of bits,   determining whether accuracy of the artificial neural network according to the bit quantization applied to the selected at least one parameter or the selected at least one parameter group is greater than or equal to a target value, and   responsive to the accuracy of the artificial neural network being greater than or equal to the target value, repeatedly executing the bit quantization; and   a processing unit for performing convolution, including a plurality of multipliers or a plurality of adders designed to have a bit size corresponding to a number of quantization bits of the quantized artificial neural network.   
     
     
         2 . The hardware of  claim 1 , wherein in the quantized artificial neural network, the at least one parameter or the at least one parameter group is sequentially quantized based on an amount of computation or an amount of memory. 
     
     
         3 . The hardware of  claim 1 , wherein the processing unit is further configured to process the quantized artificial neural network by at least one of a computational cost bit quantization method, a forward bit quantization method, or a backward bit quantization method. 
     
     
         4 . The hardware of  claim 2 , wherein the amount of computation and the amount of memory of the quantized artificial neural network are relatively reduced compared to those before quantization, and a number of bits of each data of the at least one parameter or the at least one parameter group stored in the memory is reduced. 
     
     
         5 . The hardware of  claim 1 , wherein the memory includes at least one of a buffer memory, a register memory, or a cache memory. 
     
     
         6 . The hardware of  claim 1 , wherein the quantized artificial neural network includes a plurality of layers, wherein a size of a data bit of a data path through which data of a specific layer among the plurality of layers is transmitted is reduced in a unit of bits. 
     
     
         7 . The hardware of  claim 1 , wherein in the quantized artificial neural network, bit quantization is executed to reduce a storage size of the memory configured to store the at least one parameter or the at least one parameter group. 
     
     
         8 . The hardware of  claim 1 , wherein the memory further includes at least one of a weight kernel cache or an input feature map cache. 
     
     
         9 . The hardware of  claim 1 , wherein the processing unit further includes a tree adder configured to sum result values of elementwise multiplication by the plurality of multipliers. 
     
     
         10 . The hardware of  claim 1 , further comprising: an adder connected to the processing unit and an accumulator connected to the adder. 
     
     
         11 . The hardware of  claim 1 , further comprising: an output activation map cache configured to store a result value of convolution of the processing unit. 
     
     
         12 . The hardware of  claim 1 , wherein the processing unit further includes a plurality of convolution processing units. 
     
     
         13 . The hardware of  claim 12 , wherein the processing unit further includes a tree adder configured to sum result values of convolution of each of the plurality of convolution processing units. 
     
     
         14 . A method for quantizing bits of a multi-layered artificial neural network having a plurality of layers, executed by a system, the method comprising:
 selecting at least one layer from among the plurality of layers in an order of having a large amount of memory or a small amount of memory;   bit quantizing to reduce a size of a data representation for a parameter of the selected layer to a unit of bits;   determining whether accuracy of the multi-layered artificial neural network after the bit quantizing is greater than or equal to a target value; and   executing the bit quantizing responsive to an accuracy of the artificial neural network being greater than or equal to the target value.   
     
     
         15 . The method of  claim 14 , further comprising: determining the size of the data representation for the parameter of the selected layer that satisfies the accuracy greater than or equal to the target value as a final number of bits for the parameter of the selected layer, responsive to the accuracy of the artificial neural network being less than the target value. 
     
     
         16 . The method of  claim 15 , further comprising:
 selecting at least one layer in which the final number of bits for the parameter is not determined among the plurality of layers, and repeatedly executing the bit quantizing to determine the final number of bits of the selected at least one layer in which the final number of bits is not determined.   
     
     
         17 . The method of  claim 14 , wherein the bit quantizing to reduce the size to the unit of bits is configured to reduce the size to a unit of 1 bit. 
     
     
         18 . The method of  claim 14 , wherein the parameter of the selected at least one layer includes at least one of weight data, feature map data, or activation map data. 
     
     
         19 . The method of  claim 14 , wherein a number of bits of a multiplier and an adder of the processing unit for processing the multi-layered artificial neural network, the bits of which are quantized, is designed to correspond to the number of bits according to a result of the bit quantizing. 
     
     
         20 . A convolutional multiplication processing apparatus, comprising:
 a memory configured to store a quantized artificial neural network, a weight kernel and a feature map of the quantized artificial neural network, wherein the quantized artificial neural network is quantized by:   selecting a parameter of at least one layer,   executing bit quantization to reduce a size of a data representation for the selected parameter of the at least one layer to a unit of bits,   determining whether accuracy of the artificial neural network according to the bit quantization applied to the selected at least one parameter of the at least one layer or at least one parameter group is greater than or equal to a target value, and   responsive to the accuracy of the artificial neural network being greater than or equal to the target value, repeatedly executing the bit quantization; and   a plurality of multipliers or a plurality of adders configured to process convolution by receiving the weight kernel and the feature map of the quantized artificial neural network, and designed to have a bit size corresponding to a number of quantization bits of the quantized artificial neural network.