Neural network accelerator with systolic array structure

Abstract

A neural network accelerator in which processing elements are configured in a systolic array structure includes a memory to store a plurality of feature data including first and second feature data and a plurality of kernel data including first and second kernel data, a first processing element to perform an operation based on the first feature data and the first kernel data and output the first feature data, a selection circuit to select one of the first feature data and the second feature data, based on a control signal, and output the selected feature data, a second processing element to perform an operation based on the selected feature data and one of the first and the second kernel data, and a controller to generate the control signal, based on a neural network characteristic associated with the plurality of feature data and kernel data.

What is claimed is: 
     
         1 . A neural network accelerator comprising:
 a memory configured to store a plurality of feature data including first feature data and second feature data and a plurality of kernel data including first kernel data and second kernel data;   a first processing element configured to perform an operation based on the first feature data and the first kernel data and output the first feature data;   a selection circuit configured to select one of the first feature data output from the first processing element and the second feature data output from the memory, based on a control signal, and output the selected feature data;   a second processing element configured to perform an operation based on the selected feature data and one of the first kernel data and the second kernel data; and   a controller configured to generate the control signal, based on a neural network characteristic associated with the plurality of feature data and the plurality of kernel data.   
     
     
         2 . The neural network accelerator of  claim 1 , wherein, when the plurality of feature data are represented as a first matrix and the plurality of kernel data are represented as a second matrix, the neural network characteristic includes size information of the first matrix and size information of the second matrix. 
     
     
         3 . The neural network accelerator of  claim 1 , wherein, when the first feature data are selected from the selection circuit, the second processing element is configured to perform an operation based on the first feature data and the second kernel data. 
     
     
         4 . The neural network accelerator of  claim 1 , wherein, when the second feature data are selected from the selection circuit, the second processing element is configured to perform an operation based on the second feature data and one of the first kernel data and the second kernel data. 
     
     
         5 . The neural network accelerator of  claim 1 , wherein the memory is positioned between the first processing element and the second processing element. 
     
     
         6 . The neural network accelerator of  claim 1 , wherein a first operation result generated by the first processing element and a second operation result generated by the second processing element are stored in the memory. 
     
     
         7 . The neural network accelerator of  claim 1 , wherein the first processing element and the second processing element form a systolic array structure. 
     
     
         8 . A neural network accelerator comprising:
 a memory configured to store a plurality of input data including first input data and second input data;   a processing element array including a first processing element configured to perform an operation based on the first input data, and a second processing element configured to perform an operation based on a selected one of the first input data output from the first processing element and the second input data output from the memory; and   a controller configured to select input data to be operated in the second processing element, based on a neural network characteristic associated with the plurality of input data.   
     
     
         9 . The neural network accelerator of  claim 8 , wherein the neural network characteristic includes matrix size information that is made from the plurality of input data. 
     
     
         10 . The neural network accelerator of  claim 8 , wherein the first input data includes first feature data, and the second input data includes second feature data. 
     
     
         11 . The neural network accelerator of  claim 10 , wherein the first processing element performs an operation based on the first feature data and kernel data transferred to the first processing element, and the second processing element performs an operation based on one of the first feature data and the second feature data, and kernel data transferred to the second processing element. 
     
     
         12 . The neural network accelerator of  claim 8 , further comprising:
 a selection circuit configured to select one of the first input data output from the first processing element and the second input data output from the memory, based on a control signal from the controller, and provide the second processing element with the selected input data.   
     
     
         13 . The neural network accelerator of  claim 12 , the processing element array includes:
 a first sub-array including the first processing element; and   a second sub-array including the second processing element.   
     
     
         14 . The neural network accelerator of  claim 13 , the selection circuit is positioned on a data path between the first sub-array and the second sub-array. 
     
     
         15 . The neural network accelerator of  claim 8 , the processing element array forms a systolic array structure.