Data management device for supporting high speed artificial neural network operation by using data caching based on data locality of artificial neural network

Abstract

Disclosed is a data cache or data management device for caching data between at least one processor and at least one memory, and supporting an artificial neural network (ANN) operation executed by the at least one processor. The data cache device or the data management device can comprise an internal controller for predicting the next data operation request on the basis of ANN data locality of the ANN operation. The internal controller monitors data operation requests associated with the ANN operation from among data operation requests actually made between the at least one processor and the at least one memory, thereby extracting the ANN data locality of the ANN operation.

1 . A data management device, comprising:
 a data order memory configured to store data-address order information; and   at least one buffer memory configured to cache data of an artificial neural network data operation; and   wherein the data management device is configured to:
 receive a first memory read request from a processor; 
 predict a second memory read request to be requested following the first memory read request by referring to the data-address order information; 
 cache a data corresponding to the second memory read request from at least one memory to the at least one buffer memory; 
 determine whether the second memory read request and a third memory read request generated by the processor are a same; and 
 provide the data corresponding to the second memory read request cached in the at least one buffer memory to the processor. 
   
     
     
         2 . The data management device of  claim 1 , further comprising:
 a read/write address generator module configured to generate the data-address order information.   
     
     
         3 . The data management device of  claim 1 , wherein the first memory read request is an artificial neural network data operation request. 
     
     
         4 . The data management device of  claim 1 , wherein the data-address order information is generated based on artificial neural network data locality. 
     
     
         5 . The data management device of  claim 1 , wherein the processor is one of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NNP), and a customized processor for artificial neural network data operation. 
     
     
         6 . The data management device of  claim 1 , wherein the data corresponding to the second memory read request is provided to the processor in response to the third memory read request. 
     
     
         7 . The data management device of  claim 1 , wherein the data corresponding to the second memory read request is transmitted to the at least one memory before receiving the third memory read request. 
     
     
         8 . The data management device of  claim 1 , further comprising:
 a check legitimate access submodule configured to check whether address information of the second memory read request and the third memory read request match.   
     
     
         9 . The data management device of  claim 1 , further comprising:
 a first interface circuit in communication with the processor; and   a second interface circuit in communication with the at least one memory.   
     
     
         10 . The data management device of  claim 1 , wherein a memory area for input data, a memory area for weights, and a memory area for feature maps are allocated to the at least one memory. 
     
     
         11 . A data management device, comprising:
 a data order memory configured to store data-address order information; and   wherein the data management device is configured to:
 receive a first memory read request from a processor; 
 predict a second memory read request to be requested following the first memory read request by referring to the data-address order information; and 
 control at least one memory storing a data corresponding to the second memory read request to maintain a ready state in which the second memory read request can be executed. 
   
     
     
         12 . The data management device of  claim 11 , further comprising:
 a read/write address generator module configured to generate the data-address order information.   
     
     
         13 . The data management device of  claim 11 , wherein the first memory read request is an artificial neural network data operation request. 
     
     
         14 . The data management device of  claim 11 , wherein the data-address order information is generated based on artificial neural network data locality. 
     
     
         15 . The data management device of  claim 11 , wherein the processor is one of central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NNP), and a customized processor for artificial neural network data operation. 
     
     
         16 . The data management device of  claim 11 , wherein the data management device is configured to:
 determine whether the second memory read request and a third memory read request generated by the processor are a same, and provide the data corresponding to the second memory read request the processor.   
     
     
         17 . The data management device of  claim 11 , wherein the data stored in the at least one memory includes at least one of a weight, an activation map, and a feature map. 
     
     
         18 . The data management device of  claim 11 , wherein the data includes data of a multi-layered artificial neural network and an output feature map from a first layer of the multi-layered artificial neural network is temporarily stored in the at least one memory, the output feature map is used as an input feature map of a second layer of the multi-layered artificial neural network. 
     
     
         19 . The data management device of  claim 11 , further comprising:
 at least one buffer memory configured to cache a data of artificial neural network data operations.   
     
     
         20 . An apparatus comprising:
 a data order memory storing data-address order information; and   a buffer memory configured to cache data of an artificial neural network data operation; and   wherein the apparatus is configured to:   receive a first memory read request through a first interface circuit;   predict a second memory read request to be requested following the first memory read request by referring to the data-address order information;   cache a data corresponding to the second memory read request from a memory to the buffer memory;   determine whether the second memory read request and a third memory read request generated by a processor are identical; and   transmit the cached data corresponding to the second memory read request in the buffer memory to the processor.