Optimized neural network input stride method and apparatus

Abstract

A convolutional layer in a convolutional neural network uses a predetermined horizontal input stride and a predetermined vertical input stride that are greater than 1 while the hardware forming the convolutional layer operates using an input stride of 1. Each original weight kernel of a plurality of sets of original weight kernels is subdivided based on the predetermined horizontal and vertical input strides to form a set of a plurality of sub-kernels for each set of original weight kernels. Each of a plurality of IFMs is subdivided based on the predetermined horizontal and vertical input strides to form a plurality of sub-maps. Each sub-map is convolved by the corresponding sub-kernel for a set of original weight kernels using an input stride of 1. A convolved result of each sub-map and the corresponding sub-kernel is summed to form an output feature map.

What is claimed is: 
     
         1 . A neural network, comprising a convolutional layer that is configured to receive a plurality of sub-maps for each input feature map (IFM) of a plurality of IFMs and configured to convolve each sub-map by a corresponding sub-kernel using a horizontal input stride equal to 1 and a vertical input stride equal to 1, each sub-kernel being subdivided from an original weight kernel of a plurality of sets of original weight kernels based on a predetermined horizonal input stride that is greater than 1 and a predetermined vertical input stride that is greater than 1. 
     
     
         2 . The neural network of  claim 1 , wherein the convolutional layer is further configured to sum a convolved result of each sub-map and the corresponding sub-kernel to form an output feature map (OFM) corresponding to a set of original weight kernels for each set of original weight kernels. 
     
     
         3 . The neural network of  claim 2 , further comprising a weight-kernel subdivider configured to subdivide each original weight kernel of the plurality of sets of original weight kernels based on the predetermined horizontal input stride and the predetermined vertical input stride to form sub-kernels for each original weight kernel. 
     
     
         4 . The neural network of  claim 3 , wherein a weight in a sub-kernel comprises a weight in the original weight kernel from which the sub-kernel was subdivided based on a modulo of the predetermined horizontal input stride and based on a modulo of the predetermined vertical input stride of a position of the weight in the original weight kernel. 
     
     
         5 . The neural network of  claim 3 , further comprising an IFM subdivider configured to subdivide each of the plurality of IFMs based on the predetermined horizontal input stride and the predetermined vertical input stride to form the plurality of sub-maps for each IFM of the plurality of IFMs. 
     
     
         6 . The neural network of  claim 5 , wherein the predetermined horizontal input stride and the predetermined vertical input stride are equal to a predetermined second value,
 wherein the weight-kernel subdivider is further configured to subdivide each original weight kernel of a plurality of sets of original weight kernels by a square of the predetermined second value to form a plurality of sub-kernels for each set of original weight kernels, and   wherein the IFM subdivider is further configured to subdivide each of the plurality of IFMs based on the square of the predetermined second value to form the plurality of sub-maps for each IFM of the plurality of IFMs.   
     
     
         7 . The neural network of  claim 5 , further comprising a domain converter coupled to the weight-kernel subdivider and the IFM subdivider, the domain converter being configured to convert elements of each sub-map into a Winograd domain. 
     
     
         8 . The neural network of  claim 1 , wherein the predetermined horizontal input stride is 2 and the predetermined vertical input stride is 2, or
 wherein the predetermined horizontal input stride is 3 and the predetermined vertical input stride is 3.   
     
     
         9 . A neural network, comprising a convolutional layer that is configured to receive a plurality of sub-maps for each input feature map (IFM) of a plurality of IFMs, the plurality of sub-maps for each IFM comprising an increased dimensionality with respect to a corresponding IFM, the convolutional layer being further configured to convolve each sub-map by a corresponding sub-kernel using a horizontal input stride equal to 1 and a vertical input stride equal to 1, each sub-kernel being subdivided from an original weight kernel of a plurality of sets of original weight kernels based on a predetermined horizonal input stride that is greater than 1 and a predetermined vertical input stride that is greater than 1 and each sub-kernel comprising an increased dimensionality with respect to a corresponding original weight kernel. 
     
     
         10 . The neural network of  claim 9 , wherein the convolutional layer is further configured to sum a convolved result of each sub-map and the corresponding sub-kernel to form an output feature map (OFM) corresponding to a set of original weight kernels for each set of original weight kernels. 
     
     
         11 . The neural network of  claim 10 , further comprising a weight-kernel subdivider to subdivide each original weight kernel of the plurality of sets of original weight kernels based on the predetermined horizontal input stride and the predetermined vertical input stride to form sub-kernels for each set of original weight kernels. 
     
     
         12 . The neural network of  claim 11 , wherein a weight in a sub-kernel comprises a weight in the original weight kernel from which the sub-kernel was subdivided based on a modulo of the predetermined horizontal input stride and the predetermined vertical input stride of a position of the weight in the original weight kernel. 
     
     
         13 . The neural network of  claim 11 , further comprising an IFM subdivider configured to subdivide each of the plurality of IFMs based on the predetermined horizontal input stride and the predetermined vertical input stride to form the plurality of sub-maps for each IFM of the plurality of IFMs. 
     
     
         14 . The neural network of  claim 13 , wherein the predetermined horizontal input stride and the predetermined vertical input stride are equal to a predetermined second value,
 wherein the weight-kernel subdivider is further configured to subdivide each original weight kernel of a plurality of sets of original weight kernels by a square of the predetermined second value to form a plurality of sub-kernels for each set of original weight kernels, and   wherein the IFM subdivider is further configured to subdivide each of the plurality of IFMs based on the square of the predetermined second value to form the plurality of sub-maps for each IFM of the plurality of IFMs.   
     
     
         15 . The neural network of  claim 13 , further comprising a domain converter coupled to the weight-kernel subdivider and the IFM subdivider, the domain converter being configured to convert elements of each sub-map into a Winograd domain. 
     
     
         16 . The neural network of  claim 13 , wherein the predetermined horizontal input stride is 2, and the predetermined vertical input stride is 2, or
 wherein the predetermined horizontal input stride is 3, and the predetermined vertical input stride is 3.   
     
     
         17 . A method to form at least one output feature map (OFM) from at least one input feature map (IFM) at a convolutional layer in a neural network, the method comprising:
 receiving, by the convolutional layer, a plurality of sub-maps for each input feature map (IFM) of a plurality of IFMs, and   convolving, by the convolutional layer, each sub-map by a corresponding sub-kernel using a horizontal input stride equal to 1 and a vertical input stride equal to 1, each sub-kernel being subdivided from an original weight kernel of a plurality of sets of original weight kernels based on a predetermined horizontal input stride that is greater than 1 and a predetermined vertical input stride that is greater than 1.   
     
     
         18 . The method of  claim 17 , further comprising summing, by the convolutional layer, a convolved result of each sub-map and the corresponding sub-kernel to form an output feature map (OFM) based on a set of original weight kernels for each set of original weight kernels. 
     
     
         19 . The method of  claim 17 , further comprising:
 subdividing each original weight kernel of the plurality of sets of the original weight kernels based on the predetermined horizontal input stride and the predetermined vertical input stride to form sub-kernels for each original weight kernel in a set of original weight kernels; and   subdividing each of the plurality of IFMs based on the predetermined horizontal input stride and the predetermined vertical input stride to form the plurality of sub-maps for each IFM of the plurality of IFMs, the plurality of IFMs corresponding to the convolutional layer, and each sub-map corresponding to the sub-kernel.   
     
     
         20 . The method of  claim 17 , wherein a weight in a sub-kernel comprises a weight in the original weight kernel from which the sub-kernel was subdivided based on a modulo of the predetermined horizontal input stride and on a modulo of the predetermined vertical input stride of a position of the weight in the original weight kernel,
 the method further comprising converting elements of each sub-map into a Winograd domain before convolving each sub-map.