Network module and distribution method and apparatus, electronic device, and storage medium

Abstract

A distribution method includes: distributing, when multiple feature maps exist in an image processing model, to each feature map a neuron through which the feature map passes; filtrating, according to importance of neurons of multiple convolution layers in an image processing model, the neurons to obtain a first result; collecting, according to a position attribute of the each neuron in the first result, statistics on a scale of the feature map corresponding to each neuron to obtain a distribution relationship; the distribution relationship indicates a correspondence between the each feature map and the neuron through which the feature map passes; and distributing, according to the distribution relationship, to the each feature map the neuron through which the feature map passes.

The invention claimed is: 
     
       1. A distribution method, comprising:
 distributing, when multiple feature maps exist in an image processing model, to each feature map a neuron through which the feature map passes; 
 filtrating, according to importance of neurons of multiple convolution layers in an image processing model, the neurons to obtain a first result; 
 wherein the multiple convolution layers are located at a same depth of the image processing model, and 
 the multiple convolution layers are respectively configured to process the feature maps of different scales; 
 the first result comprises multiple neurons; 
 collecting, according to a position attribute of the each neuron in the first result, statistics on a scale of the feature map corresponding to each neuron to obtain a distribution relationship; 
 wherein the position attribute indicates a convolution layer to which the each neuron belongs; 
 the distribution relationship indicates a correspondence between the each feature map and the neuron through which the feature map passes; and 
 distributing, according to the distribution relationship, to the each feature map the neuron through which the feature map passes. 
 
     
     
       2. The method according to  claim 1 , wherein the image processing model comprises at least one network module, and each network module comprises:
 a first network layer, a second network layer, and a third network layer that are cascaded sequentially; 
 wherein the first network layer is configured to process an inputted image to obtain a first feature map; 
 the second network layer comprises multiple parallel branches; 
 each branch of the multiple parallel branches comprises a first sampling layer; 
 the first sampling layer is configured to perform downsampling on the first feature map to obtain a second feature map; 
 wherein scales of second feature maps obtained by first sampling layers in different branches of the multiple parallel branches are different; 
 the third network layer is configured to merge feature map data outputted by the each branch. 
 
     
     
       3. The method according to  claim 2 , wherein the first sampling layer is a pooling layer. 
     
     
       4. The method according to  claim 3 , wherein the pooling layer is a maximum pooling layer. 
     
     
       5. The method according to  claim 2 , wherein the each branch further comprises a first convolution layer and a second sampling layer that are cascaded sequentially with the first sampling layer;
 the first convolution layer is configured to perform a convolution operation on the second feature map to obtain a third feature map; 
 the second sampling layer is configured to restore a scale of the third feature map to a scale of the first feature map. 
 
     
     
       6. The method according to  claim 2 , wherein the second network layer further comprises an original proportional branch; the original proportional branch comprises a second convolution layer;
 the second convolution layer is configured to perform a convolution operation on the first feature map, and input feature map data obtained by the convolution operation into the third network layer; 
 the third network layer is further configured to merge feature map data outputted by the multiple parallel branches in the second network layer with feature map data outputted by the original proportional branch. 
 
     
     
       7. The method according to  claim 1 , before filtrating, according to the importance of the neurons of the multiple convolution layers in the image processing model, the neurons, further comprising:
 determining a number of branches in each network module, and constructing the image processing model according to a preset number of network modules; and 
 training the image processing model to obtain a scale parameter of a batchnorm layer in the image processing model; 
 wherein the scale parameter indicates the importance of the neurons of the convolution layers in the branches in the network module. 
 
     
     
       8. The method according to  claim 1 , wherein filtrating, according to the importance of the neurons of the multiple convolution layers in the image processing model, the neurons to obtain the first result comprises:
 sorting, according to a scale parameter of a batchnorm layer obtained by pre-training the image processing model, the neurons of the multiple convolution layers to obtain a first sequence; 
 wherein the first sequence indicates an arrangement order of the neurons of the multiple convolution layers; 
 determining a number of neurons to be used for processing the multiple feature maps according to a preset amount of calculation; and 
 sequentially extracting, according to a determined number of the neurons to be used, required neurons from the first sequence to obtain the first result. 
 
     
     
       9. The method according to  claim 8 , after distributing, according to the distribution relationship, to the each feature map the neuron through which the feature map passes, further comprising:
 determining, according to the distribution relationship, a first network structure of the image processing model; and 
 training the image processing model of the first network structure. 
 
     
     
       10. An electronic device, comprising:
 a processor; and 
 a memory configured to store processor-executable instructions; 
 wherein when the processor-executable instructions are executed, the processor is configured to: 
 distribute, when multiple feature maps exist in an image processing model, to each feature map a neuron through which the feature map passes; 
 filtrate neurons according to importance of the neurons of multiple convolution layers in the image processing model to obtain a first result; 
 wherein the multiple convolution layers are located at a same depth of the image processing model, and 
 the multiple convolution layers are respectively configured to process the feature maps of different scales; 
 the first result comprises multiple neurons; 
 collect statistics on a scale of the feature map corresponding to each neuron according to a position attribute of the each neuron in the first result to obtain a distribution relationship; 
 wherein the position attribute indicates a convolution layer to which the each neuron belongs; 
 the distribution relationship indicates a correspondence between the each feature map and the neuron through which the feature map passes; and 
 distribute, according to the distribution relationship, to the each feature map the neuron through which the feature map passes. 
 
     
     
       11. The electronic device according to  claim 10 , wherein the image processing model comprises at least one network module, and each network module comprises:
 a first network layer, a second network layer, and a third network layer that are cascaded sequentially; 
 wherein the first network layer is configured to process an inputted image to obtain a first feature map; 
 the second network layer comprises multiple parallel branches; 
 each branch of the multiple parallel branches comprises a first sampling layer; 
 the first sampling layer is configured to perform downsampling on the first feature map to obtain a second feature map; 
 wherein scales of second feature maps obtained by first sampling layers in different branches of the multiple parallel branches are different; 
 the third network layer is configured to merge feature map data outputted by the each branch. 
 
     
     
       12. The electronic device according to  claim 11 , wherein the first sampling layer is a pooling layer. 
     
     
       13. The electronic device according to  claim 12 , wherein the pooling layer is a maximum pooling layer. 
     
     
       14. The electronic device according to  claim 11 , wherein the each branch further comprises a first convolution layer and a second sampling layer that are cascaded sequentially with the first sampling layer;
 the first convolution layer is configured to perform a convolution operation on the second feature map to obtain a third feature map; 
 the second sampling layer is configured to restore a scale of the third feature map to a scale of the first feature map. 
 
     
     
       15. The electronic device according to  claim 11 , wherein the second network layer further comprises an original proportional branch; the original proportional branch comprises a second convolution layer;
 the second convolution layer is configured to perform a convolution operation on the first feature map, and input feature map data obtained by the convolution operation into the third network layer; 
 the third network layer is further configured to merge feature map data outputted by the multiple parallel branches in the second network layer with feature map data outputted by the original proportional branch. 
 
     
     
       16. The electronic device according to  claim 10 , wherein the processor is further configured to:
 determine a number of branches in each network module, and construct the image processing model according to a preset number of network modules before the processor filtrates the neurons according to the importance of the neurons of the multiple convolution layers in the image processing model; 
 train the image processing model to obtain a scale parameter of a batchnorm layer in the image processing model; 
 wherein the scale parameter indicates the importance of the neurons of the convolution layers in the branches in the network module. 
 
     
     
       17. The electronic device according to  claim 10 , wherein the processor is specifically configured to:
 sort the neurons of the multiple convolution layers according to a scale parameter of a batchnorm layer obtained by pre-training the image processing model to obtain a first sequence; 
 wherein the first sequence indicates an arrangement order of the neurons of the multiple convolution layers; 
 determine a number of neurons to be used for processing the multiple feature maps according to a preset amount of calculation; and 
 sequentially extract required neurons from the first sequence according to a determined number of the neurons to be used to obtain the first result. 
 
     
     
       18. The electronic device according to  claim 17 , wherein the processor is further configured to:
 determine a first network structure of the image processing model according to the distribution relationship after the processor distributes, according to the distribution relationship, to the each feature map the neuron through which the feature map passes; and 
 train the image processing model of the first network structure. 
 
     
     
       19. A non-transitory computer-readable storage medium, having computer program instructions thereon, wherein when the computer program instructions are executed by a processor, the processor is configured to perform:
 distributing, when multiple feature maps exist in an image processing model, to each feature map a neuron through which the feature map passes; 
 filtrating, according to importance of neurons of multiple convolution layers in an image processing model, the neurons to obtain a first result; 
 wherein the multiple convolution layers are located at a same depth of the image processing model, and 
 the multiple convolution layers are respectively configured to process the feature maps of different scales; 
 the first result comprises multiple neurons; 
 collecting, according to a position attribute of the each neuron in the first result, statistics on a scale of the feature map corresponding to each neuron to obtain a distribution relationship; 
 wherein the position attribute indicates a convolution layer to which the each neuron belongs; 
 the distribution relationship indicates a correspondence between the each feature map and the neuron through which the feature map passes; and 
 distributing, according to the distribution relationship, to the each feature map the neuron through which the feature map passes. 
 
     
     
       20. The non-transitory computer-readable storage medium according to  claim 19 , wherein the image processing model comprises at least one network module, and each network module comprises:
 a first network layer, a second network layer, and a third network layer that are cascaded sequentially; 
 wherein the first network layer is configured to process an inputted image to obtain a first feature map; 
 the second network layer comprises multiple parallel branches; 
 each branch of the multiple parallel branches comprises a first sampling layer; 
 the first sampling layer is configured to perform downsampling on the first feature map to obtain a second feature map; 
 wherein scales of second feature maps obtained by first sampling layers in different branches of the multiple parallel branches are different; 
 the third network layer is configured to merge feature map data outputted by the each branch.