Method for transforming an abstract representation of a trained neural network into program code in a target language

Abstract

A method for transforming an abstract representation of a trained neural network into program code in a target language. The program code is convertible into executable program code using a compiler for the target language. The method includes: reading an abstract representation of a neural network which has already been trained; calculating an intermediate representation of the neural network from the abstract representation, the intermediate representation specifying a computation graph; ascertaining a plurality of plan proposals for planning the memory usage during the execution of the computation graph; ascertaining a quality level for each plan proposal; selecting a plan proposal based on the ascertained quality level; and generating the sought program code in the target language from the intermediate representation and the selected plan proposal.

1 - 17  (canceled) 
     
     
         18 . A method for transforming an abstract representation of a trained neural network into program code in a target language, the program code being convertible into executable program code using a compiler for the target language, the method comprising the following steps:
 reading an abstract representation of a neural network which has already been trained, wherein the abstract representation including at least an architecture of the neural network and parameters which are obtained from the training process and which characterize a behavior of the neural network;   calculating an intermediate representation of the neural network from the abstract representation, wherein the intermediate representation specifies a computation graph for output of the neural network;   ascertaining a plurality of plan proposals for planning the memory usage during execution of the computation graph, wherein the plan proposals assign an address within a predetermined, allocated memory of a terminal device to each buffer provided in the structure of the neural network;   ascertaining a quality level for each plan proposal using at least one specified criterion, wherein the quality level is a measure of a maximum total memory requirement during the execution of the computation graph, wherein a lowest possible total memory volume should be used for all buffers;   selecting a plan proposal from the plurality of plan proposals based on the ascertained quality levels; and   generating the program code in the target language from the intermediate representation and the selected plan proposal.   
     
     
         19 . The method according to  claim 18 , wherein the plan proposals are ascertained using a plurality of different predetermined memory planning algorithms and/or using a parameterized approach with a plurality of different values of the free parameters. 
     
     
         20 . The method according to  claim 18 , wherein at least one plan proposal includes dynamically assigning memory space for intermediate results arising during the execution of the computation graph, and subsequently releasing the assigned memory space. 
     
     
         21 . The method according to  claim 18 , wherein the quality level is a measure of the maximum total memory space requirement and optionally of the program runtime depending on a number of dynamically placed buffers during the execution of the computation graph. 
     
     
         22 . The method according to  claim 18 , wherein the computation graph of the neural network includes:
 transmitting intermediate results from a layer, or from an operator, of the neural network to a subsequent layer, or to a subsequent operator, of the neural network through an interbuffer, and/or   temporarily storing the intermediate results within a layer f the neural network in an intrabuffer for subsequent use in the same layer.   
     
     
         23 . The method according to  claim 19 , wherein one of the algorithms of the plurality of predetermined memory planning algorithms includes the following steps:
 providing a global memory area having a predetermined size which corresponds at least to MaxLB, wherein MaxLB corresponds to a combined memory requirement of all intermediate results whose simultaneous retention is still necessary at the time of execution of an operator;   selecting, for each intermediate result, a smallest free memory area within the global memory area which can accommodate the intermediate result and is available for a necessary retention time of the intermediate result; and   assigning the smallest free memory area to the intermediate result for the necessary retention time.   
     
     
         24 . The method according to  claim 19 , wherein one of the algorithms of the plurality of predetermined algorithms includes an optimization of N start addresses o i , i=1, . . . , N, for intermediate results using integer linear programming, wherein
 the i-th intermediate result has a memory requirement s i  and, in accordance with said memory requirement s i , occupies a memory area in the memory between the start address o i  and an end address e i ,   pairs (u, v) of intermediate results u and v whose simultaneous retention is necessary are stored in a conflict set C,   the memory areas occupied by the intermediate results u and v must not overlap, and   the optimization is aimed at minimizing a highest resulting end address e i .   
     
     
         25 . The method according to  claim 24 , further comprising the following steps:
 providing a global memory area having a predetermined size which corresponds at least to MaxLB, wherein MaxLB corresponds to a combined memory requirement of all intermediate results whose simultaneous retention is the maximum still necessary at the time of execution of an operator, and   optimizing (B 200 ) the start addresses o i  using the following equations:   
       
         
           
             
               
                 
                   e 
                   i 
                 
                 = 
                 
                   
                     o 
                     i 
                   
                   + 
                   
                     s 
                     i 
                   
                 
               
               , 
             
           
         
         
           
             
               
                 0 
                 ≤ 
                 
                   o 
                   i 
                 
                 ≤ 
                 
                   MaxLB 
                   - 
                   
                     s 
                     i 
                   
                 
               
               , 
             
           
         
         
           
             
               
                 
                   o 
                   u 
                 
                 ≥ 
                 
                   
                     e 
                     v 
                   
                   ⁢ 
                       
                   or 
                   ⁢ 
                     
                   
                     o 
                     v 
                   
                 
                 ≥ 
                 
                   e 
                   u 
                 
               
               , 
               
                 
                   if 
                   ⁢ 
                       
                   
                     ( 
                     
                       u 
                       , 
                       v 
                     
                     ) 
                   
                 
                 ∈ 
                 
                   C 
                   . 
                 
               
             
           
         
       
     
     
         26 . The method according to  claim 24 , wherein, in response to the optimization not leading to a solution, a size of the global memory area is increased to a value between MaxLB and 2*MaxLB. 
     
     
         27 . The method according to  claim 19 , wherein one of the algorithms of the plurality of predetermined memory planning algorithms includes the following steps:
 ascertaining the intermediate results which are expected to require the most memory space and/or layers of the neural network whose computation is expected to require the most memory space for intermediate results,   creating an initial memory plan for the ascertained intermediate results without taking into account the other intermediate results, and   placing remaining intermediate results using another memory planning algorithm.   
     
     
         28 . The method according to  claim 27 , wherein one of the algorithms of the plurality of predetermined memory planning algorithms includes the following steps:
 providing a global memory area having a predetermined size which corresponds at least to MaxLB, wherein MaxLB corresponds to a combined memory requirement of all intermediate results whose simultaneous retention is still necessary at the time of execution of an operator,   selecting, for each intermediate result, a smallest free memory area within the global memory area which can accommodate the intermediate result and is available for a necessary retention time of the intermediate result, and   assigning the smallest free memory area to the intermediate result for the necessary retention time; and   
       wherein the one of the algorithms is selected for creating an initial memory plan and/or for placing remaining intermediate results. 
     
     
         29 . The method according to  claim 19 , further comprising the following steps:
 selecting intermediate results which are not to be included in the memory planning but are to be dynamically stored on a stack, and   executing the plurality of predetermined memory planning algorithms taking into account only the intermediate results not to be stored on the stack.   
     
     
         30 . The method according to  claim 18 , wherein, for debugging purposes, the plan proposals are ascertained under an additional boundary condition that a memory area once assigned for an intermediate result is not reused for another intermediate result. 
     
     
         31 . The method according to  claim 18 , further comprising:
 converting the program code in the target language into executable program code; and   executing the executable program code on at least one computer and/or on at least one compute instance so that at least one input of the neural network is converted into at least one output of the neural network.   
     
     
         32 . A non-transitory machine-readable data carrier on which is stored a computer program including machine-readable instructions for transforming an abstract representation of a trained neural network into program code in a target language, the program code being convertible into executable program code using a compiler for the target language, the instructions when executed by one or more computers and/or computer instances, causing the one or more computers and/or computer instances to perform the following steps:
 reading an abstract representation of a neural network which has already been trained, wherein the abstract representation including at least an architecture of the neural network and parameters which are obtained from the training process and which characterize a behavior of the neural network;   calculating an intermediate representation of the neural network from the abstract representation, wherein the intermediate representation specifies a computation graph for output of the neural network;   ascertaining a plurality of plan proposals for planning the memory usage during execution of the computation graph, wherein the plan proposals assign an address within a predetermined, allocated memory of a terminal device to each buffer provided in the structure of the neural network;   ascertaining a quality level for each plan proposal using at least one specified criterion, wherein the quality level is a measure of a maximum total memory requirement during the execution of the computation graph, wherein a lowest possible total memory volume should be used for all buffers;   selecting a plan proposal from the plurality of plan proposals based on the ascertained quality levels; and   generating the program code in the target language from the intermediate representation and the selected plan proposal.   
     
     
         33 . One or more computers and/or compute instances equipped a non-transitory machine-readable data carrier on which is stored a computer program including machine-readable instructions for transforming an abstract representation of a trained neural network into program code in a target language, the program code being convertible into executable program code using a compiler for the target language, the instructions when executed by the one or more computers and/or computer instances, causing the one or more computers and/or computer instances to perform the following steps:
 reading an abstract representation of a neural network which has already been trained, wherein the abstract representation including at least an architecture of the neural network and parameters which are obtained from the training process and which characterize a behavior of the neural network;   calculating an intermediate representation of the neural network from the abstract representation, wherein the intermediate representation specifies a computation graph for output of the neural network;   ascertaining a plurality of plan proposals for planning the memory usage during execution of the computation graph, wherein the plan proposals assign an address within a predetermined, allocated memory of a terminal device to each buffer provided in the structure of the neural network;   ascertaining a quality level for each plan proposal using at least one specified criterion, wherein the quality level is a measure of a maximum total memory requirement during the execution of the computation graph, wherein a lowest possible total memory volume should be used for all buffers;   selecting a plan proposal from the plurality of plan proposals based on the ascertained quality levels; and   generating the program code in the target language from the intermediate representation and the selected plan proposal.