Cooperative execution of a genetic algorithm with an efficient training algorithm for data-driven model creation

Abstract

A method includes determining a trainable model to provide to a trainer, the trainable model determined based on modification of one or more models of a plurality of models. The plurality of models is generated based on a genetic algorithm and corresponds to a first epoch of the genetic algorithm. Each of the plurality of models includes data representative of a neural network. The method also includes providing the trainable model to the trainer. The method further includes adding a trained model, output by the trainer based on the trainable model, as input to a second epoch of the genetic algorithm, the second epoch subsequent to the first epoch.

What is claimed is: 
     
         1 . A computer system comprising:
 a memory configured to store an input data set and a plurality of data structures, each of the plurality of data structures including data representative of a neural network; and   a processor configured to execute a recursive search, wherein executing the recursive search comprises, during a first iteration:
 determining a trainable data structure based on modification of one or more data structures of the plurality of data structures; and 
 providing the trainable data structure to an optimization trainer, the optimization trainer configured to: 
 train the trainable data structure based on a portion of the input data set to generate a trained data structure; and 
 provide the trained data structure as input to a second iteration of the recursive search, the second iteration subsequent to the first iteration. 
   
     
     
         2 . The computer system of  claim 1 , wherein executing the recursive search further comprises, during the first iteration, selecting the one or more data structures based on their respective fitness values. 
     
     
         3 . The computer system of  claim 1 , wherein the modification corresponds to at least one of a crossover operation or a mutation operation with respect to the one or more data structures. 
     
     
         4 . The computer system of  claim 1 , wherein the optimization trainer is executed on a different device, graphics processing unit (GPU), processor, core, thread, or any combination thereof, than the recursive search. 
     
     
         5 . A method comprising:
 during a first epoch of a genetic algorithm, determining a trainable model to provide to an optimization trainer, the trainable model determined based on modification of one or more models of a plurality of models, wherein each of the plurality of models includes data representative of a neural network;   providing the trainable model to an optimization trainer; and   adding a trained model, output by the optimization trainer based on the trainable model, as input to a second epoch of the genetic algorithm, the second epoch subsequent to the first epoch.   
     
     
         6 . The method of  claim 5 , further comprising selecting the one or more models based on their respective fitness values. 
     
     
         7 . The method of  claim 5 , further comprising generating the trainable model including performing at least one crossover operation with respect to the one or more models. 
     
     
         8 . The method of  claim 5 , wherein the optimization trainer is configured to use a portion of an input data set associated with the genetic algorithm to train the trainable model. 
     
     
         9 . The method of  claim 5 , wherein a particular model of the plurality of models includes data representative of a particular neural network, and the data representative of the particular neural network is indicative of connections between nodes of the particular neural network. 
     
     
         10 . The method of  claim 5 , wherein a particular model of the plurality of models includes data representative of a particular neural network, and the data representative of the particular neural network is indicative of an activation function associated with one or more nodes of the particular neural network. 
     
     
         11 . The method of  claim 5 , wherein the first epoch is an initial epoch of the genetic algorithm. 
     
     
         12 . The method of  claim 5 , wherein the first epoch is a non-initial epoch of the genetic algorithm. 
     
     
         13 . The method of  claim 5 , wherein the second epoch and the first epoch are separated by at least one intervening epoch. 
     
     
         14 . The method of  claim 5 , further comprising, during the first epoch or the second epoch, removing from the plurality of models one or more models that satisfy a stagnation criterion. 
     
     
         15 . The method of  claim 5 , wherein each of the plurality of models includes at least one output node configured to generate a classifier result. 
     
     
         16 . The method of  claim 5 , further comprising generating the trainable model including performing at least one mutation operation with respect to the one or more models. 
     
     
         17 . A non-transitory computer-readable storage device storing instructions that, when executed, cause a computer to perform operations comprising:
 during a first iteration, determining a trainable model to provide to an optimization trainer, the trainable model determined based on modification of one or more models of a plurality of models, wherein each of the plurality of models includes data representative of a neural network;   providing the trainable model to the optimization trainer; and   adding a trained model, output by the optimization trainer based on the trainable model, as input to a second iteration, the second iteration subsequent to the first iteration.   
     
     
         18 . The non-transitory computer-readable storage device of  claim 17 , wherein the optimization trainer comprises a backpropagation trainer. 
     
     
         19 . The non-transitory computer-readable storage device of  claim 17 , wherein the operations further comprise selecting the one or more models based on their respective fitness values. 
     
     
         20 . The non-transitory computer-readable storage device of  claim 17 , wherein the trainable model is generated by performing at least one of a crossover operation or a mutation operation with respect to the one or more models.