Cooperative use of a genetic algorithm and an optimization trainer for autoencoder generation

Abstract

A method includes, during an epoch of a genetic algorithm, determining a fitness value for each of a plurality of autoencoders. The fitness value for an autoencoder indicates reconstruction error responsive to data representing a first operational state of one or more devices. The method includes selecting, based on the fitness values, a subset of autoencoders. The method also includes performing a genetic operation with respect to at least one autoencoder to generate a trainable autoencoder. The method includes training the trainable autoencoder to reduce a loss function value to generate a trained autoencoder. The loss function value is based on reconstruction error of the trainable autoencoder responsive to data representative of a second operational state of the device(s). The method includes adding the trained autoencoder to a population to be provided as input to a subsequent epoch of the genetic algorithm.

What is claimed is: 
     
         1 . A computer system comprising:
 a memory configured to store a data set and a plurality of data structures representative of a plurality of dimensional-reduction models;   a processor configured to execute a recursive search, wherein executing the recursive search comprises, during a first iteration:
 determining fitness values for the plurality of dimensional-reduction models, wherein a fitness value of a particular dimensional-reduction models is indicative of error associated with dimensional reduction of a first portion of the data set by the particular dimensional-reduction model; 
 selecting a subset of dimensional-reduction models from the plurality of dimensional-reduction models based on the fitness values of the subset of dimensional-reduction models; 
 modifying a topology of at least one dimensional-reduction model of the subset to generate a trainable dimensional-reduction model; and 
 providing the trainable dimensional-reduction model to an optimization trainer, the optimization trainer configured to:
 train the trainable dimensional-reduction model to reduce a loss function value to generate a trained dimensional-reduction model, wherein the loss function value is indicative of error associated with dimensional reduction of a second portion of the data set by the trained dimensional-reduction model; and 
 provide the trained dimensional-reduction model as input to a second iteration of the recursive search that is subsequent to the first iteration. 
 
   
     
     
         2 . The computer system of  claim 1 , wherein dimensional-reduction models having fitness values indicating higher error associated with dimensional reduction of the first portion of the data set are selected for the subset of dimensional-reduction models in preference to dimensional-reduction models having fitness values indicating lower error associated with dimensional reduction of the first portion of the data set. 
     
     
         3 . The computer system of  claim 2 , wherein the first portion of the data set represents an anomalous operational state of one or more devices and the second portion of the data set represents a normal operational state of the one or more devices. 
     
     
         4 . A method of generating a data model for dimensionally reducing a data set, the method comprising:
 during a first iteration of a search:
 determining, by a processor of a computing device, a value of a fitness function for each of a plurality of dimensional-reduction models, wherein the value of the fitness function for a particular dimensional-reduction model is indicative of error associated with dimensional reduction of a first portion of the data set by the particular dimensional-reduction model; 
 selecting, by the processor of the computing device based on the values of the fitness function, a subset of dimensional-reduction models from the plurality of dimensional-reduction models; 
 modifying a topology of at least one dimensional-reduction model of the subset to generate a trainable dimensional-reduction model; 
 training the trainable dimensional-reduction model to reduce a loss function value to generate a trained dimensional-reduction model, wherein the loss function value is indicative of error associated with dimensional reduction of a second portion of the data set by the trained dimensional-reduction model; and 
 adding the trained dimensional-reduction model to a population of dimensional-reduction models to be provided as input to a second iteration of the search that is subsequent to the first iteration of the search. 
   
     
     
         5 . The method of  claim 4 , wherein dimensional-reduction models having fitness values indicating higher error associated with dimensional reduction of the first portion of the data set are selected for the subset of dimensional-reduction models in preference to dimensional-reduction models having fitness values indicating lower error associated with dimensional reduction of the first portion of the data set. 
     
     
         6 . The method of  claim 5 , wherein the first portion of the data set represents an anomalous operational state of one or more devices and the second portion of the data set represents a normal operational state of the one or more devices. 
     
     
         7 . The method of  claim 4 , wherein one or more dimensional-reduction models having the values of the fitness function indicating lower error associated with dimensional reduction of the first portion of the data set are omitted from the population of dimensional-reduction models to be provided as input to the second iteration. 
     
     
         8 . The method of  claim 4 , wherein the training the trainable dimensional-reduction model modifies connection weights of the trainable dimensional-reduction model and does not modify the topology of the trainable dimensional-reduction model. 
     
     
         9 . The method of  claim 4 , wherein the topology of the at least one dimensional-reduction model is modified via a genetic operation. 
     
     
         10 . The method of  claim 9 , wherein the genetic operation modifies a layer type of at least one layer of the at least one dimensional-reduction model. 
     
     
         11 . The method of  claim 9 , wherein the genetic operation includes crossover, mutation, or a combination thereof. 
     
     
         12 . The method of  claim 4 , further comprising modifying an activation function of at least one node of the at least one dimensional-reduction model to generate the trainable dimensional-reduction model. 
     
     
         13 . The method of  claim 4 , wherein the first iteration corresponds to an epoch of a genetic algorithm. 
     
     
         14 . The method of  claim 4 , wherein the second iteration and the first iteration are separated by at least one iteration. 
     
     
         15 . A computer-readable storage device storing instructions that, when executed, cause a computer to perform operations comprising:
 during a first iteration of a search:
 determining a value of a fitness function for each of a plurality of dimensional-reduction models, wherein the value of the fitness function for a particular dimensional-reduction model is indicative of error associated with dimensional reduction of a first portion of a data set by the particular dimensional-reduction model; 
 selecting, based on the values of the fitness function, a subset of dimensional-reduction models from the plurality of dimensional-reduction models; 
 modifying a topology of at least one dimensional-reduction model of the subset to generate a trainable dimensional-reduction model; 
 training the trainable dimensional-reduction model to reduce a loss function value to generate a trained dimensional-reduction model, wherein the loss function value is indicative of error associated with dimensional reduction of a second portion of the data set by the trained dimensional-reduction model; and 
 adding the trained dimensional-reduction model to a population of dimensional-reduction models to be provided as input to a second iteration of the search that is subsequent to the first iteration of the search. 
   
     
     
         16 . The computer-readable storage device of  claim 15 , wherein at a beginning of the first iteration, each dimensional-reduction model of the plurality of dimensional-reduction models comprises a latent-space layer, and a count of nodes in the latent-space layer is identical for each dimensional-reduction model of the plurality of dimensional-reduction models. 
     
     
         17 . The computer-readable storage device of  claim 16 , wherein modifying the topology of the at least one dimensional-reduction model includes changing a count of nodes in the latent-space layer of the at least one dimensional-reduction model. 
     
     
         18 . The computer-readable storage device of  claim 15 , wherein the at least one dimensional-reduction model includes an encoder portion having a first set of layers and a decoder portion having a second set of layers, and wherein modifying the topology of the at least one dimensional-reduction model includes selectively modifying the first set of layers without modifying the second set of layers or includes selectively modifying the second set of layers without modifying the first set of layers. 
     
     
         19 . The computer-readable storage device of  claim 18 , wherein the operations further comprise performing a randomized selection of the encoder portion or the decoder portion for modification. 
     
     
         20 . The computer-readable storage device of  claim 15 , wherein modifying the topology of the at least one dimensional-reduction model includes selectively modifying a count of nodes in one or more layers of the at least one dimensional-reduction model. 
     
     
         21 . The computer-readable storage device of  claim 20 , wherein the trained dimensional-reduction model includes a non-hourglass autoencoder. 
     
     
         22 . The computer-readable storage device of  claim 20 , wherein the trained dimensional-reduction model includes a non-mirrored autoencoder.