Imitation training for machine learning networks

Abstract

A system and method for controlling a nodal network. The method includes estimating an effect on the objective caused by the existence or non-existence of a direct connection between a pair of nodes and changing a structure of the nodal network based at least in part on the estimate of the effect. A nodal network includes a strict partially ordered set, a weighted directed acyclic graph, an artificial neural network, and/or a layered feed-forward neural network.

1 - 101 . (canceled) 
     
     
         102 . A method for training an imitation machine learning network to imitate performance of a reference machine learning network, the method comprising:
 classifying, by the reference machine learning network, training data examples in a set of training data; and   training, by a computer system, through machine learning, the imitation machine leaning network to make a same classification as the reference machine learning network on each of the training data examples in the set of training data, wherein classifications by the reference machine learning network for the training data examples are used as training labels for the training data examples when training the imitation machine learning network.   
     
     
         103 . The method of  claim 102 , wherein the training data examples in the set of training data comprises:
 labeled training data examples;   unlabeled training data examples; and   training data examples generated by a data generator.   
     
     
         104 . The method of  claim 102 , wherein the imitation machine learning network comprises more nodes that the reference machine learning network. 
     
     
         105 . The method of  claim 102 , wherein capabilities of the imitation machine learning network are a superset of capabilities of the reference machine learning network. 
     
     
         106 . The method of  claim 102 , wherein capabilities of the imitation machine learning network are a proper subset of capabilities of the reference machine learning network. 
     
     
         107 . The method of  claim 102 , wherein the imitation machine learning network comprises a different network topology than the reference machine learning network. 
     
     
         108 . The method of  claim 102 , wherein the imitation machine learning network comprises a self-organizing partially ordered network. 
     
     
         109 . The method of  claim 108 , wherein the reference machine learning network comprises a self-organizing partially ordered network. 
     
     
         110 . The method of  claim 102 , wherein the reference machine learning network is a sub-network of a main reference network, wherein the reference machine learning network culminates in a node B of the main reference network. 
     
     
         111 . The method of  claim 110 , wherein
 the main reference network comprises a node A, such that node B covers node A in the main reference network; and   training the imitation machine learning network comprises using, in the imitation machine learning network, a node C of the reference machine learning network, wherein node A covers node C in the reference machine learning network.   
     
     
         112 . The method of  claim 111 , wherein
 the main reference network comprises a node A, such that node B covers node A in the main reference network; and   training the imitation machine learning network comprises using, in the imitation machine learning network, a node C of the reference machine learning network, wherein node C is incomparable to node A in the reference machine learning network.   
     
     
         113 . A method comprising:
 training, by a computer system, through machine learning, an imitation machine learning network such that, for a plurality of training data items, activations of an output node N of the imitation machine learning network have a high correlation with derivative vectors of a node A in a reference machine learning network.   
     
     
         114 . The method of  claim 113 , further comprising placing, by the computer system, the imitation machine learning network in the reference machine learning network such that node A in the reference machine learning network covers the output node N of the imitation machine learning network. 
     
     
         115 . The method of  claim 114 , further comprising, after placing the imitation machine learning network in the reference machine learning network, training, by the computer system, the reference machine learning network. 
     
     
         116 . A computer system for training an imitation machine learning network to imitate performance of a reference machine learning network, the computer system comprising:
 one or more processor cores; and   a memory in communication with the one or more processor cores, wherein the memory stores instructions that when executed by the one or more processor cores cause the one or more processor cores to:   classify, by the reference machine learning network, training data examples in a set of training data; and   train, through machine learning, the imitation machine leaning network to make a same classification as the reference machine learning network on each of the training data examples in the set of training data, wherein classifications by the reference machine learning network for the training data examples are used as training labels for the training data examples when training the imitation machine learning network.   
     
     
         117 . The computer system of  claim 116 , wherein the imitation machine learning network comprises more nodes that the reference machine learning network. 
     
     
         118 . The computer system of  claim 116 , wherein capabilities of the imitation machine learning network are a superset of capabilities of the reference machine learning network. 
     
     
         119 . The computer system of  claim 116 , wherein capabilities of the imitation machine learning network are a proper subset of capabilities of the reference machine learning network. 
     
     
         120 . The computer system of  claim 116 , wherein the reference machine learning network is a sub-network of a main reference network, wherein the reference machine learning network culminates in a node B of the main reference network. 
     
     
         121 . The computer system of  claim 120 , wherein
 the main reference network comprises a node A, such that node B covers node A in the main reference network; and   the imitation machine learning network comprises a node C of the reference machine learning network, wherein node A covers node C in the reference machine learning network.   
     
     
         122 . The computer system of  claim 121 , wherein
 the main reference network comprises a node A, such that node B covers node A in the main reference network; and   the imitation machine learning network comprises a node C of the reference machine learning network, wherein node C is incomparable to node A in the reference machine learning network.   
     
     
         123 . A computer system comprising:
 one or more processor cores; and   a memory in communication with the one or more processor cores, wherein the memory stores instructions that when executed by the one or more processor cores cause the one or more processor cores to train, through machine learning, an imitation machine learning network such that, for a plurality of training data items, activations of an output node N of the imitation machine learning network have a high correlation with derivative vectors of a node A in a reference machine learning network.   
     
     
         124 . The computer system of  claim 123 , wherein the memory further stores instructions that when executed by the one or more processor cores cause the one or more processor cores to include the imitation machine learning network in the reference machine learning network such that node A in the reference machine learning network covers the output node N of the imitation machine learning network. 
     
     
         125 . The computer system of  claim 124 , wherein the memory further stores instructions that when executed by the one or more processor cores cause the one or more processor cores to, after including the imitation machine learning network in the reference machine learning network, train the reference machine learning network.