Feedback systems and methods for recognizing patterns

Abstract

Pattern classification system and methods are disclosed. The pattern classification systems and methods employ one or more classification networks that can parse multiple patterns simultaneously while providing a continuous feedback about its progress. Per-synaptic inhibition is employed to inhibit feedback connections to permit more flexible processing. Various additional improvements result in highly robust pattern recognition systems and methods that are suitable for use in research, development, and production.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 obtaining an input;   classifying features of the input using a trained neural network that includes a set of input nodes and a set of output nodes, wherein each of the set of input nodes represents any feature, wherein the classifying further includes:
 deriving, by each of the set of output nodes during each of a number of recognition iterations, a representation from each feature provided by the one or more input nodes connected to each of the set of output nodes and a set of feedback; 
 providing, by each of the set of output nodes during each of the number of recognition iterations, the set of feedback derived by each of the set of output nodes to the one or more input nodes connected to each of the set of output nodes; and 
 updating, at each of the number of recognition iterations, a state at each recognition iteration based on the set of feedback from each corresponding output node of the set of output nodes, wherein modification of the input nodes are inhibited by the set of feedback, wherein the number of recognition iterations are based on the updated state of each input node updated based on the set of feedback from each corresponding output node of the set of output nodes; and 
   identifying one or more patterns from the set of output nodes after the number of recognition iterations.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the set of input nodes and the set of output nodes are composed in a modular composition configured to avoid combinational problems. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein each of said one or more output nodes represents a labeled representation that includes at least one of the features, the neural network trained with a training set having one or more labeled training samples, and wherein each of said one or more output nodes is associated with at least one of said one or more labeled training samples. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein, in each of said one or more iterations, all of the output nodes feedback to all their associated input nodes at a same time. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein the feedback from each output node to the associated set of input nodes inhibits the input nodes using subtractive inhibition or shunting inhibition. 
     
     
         6 . The computer-implemented method of  claim 5 , wherein activation from an inhibited input is projected to a corresponding output, completing an iteration and preparing for a subsequent iteration. 
     
     
         7 . The computer-implemented method of  claim 1 , further comprising:
 determining whether the number of recognition iterations exceeds a predefined threshold number; and, responsive to determining that the number of recognition iterations-exceeds the predefined threshold number, changing an attention window to acquire sensor inputs providing the input.   
     
     
         8 . The computer-implemented method of  claim 1 , further comprising:
 determining whether feedback from an output node to an associated input node does not satisfy a matching criteria after a predefined number of iterations; and   responsive to determining that feedback from the output node to the associated input node does not satisfy the matching criteria, changing an attention window to acquire sensor inputs providing the spatial input.   
     
     
         9 . The computer-implemented method of  claim 1 , further comprising:
 determining whether the number of recognition iterations exceeds a predefined threshold number; and,   based on determining that number of recognition iterations exceeds the predefined threshold number, performing at least one of: adding nodes, modifying weights, modifying a hierarchy of nodes, and modifying connections.   
     
     
         10 . The computer-implemented method of  claim 1 , further comprising:
 determining whether feedback from an output node to an associated input node does not satisfy a matching criteria after a predefined number of iterations; and,   based on determining that the feedback from the output node to the associated input node does not satisfy the matching criteria, performing at least one of: adding nodes, modifying weights, and modifying connections.   
     
     
         11 . The computer-implemented method of  claim 1 , wherein the trained neural network comprises a hierarchy of layers, each layer in the hierarchy of layers comprising one or more of said input nodes and one or more of said output nodes, wherein output nodes of a given layer function as input nodes of a layer that is higher than said given layer in said hierarchy of layers; and
 wherein said neural network comprises bidirectional connections between each output node and one or more input nodes of each layer, and wherein feedback from one or more input nodes of a first layer acts as a bias to one or more output nodes of a second layer that is lower than the first layer.   
     
     
         12 . The computer-implemented method of  claim 11 , wherein output nodes of a given layer function as input nodes of a layer that is immediately higher than said given layer, and wherein feedback from input nodes of a first layer acts as a bias to the output nodes of a second layer that is immediately lower than said first layer. 
     
     
         13 . The computer-implemented method of  claim 11 , wherein the feedback from said one or more input nodes of the first layer also biases at least one lower layer other than a lower layer that is immediately lower than the first layer. 
     
     
         14 . The computer-implemented method of  claim 11 , wherein the feedback from the one or more output nodes of the first layer is employed to generate patterns previously learned. 
     
     
         15 . The computer-implemented method of  claim 14 , further comprising
 further analyzing at least one of the patterns generated using the feedback to discover characteristics pertaining to at least one of the patterns.   
     
     
         16 . The computer-implemented method of  claim 14  wherein output nodes of another layer also function as input nodes of another layer that is immediately higher than the given layer, the another layer also being lower than the given layer in the hierarchy of layers. 
     
     
         17 . The computer-implemented method of  claim 11  further comprising:
 biasing an output node to reveal a strength of connectivity to input nodes associated with the output node. 
 
     
     
         18 . A system comprising:
 a processor; and   a computer readable medium containing instructions that, when executed by the processor, cause the processor to perform operations comprising:   classifying features of the input using a trained neural network that includes a set of input nodes and a set of output nodes, wherein each of the set of input nodes represents any feature, wherein the classifying further includes:
 deriving, by each of the set of output nodes during each of a number of recognition iterations, a representation from each feature provided by the one or more input nodes connected to each of the set of output nodes and a set of feedback; 
 providing, by each of the set of output nodes during each of the number of recognition iterations, the set of feedback derived by each of the set of output nodes to the one or more input nodes connected to each of the set of output nodes; 
 updating, at each of the number of recognition iterations, a state at each recognition iteration based on the set of feedback from each corresponding output node of the set of output nodes, wherein modification of the input nodes are inhibited by the set of feedback, wherein the number of recognition iterations are based on the updated state of each input node updated based on the set of feedback from each corresponding output node of the set of output nodes; and 
   identifying one or more patterns from the set of output nodes after the number of recognition iterations.   
     
     
         19 . The system of  claim 18 , wherein the set of input nodes and the set of output nodes are composed in a modular composition configured to avoid combinational problems. 
     
     
         20 . The system of  claim 18 , wherein each of the set of output nodes represents a labeled representation that includes at least one of the features, the neural network trained with a training set having one or more labeled training samples, and wherein each of said one or more output nodes is associated with at least one of said one or more labeled training samples.