Multi-stage image pattern recognizer

Abstract

An image-based pattern recognizer and a method and apparatus for making such a pattern recognizer are disclosed. By employing positional coding, the meaning of any feature present in an image can be defined implicitly in space. The pattern recognizer can be a neural network including a plurality of stages of observers. The observers are configured to cooperate to identify the presence of features in the input image and to recognize a pattern in the input image based on the features. Each of the observers includes a plurality of neurons. The input image includes a plurality of units, and each of the observers is configured to generate a separate output set that includes zero or more coordinates of such units.

1 . An apparatus comprising:
 a plurality of converters, each to input an input image and to compute a potential as a measure of contrast in the input image for each of a plurality of units of the input image, each converter further to generate an output set including a ranked set of coordinates, the ranked set of coordinates containing a coordinate of each unit in the input image whose potential exceeds a first threshold, the set of coordinates being ranked based on potential; and   a first plurality of observers, each observer to process independently the output set of each of the converters, each of the observers configured to recognize a different type of feature in the input image when a coordinate of a feature of the corresponding type is present in the output set of one or more of the converters.   
     
     
         2 . An apparatus as recited in  claim 1 , further comprising
 a first plurality of pattern filters, each including a plurality of weight matrices, wherein each of the first plurality of observers is configured to use a different one of the pattern filters to recognize the corresponding type of feature in the input image.   
     
     
         3 . An apparatus as recited in  claim 2 , wherein the first plurality of pattern filters are individually configured so that each observer of the first plurality of observers can recognize features at a different angular orientation in the input image. 
     
     
         4 . (canceled) 
     
     
         5 . An apparatus as recited in  claim 1 , further comprising:
 a second plurality of observers, each to process independently the output set of each of the first plurality of observers, each observer of the second plurality of observers configured to generate an output representing a relaxation of locality of a feature recognized in the input image.   
     
     
         6 . An apparatus as recited in  claim 5 , each observer of the second plurality of observers further being configured to generate the output to represent a prioritizing of endpoints of feature recognized in the input image more highly than a mid-section of the feature. 
     
     
         7 . An apparatus comprising:
 a processor; and   a memory storing code which, when executed by the processor, instantiates
 a plurality of converters, each to input an input image and to compute a potential as a measure of contrast in the input image for each of a plurality of units of the input image, each converter further to generate as output a ranked set of coordinates containing a coordinate of each unit in the input image whose potential exceeds a first threshold, the set of coordinates being ranked based on potential; 
 a first plurality of observers to process the outputs of the converters to recognize features in the input image that correspond to coordinates in the outputs of the converters; and 
 a first plurality of pattern filters, each operatively coupled between a different pair of a converter of the plurality of converters and an observer of the first plurality of observers, the first plurality of pattern filters being individually configured so that each observer of the first plurality of observers can recognize features at a different angular orientation in the input image. 
   
     
     
         8 . (canceled) 
     
     
         9 . An apparatus as recited in  claim 7 , wherein the memory further stores code which, when executed by the processor, instantiates:
 a second plurality of observers to process outputs of the first plurality of observers; and   a second plurality of pattern filters, each operatively coupled between a different pair of an observer of the first plurality of observers and an observer of the second plurality of observers, the second plurality of pattern filters being individually configured so that each observer of the second plurality of observers generates an output representing a relaxation of locality of a feature recognized in the input image.   
     
     
         10 . An apparatus as recited in  claim 9 , wherein the second plurality of pattern filters further are individually configured so that each observer of the second plurality of observers generates an output representing prioritizing endpoints of feature recognized in the input image more highly than a mid-section of the feature. 
     
     
         11 . An apparatus as recited in  claim 9 , wherein each observer of the first plurality of observers is configured to
 for each coordinate in the output of each of the first plurality of observers, integrate a corresponding potential over a range of time slices, and   for each said unit whose potential exceeds a second threshold after integration by the observer, including the coordinate of the unit in an output set of the observer;   and wherein each observer of the second plurality of observers is configured to,
 for each coordinate in the output of each of the first plurality of observers, integrate a corresponding potential over a range of time slices, and 
 for each said unit whose potential exceeds a third threshold after integration by the observer, including the coordinate of the unit in an output set of the observer. 
   
     
     
         12 . A method comprising:
 using a plurality of contrast converters to identify a plurality of units of an input image as potentially representing a feature in the input image;   using the plurality of contrast converters to generate a first output set that contains a ranking of coordinates of the identified units of the input image;   using a plurality of observers to attempt to recognize, in the input image, a feature from each of a first plurality of feature categories, based on the first output set, by independently using each of a first plurality of weight patterns to integrate a potential for each of the identified units, based on the ranking of coordinates in the first output set; and   using the plurality of observers to generate a plurality of second output sets as results of attempting to recognize a feature from each of the first plurality of feature categories, each said second output set corresponding to a different one of the first plurality of feature categories.   
     
     
         13 . A method as recited in  claim 12 , further comprising:
 automatically triggering a specified action in response to recognizing a pattern in the input image, based on the plurality of second output sets.   
     
     
         14 . A method as recited in  claim 12 , wherein identifying a plurality of units of an input image as potentially representing a feature comprises, for each of the plurality of units:
 computing a measure of contrast for the unit; and   identifying the unit as potentially representing a feature based on the measure of contrast.   
     
     
         15 . A method as recited in  claim 14 , wherein computing a measure of contrast for a unit comprises:
 computing a measure of positive contrast; and   computing a measure of negative contrast.   
     
     
         16 . A method as recited in  claim 14 , wherein the ranking of coordinates in the first output set is based on the measures of contrast of the corresponding units of the input image. 
     
     
         17 . A method as recited in  claim 12 , wherein each of the first plurality of weight patterns corresponds to a different angular orientation in the input image. 
     
     
         18 . A method as recited in  claim 12 , wherein using each of the first plurality of weight patterns to integrate a potential for each of the identified units comprises:
 applying a modulation factor to a weight, for each of a plurality of iterations of integration, to decode said ranking.   
     
     
         19 . A method as recited in  claim 12 , further comprising:
 attempting to recognize, in the input image, a feature from a second feature category, based on one of the second output sets, by using a second weight pattern to integrate a potential for each unit whose coordinate is represented in said second output set, based on a ranking of coordinates in said second output set, and generating a third output set based thereon.   
     
     
         20 . A method as recited in  claim 19 , further comprising:
 automatically triggering a specified action in response to recognizing a pattern represented by the third output set.   
     
     
         21 . A method as recited in  claim 20 , further comprising:
 attempting to recognize, in the input image, a feature from each of a second plurality of feature categories, based on the second output set, by independently using a second plurality of weight patterns to integrate a potential for each unit whose coordinate is represented in the second output set, based on a ranking of coordinates in the second output set; and   generating a plurality of third output sets, each said third output set corresponding to a different one of the second plurality of feature categories.   
     
     
         22 . An apparatus comprising:
 means for inputting an input image;   means for computing a potential as a measure of contrast in the input image for each of a plurality of units of the input image;   means for producing a first output set including a coordinate of each unit of the input image whose potential exceeds a first threshold, such that coordinates are located in the first output set according to a ranking based on potential;   means for integrating, for each coordinate in the first output set, a corresponding potential, based on the ranking;   means for identifying one or more coordinates in the first output set whose potential exceeds a second threshold after said integrating; and   means for including, for each said unit whose potential exceeds the second threshold, the coordinate of the unit in a second output set.   
     
     
         23 . An apparatus as recited in  claim 22 , further comprising:
 means for automatically triggering a specified action in response to recognizing a pattern in the input image, based on the second output set.   
     
     
         24 . An apparatus as recited in  claim 22 , further comprising:
 means for computing the measure of contrast by concurrently   means for identifying positive contrast in the input image, and   means for identifying negative contrast in the input image.   
     
     
         25 . An apparatus as recited in  claim 22 , wherein producing the first output set comprises:
 for each said unit whose potential exceeds the first threshold, linearly rescaling the potential of said unit to a range which corresponds to said number of time slices, and adding the coordinate of the unit to the first output set in a position which is based on the potential of the unit.   
     
     
         26 . An apparatus as recited in  claim 22 , wherein integrating the corresponding potential over a number of time slices comprises:
 adding to a previously computed potential the product of a modulation factor and a corresponding weight value, for each of a plurality of iterations.   
     
     
         27 . An apparatus as recited in  claim 26 , wherein the weight value is from a matrix of weight values, the matrix corresponding to a particular angular orientation, such that a result of said integrating and said identifying is to recognize a feature in the input image which has the particular angular orientation. 
     
     
         28 . An apparatus as recited in  claim 27 , further comprising:
 means for independently and concurrently performing said integrating by using a plurality of different matrices of weight values, wherein said identifying and said including produce a plurality of separate second output sets based on the plurality of different matrices of weight values.   
     
     
         29 . An apparatus as recited in  claim 22 , further comprising:
 means for integrating, for each coordinate in the second output set, the corresponding potential over a range of time slices;   means for identifying one or more coordinates in the second output set whose potential exceeds the second threshold after integration; and   means for including the coordinate of the unit in a third output set, for each said unit whose potential exceeds the second threshold.   
     
     
         30 . An apparatus as recited in  claim 29 , further comprising:
 means for automatically triggering a specified action based on recognizing a pattern in the third output set.   
     
     
         31 . An apparatus comprising:
 a processor; and   a memory storing code which, when executed by the processor, causes instantiation of
 a plurality of converters, each to input an input image and to compute a potential as a measure of contrast in the input image for each of a plurality of units of the input image, each converter further to generate as output a set of coordinates of each unit in the input image whose potential exceeds a first threshold, said output ranked based on potential; and 
 a plurality of observers coupled downstream from the plurality of converters, each observer to process output of an upstream converter or observer, by, 
 for each coordinate in the output of the upstream converter or observer, integrating a corresponding potential over a range of time slices, and 
 for each said unit whose potential exceeds a second threshold after said integrating, including the coordinate of the unit in an output set. 
   
     
     
         32 . An apparatus as recited in  claim 31 , wherein the plurality of converters include a first converter to identify positive contrast in the input image and a second converter to identify negative contrast in the input image. 
     
     
         33 . An apparatus as recited in  claim 31 , wherein each converter generates its corresponding output by
 computing the potential for each of the plurality of units of the input image based on the measure of contrast;   applying each of the potentials to the specified threshold; and   for each said unit whose potential exceeds the specified threshold, linearly rescaling the potential to a range which corresponds to said number of time slices, and adding the coordinate of the unit in the input image to the output in a position which is based on the potential of the unit.   
     
     
         34 . An apparatus as recited in  claim 31 , wherein the memory further stores code which, when executed by the processor, causes execution of an engine to form a detection list for each of the plurality of observers by keeping only a coordinate representing a local maximum from the output set of each said observer. 
     
     
         35 . An apparatus as recited in  claim 31 , wherein each of the observers performs the integrating by adding to a previously computed potential, the product of a modulation factor and a corresponding weight value, for each of a plurality of iterations. 
     
     
         36 . An apparatus as recited in  claim 35 , wherein the weight value is from a matrix of weight values. 
     
     
         37 . An apparatus as recited in  claim 36 , wherein the region is a user selected region. 
     
     
         38 . An apparatus as recited in  claim 36 , wherein each of the observers uses a different matrix of weight values to perform the integrating, and wherein each said matrix corresponds to a different particular angular orientation. 
     
     
         39 . An apparatus comprising:
 a plurality of converters, each to input an input image and to generate an output list indicative of a measure of contrast in the input image, by
 computing a potential for each of a plurality of units of the input image based on the measure of contrast, each unit having a coordinate in the input image, 
 applying each of the potentials to a first threshold, and 
 for each said unit whose potential exceeds the threshold, linearly rescaling the potential to a range corresponding to a number of time slices, and adding the coordinate of the unit in the input image to the output list in a position which is based on the potential of the unit, such that coordinates in the output list are ranked according to potential; 
   a plurality of first stage observers coupled downstream from the plurality of converters, each first stage observer to process an output list of an upstream converter of the plurality of converters, by,
 for each coordinate in the output list in the upstream converter, integrating the corresponding potential over said range of time slices, including applying a modulation factor to a corresponding weight value, wherein the weight value is from a first matrix of weight values, and 
 for each said unit whose potential exceeds a second threshold after said integrating, adding the coordinate of the unit to an output list; 
   a plurality of second stage observers coupled downstream from the plurality of first stage observers, each second stage observer to process an output list of an upstream observer of the plurality of first stage observers, by,
 for each coordinate in the output list in the upstream observer, integrating the corresponding potential over a range of values, including applying a modulation factor to a corresponding weight value, wherein the weight value is from a second matrix of weight values, and 
 for each said unit whose potential exceeds the threshold after said integrating, adding the coordinate of the unit to an output list; and 
   an engine to form a detection list for each of the plurality of first stage and second stage observers by keeping only a coordinate representing a local maximum from the output list of each said observer.   
     
     
         40 . An apparatus as recited in  claim 39 , wherein the plurality of converters include a first converter to identify positive contrast in the input image and a second converter to identify negative contrast in the input image. 
     
     
         41 . An apparatus as recited in  claim 39 , wherein each of observer of the plurality of first and second stage observers uses a pattern filter to determine the corresponding weight value, the pattern filter including:
 a projective weight matrix; and   a receptive weight matrix.   
     
     
         42 . An apparatus as recited in  claim 39 , wherein each of the plurality of first stage observers is configured to recognize a feature at a different particular angle of rotation. 
     
     
         43 . An apparatus as recited in  claim 42 , wherein each of the plurality of second stage observers is configured to generate an output representing a relaxation of locality of a feature recognized in the input image and further representing a prioritizing of endpoints of feature recognized in the input image more highly than a mid-section of the feature.