Processor and processing element for use in a neural network

Abstract

A synaptic processor for use in a neural network produces a result signal in response to an input signal. The synaptic processor initializes an input expectation and receives an input signal. The synaptic processor determines a net modification to the the input expectation. The net modification to the input expectation has an increase term and a decrease term. The increase term is determined as a function of the input signal. The decrease term is independent of the magnitude of the input signal and is a function of a decay constant. The synaptic processor produces a result signal in response to the input expectation.

I claim: 
     
       1. A synaptic processor for use in constructing artificial neurons, comprising: a memory device for storing values of first and second expectations, initial values of said first and second expectations, values of a maximum expectation, and an equilibrium expectation, said values of said first and second expectations being bounded by said values of said maximum and equilibrium expectations;   a Neuroprocessor connected to said memory device and being programmed to read said initial values of said first and second expectations from said memory device, to initialize said first and second expectations to said initial values, to receive an input signal, to modify said first expectation as a function of said input signal and said second expectation, to modify said second expectation in response to said first expectation, and to store the values of said first and second expectations in said memory device; and to generate a result signal as a function of said first expectation.   
     
     
       2. A synaptic processor, as set forth in claim 1, wherein said expectations have the relationships:   W.sup.M >W.sup.0 >W.sup.1 >W.sup.E,     where,   W M  =the maximum expectation,   W 0  =the first expectation,   W 1  =the second expectation, and   W E  =the equilibrium expectation.   
     
     
       3. A synaptic processor, as set forth in claim 1, wherein said result signal is a linear function of said first expectation. 
     
     
       4. A synaptic processor, as set forth in claim 1, wherein said result signal is a nonlinear function of said first expectation. 
     
     
       5. A synaptic processor, as set forth in claim 1, wherein said result signal is determined by:   r=k·W.sup.0 ·e.sup.-λτ,     where,   r=the result signal,   k=a first constant,   W 0  =the first expectation,   λ=a second constant, and   τ=a predetermined time differential.   
     
     
       6. A synaptic processor, as set forth in claim 1, wherein said result signal is updated at discrete time intervals of Δt and is determined by:   r(t)=k·W.sup.0, if X>0,       or       r(t)=r(t-Δt)-λ·r(t-Δt),     otherwise   where,   r=the result signal,   t=the current time,   k=a first constant,   W 0  =the first expectation, and   λ=a second constant.   
     
     
       7. A synaptic processor, as set forth in claim 1, wherein said first expectation is a function of the magnitude of said input signal. 
     
     
       8. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to decrease said first expectation in response to said second expectation. 
     
     
       9. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to determine the difference between said first and second expectations and to decrease said first expectation in response to said difference and a first decay constant. 
     
     
       10. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to increase said first expectation in response to the magnitude of said input signal and said maximum expectation. 
     
     
       11. A synaptic processor, as set forth in claim 10, wherein said Neuroprocessor is programmed to determine the difference between said first and maximum expectations and to increase said first expectation in response to said input signal, said difference, and a first rise constant. 
     
     
       12. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to increase said second expectation in response to said first expectation. 
     
     
       13. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to decrease said second expectation in response to said equilibrium expectation. 
     
     
       14. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to determine a net modification and to add said net modification to said first expectation, wherein said net modification includes a first increase term and a first decrease term, said first increase term being a function of said first expectation and a first rise constant and said first decrease term being a function of said first expectation and a first decay constant. 
     
     
       15. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to determine a net modification and to add said net modification to said second expectation, wherein said net modification includes a second increase term and a second decrease term, said second increase term being a function of said second expectation and a second rise constant, and said second decrease term being a function of said second expectation and a second decay constant. 
     
     
       16. A synaptic processor, as set forth in claim 1, wherein said Neuroprocessor is programmed to modify said equilibrium expectation, wherein the net modification to said equilibrium expectation includes an increase term and a decrease term, said increase term being a function of said second expectation and a rise constant and said decrease term being a function of a second decay constant. 
     
     
       17. A synaptic processor, as set forth in claim 1, wherein said first expectation is determined by: ##EQU14## where, W 0  =the first expectation, W 1  =the second expectation,   X=the input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant,   W M  =the maximum expectation, and   W p   0  =the previous first expectation.   
     
     
       18. A synaptic processor, as set forth in claim 1, wherein said first expectation ms determined by: ##EQU15## where, W 0  =the first expectation, W 1  =the second expectation,   X=the input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant,   W M  =the maximum expectation, and   W 0   p  =the previous first expectation.   
     
     
       19. A synaptic processor, as set forth in claim 1, wherein said second expectation is determined by: ##EQU16## where, W 1  =the second expectation, α.sup. = a second rise constant,   δ 1  =a second decay constant,   W E  =the equilibrium expectation, and   W p   1  =the previous second expectation.   
     
     
       20. A synaptic processor for use in constructing artificial neurons, comprising: a memory device for storing values of a first expectation and at least one second expectation and initial values of said first and at least one second expectation, wherein the number of second expectations is denoted by L;   a Neuroprocessor connected to said memory device, said Neuroprocessor being programmed to read said initial values of said first and second expectations from said memory device, to initialize said first and at least one second expectations to said initial values, to receive an input signal, to responsively modify said first expectation, to modify said at least one second expectation, wherein said at least one second expectation is denoted by W 1  and is a function of:   said first expectation, if l=1, or a previous second expectation,   W l-1 , otherwise, and to responsively generate a result signal.   
     
     
       21. A synaptic processor, as set forth in claim 20, wherein said neuroprocessor is programmed to receive a potentiation signal, and wherein said potentiation signal is in one of a permit state and an inhibit state, and wherein said second expectation is modified when said potentiation signal is in said permit state and where in said second expectation is not modified when said potentiation signal is in said inhibit state. 
     
     
       22. A Neurocomputer used in constructing a neural network, comprising: a memory device for storing values of an input expectation and first and second expectations, initial values of said input expectation and said first and second expectations and a value of a first decay constant;   a first synaptic processor for producing a first result signal in response to a first input signal, said first synaptic processor including: a first Neuroprocessor connected to said memory device, said Neuroprocessor being programmed to read said initial value of said input expectation, to initialize said input expectation to said initial value, to receive said first input signal, to modify said input expectation, wherein the net modification to said input expectation includes an increase term and a decrease term, said increase term being responsive to the magnitude of said input signal and said decrease term being independent of said magnitude and a function of a first decay constant, and to responsively generate a first result signal; and     a second synaptic processor for producing a second output signal in response to a second input signal, said second synaptic processor including: a second Neuroprocessor connected to said memory device, said second Neuroprocessor being programmed to read said initial values of said first and second expectations, to initialize said first and second expectations to said initial values, to receive said second input signal, to modify said first expectation in response to said second input signal and said second expectation, to receive said first result signal and to responsively modify said second expectation, and to responsively generate a second result signal.     
     
     
       23. An artificial neuron, comprising: a memory device for storing values of a threshold signal and a plurality input expectations and initial values of said input expectations;   at least two input synaptic processors for producing respective result signals in response to respective input signals, said input synaptic processors including: a Neuroprocessor connected to said memory device, said Neuroprocessor being programmed to read said initial values of said input expectations, to initialize said input expectations to said respective initial value, to receive respective input signals, to modify said input expectations, wherein the net modification to each input expectation has an increase term and a decrease term, said increase term being responsive to the magnitude of said respective input signal and said decrease term being independent of said magnitude and a function of a respective decay constant, and to responsively generate respective result signals; and   wherein said Neuroprocessor includes a summation and output processor, said summation and output processor being adapted to receive said result signals, sum said result signals, and responsively generate an output signal.     
     
     
       24. An artificial neuron, comprising: a memory device for storing values of a threshold signal and a plurality of first and second expectations and initial values of said first and second expectations;   at least two input synaptic processors connected to said memory device, said at least two input synaptic processors being programmed to generate respective result signals in response to respective input signals, each said input synaptic processor including: a Neuroprocessor connected to said memory device, said Neuroprocessor being programmed to read said initial values of said first and second expectations, to initialize said first and second expectations to said initial values, to receive an input signal, to modify said first expectation as a function of said input signal and said second expectation, and to modify said second expectation in response to said first expectation, and to generate a result signal; and     wherein said artificial neuron is adapted to receive said result signals, sum said result signals, and responsively generate an output signal.   
     
     
       25. A synaptic processor for use in a neural network, for producing a result signal in response to an input signal, comprising: means for initializing first and second expectations, said first and second expectations being bounded by a maximum expectation and an equilibrium expectation;   means for receiving said input signal and said first expectation and generating a modified first expectation;   means for receiving said second expectation and said modified first expectation, modifying said second expectation as a function of said modified first expectation, and responsively generating a modified second expectation;   wherein said modified second expectation is provided to said modified first expectation generating means and wherein said modified first expectation is a function of said modified second expectation and said input signal;   means for receiving said modified first expectation and producing said result signal as a function of said first expectation; and   means for receiving said result signal and providing said result signal to the neural network.   
     
     
       26. A synaptic processor, as set forth in claim 25, wherein said expectations have the relationships:   W.sup.M >W.sup.0 >W.sup.1 >W.sup.E,     where,   W M  =the maximum expectation,   W 0  =the modified first expectation,   W 1  =the modified second expectation, and   W E  =the equilibrium expectation.   
     
     
       27. A synaptic processor, as set forth in claim 25, wherein said result signal is a linear function of said modified first expectation. 
     
     
       28. A synaptic processor, as set forth in claim 25, wherein said result signal is a nonlinear function of said modified first expectation. 
     
     
       29. A synaptic processor, as set forth in claim 25, wherein said result signal is determined by:   r=k·W.sup.0 ·e.sup.-λτ,     where,   r=the result signal,   k=a first constant signal,   W 0  =the modified first expectation,   λ=a second constant, and   τ=a predetermined time differential.   
     
     
       30. A synaptic processor, as set forth in claim 25, wherein said result signal is updated at discrete time intervals of Δt and is determined by:   r(t)=k·W.sup.0, if X>0,       or       r(t)=r(t-Δt)-λ·r(t-Δt),     otherwise,   where   r=the result signal,   t=the current time,   k=a first constant,   W 0  =the modified first expectation, and   λ=a second constant.   
     
     
       31. A synaptic processor, as set forth in claim 25, wherein said modified first expectation is a function of the magnitude of said input signal. 
     
     
       32. A synaptic processor, as set forth in claim 25, wherein said first expectation modifying means includes means for decreasing said first expectation in response to said modified second expectation. 
     
     
       33. A synaptic processor, as set forth in claim 32, wherein said first expectation decreasing means includes means for determining the difference between said first expectation and said second modified expectation and decreasing said first expectation in response to said difference and a first decay constant. 
     
     
       34. A synaptic processor, as set forth in claim 25, wherein said first expectation modifying means includes means for increasing said first expectation in response to the magnitude of said input signal and said maximum expectation. 
     
     
       35. A synaptic processor, as set forth in claim 34, wherein said first expectation increasing means includes means for determining the difference between said first and maximum expectations and increasing said first expectation in response to said input signal, said difference, and a first rise constant. 
     
     
       36. A synaptic processor, as set forth in claim 34, wherein said second expectation increasing means includes means for determining the difference between said second expectation and said first modified expectation and increasing said second expectation in response to said difference and a second rise constant. 
     
     
       37. A synaptic processor, as set forth in claim 25, wherein said second expectation modifying means includes means for increasing said second expectation in response to said first expectation. 
     
     
       38. A synaptic processor, as set forth in claim 25, wherein said second expectation modifying means includes means for decreasing said second expectation in response to said equilibrium expectation. 
     
     
       39. A synaptic processor, as set forth in claim 25, wherein said second expectation decreasing means includes means for determining the difference between said second and equilibrium expectations and decreasing said second expectation in response to said difference and a second decay constant. 
     
     
       40. A synaptic processor, as set forth in claim 25, wherein said first modifying means includes means for increasing said first expectation in response to the magnitude of said input and decreasing said first expectation in response to said modified second expectation. 
     
     
       41. A synaptic processor, as set forth in claim 25, wherein said second modifying means includes means for increasing said second expectation in response to said modified first expectation and decreasing said second expectation in response to said equilibrium expectation. 
     
     
       42. A synaptic processor, as set forth in claim 25, wherein said first expectation modifying means includes means for determining a net modification and adding said net modification to said first expectation, wherein said net modification includes a first increase term and a first decrease term, said first increase term being a function of said first expectation and a first rise constant and said first decrease term being a function of said first expectation and a first decay constant. 
     
     
       43. A synaptic processor, as set forth in claim 25, wherein said second expectation modifying means includes means for determining a net modification and adding said net modification to said second expectation, wherein said net modification includes a second increase term and a second decrease term, said second increase term being a function of said second expectation and a second rise constant, and said second decrease term being a function of said second expectation and a second decay constant. 
     
     
       44. A synaptic processor, as set forth in claim 25, wherein said first expectation modifying means includes means for determining a first net modification and adding said first net modification to said first expectation, wherein said first net modification includes a first increase term and a first decrease term, said first increase term being a function of said first expectation and a first rise constant and said first decrease term being a function of said first expectation and a first decay constant, and wherein said second expectation modifying means includes means for determining a second net modification to said second expectation and adding said second net modification to said second expectation, wherein said second net modification includes a second increase term and a second decrease term, said second increase term being a function of said second expectation and a second rise constant, and said second decrease term being a function of said second expectation and a second decay constant. 
     
     
       45. A synaptic processor as set forth in claim 25, including means for modifying said equilibrium expectation, wherein the net modification to said equilibrium expectation includes an increase term and a decrease term, said increase term being a function of said modified second expectation and a rise constant and said decrease term being a function of a second decay constant. 
     
     
       46. A synaptic processor, as set forth in claim 25, wherein said modified first expectation is determined by: ##EQU17## where, W 0  =the modified first expectation, W 1  =the modified second expectation,   X=the input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant,   W M  =the maximum expectation, and   W p   0  =the previous modified first expectation.   
     
     
       47. A synaptic processor, as set forth in claim 25, wherein said modified first expectation is determined by: ##EQU18## where, W 0  =the modified first expectation, W 1  =the modified second expectation,   X=the input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant,   W M  =the maximum expectation, and   W p   0  =the previous modified first expectation.   
     
     
       48. A synaptic processor, as set forth in claim 25, wherein said modified second expectation is determined by: ##EQU19## where, W 1  =the modified second expectation, α.sup. = a second rise constant,   δ 1  =a second decay constant,   W E  =the equilibrium expectation, and   W p   1  =the modified previous second expectation.   
     
     
       49. A synaptic processor for use in a neural network, for producing a result signal in response to an input signal, comprising: means for initializing a first expectation and at least one second expectation, said first and second expectations being bounded by a maximum expectation and an equilibrium expectation, wherein the number of second expectations is denoted by L;   means for receiving said input signal and said first expectation and generating a modified first expectation;   means for receiving said modified first expectation and said at least one second expectation, modifying said at least one second expectation, and responsively generating at least one modified second expectation, wherein said at least one modified second expectation is denoted by W 1  and is a function of: said first expectation, if l=1, or a previous second expectation, W l-1 , otherwise;   wherein said at least one modified second expectations provided to said modified first expectation generating means and wherein said modified first expectation signa is a function of said at least one modified second expectation and said input signal;     means for receiving said modified first expectation and producing said result signal as a function of said modified first expectation; and   means for receiving said result signal and providing said result signal to the neural network.   
     
     
       50. A synaptic processor, as set forth in claim 49, wherein said modified first expectation is determined by: ##EQU20## where, W 0  =the modified first expectation, W p   0  =the previous modified first expectation,   X=the input signal,   α.sup. = a rise constant,   W M  =the maximum expectation,   δ 0  =a decay constant, and   W 1  =the modified second expectation.   
     
     
       51. A synaptic processor, as set forth in claim 49, wherein said modified first expectation is determined by: ##EQU21## where, W 0  =the modified first expectation, W p   0  =the previous modified first expectation,   X=the input signal,   α.sup. = a rise constant,   δ 0  =a decay constant, and   W 1  =the modified second expectation.   
     
     
       52. A synaptic processor, as set forth in claim 49, including means for receiving a second input signal and responsively producing a potentiation signal and wherein said at least one modified second expectation is a function of said potentiation signal. 
     
     
       53. A synaptic processor, as set forth in claim 52, wherein each of said at least one modified second expectations is determined by: ##EQU22## if l>0 AND l <L, or ##EQU23## if l=L, where, W 0  =the modified first expectation,   W l  =the modified second expectation,   W p   1  =the previous value of the modified second expectation,   P=the potentiation signal,   α.sup. = a rise constant,   W M  =the maximum expectation,   δ 1  =the decay constant, and   W E  =the equilibrium expectation.   
     
     
       54. A synaptic processor for use in a neural network, for producing a result signal in response to a first input signal and a second input signal, comprising: means for initializing first and second expectations, said first and second expectations being bounded by a maximum expectation and a equilibrium expectation;   means for receiving said first input signal and said first expectation and generating a modified first expectation;   means for receiving said second input signal and responsively producing a potentiation signal;   means for receiving said first modified expectation, said second expectation and said potentiation signal, modifying said second expectation as a function of said modified first expectation and said potentiation signal, and responsively generating a modified second expectation;   wherein said modified second expectation is provided to said first modified expectation generating means and wherein said modified first expectation is a function of said modified second expectation and said first input signal;   means for receiving said first input signal and said modified first expectation and producing said result signal as a function of said first input signal and said modified first expectation; and   means for receiving said result signal and providing said result signal to the neural network.   
     
     
       55. A synaptic processor, as set forth in claim 54, wherein said result signal is a linear function of said modified first expectation. 
     
     
       56. A synaptic processor, as set forth in claim 54, wherein said result signal is a nonlinear function of said modified first expectation. 
     
     
       57. A synaptic processor, as set forth in claim 54, wherein said result signal is determined by:   r=k·W.sup.0 ·e.sup.-λτ,     where,   r=the result signal,   k=a first constant,   W 0  =the first expectation,   λ=a second constant, and   τ=a predetermined time differential.   
     
     
       58. A synaptic processor, as set forth in claim 54, wherein said result signal is updated at discrete time intervals of Δt and is determined by:   r(t)=k·W.sup.0, if X>0,       or       r(t)=t(t-Δt)-λ·r(t-Δt),     otherwise,   where,   X=the input signal,   t=the current time,   r=the result signal,   k=a first constant,   W 0  =the first expectation, and   λ=a second constant.   
     
     
       59. A synaptic processor, as set forth in claim 54, wherein said first expectation is a function of the magnitude of said input signal. 
     
     
       60. A synaptic processor, as set forth in claim 54, wherein said initializing means includes means for initializing a gating signal, and wherein said potentiation signal producing means includes means for comparing said second input signal and said gating signal and producing said potentiation signal in response thereof. 
     
     
       61. A synaptic processor, as set forth in claim 60, wherein said potentiation signal producing means includes means for determining said gating signal and said second input signal. 
     
     
       62. A synaptic processor, as set forth in claim 61, wherein said second input signal determining means includes means for receiving a plurality of interaction signals and summing said interaction signals. 
     
     
       63. A synaptic processor, as set forth in claim 60, wherein said potentiation signal is in a permit state if said second input signal exceeds said gating signal and in an inhibit state otherwise and wherein said second expectation is modified when said potentiation signal is in said permit state and where in said second expectation is not modified when said potentiation signal is in said inhibit state. 
     
     
       64. A synaptic processor, as set forth in claim 63, wherein said second expectation modifying means includes means for increasing said second expectation in response to said first expectation if said potentiation signal is in said permit state. 
     
     
       65. A synaptic processor, as set forth in claim 54, including means for modifying said equilibrium expectation, wherein the net modification to said equilibrium expectation includes an increase term and a decrease term, said increase term being a function of said second expectation and a rise constant, and said decrease term signal being a function of a decay constant. 
     
     
       66. A synaptic processor, as set forth in claim 54, wherein said modified first expectation is determined by: ##EQU24## where, W 0  =the modified first expectation, W 1  =the modified second expectation,   W p   0  =the previous modified first expectation,   X=the first input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant, and   W M  =the maximum expectation.   
     
     
       67. A synaptic processor, as set forth in claim 54, wherein said modified first expectation is determined by: ##EQU25## where, W 0  =the modified first expectation, W p   0  =the previous modified first expectation,   W 1  =the modified second expectation,   X=the first input signal,   α.sup. = a first rise constant, and   δ 0  =a first decay constant.   
     
     
       68. A synaptic processor, as set forth in claim 54, wherein said modified second expectation is determined by: ##EQU26## where, W 0  =the modified first expectation, W 1  =the modified second expectation,   X=the first input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant,   W M  =the maximum expectation, and   W p   1  =the previous modified second expectation.   
     
     
       69. A processing element for use in a neural network, comprising: a first synaptic processor for producing a first result signal in response to a first input signal, comprising: means for initializing an input expectation, said first input expectation being bounded by a first maximum expectation and a first equilibrium expectation; means for receiving said first input signal and said input expectation, responsively modifying said input expectation, and responsively generating a modified input expectation, wherein the net modification to said input expectation includes a first increase term and a first decrease term, said first increase term being responsive to said first input signal, and said first decrease term being a function of said first equilibrium expectation; and     means for receiving said modified input expectation and producing said first result signal as a function of said modified input expectation;     a second synaptic processor for producing a second result signal in response to a second input signal, including: means for initializing first and second expectations, said first and second expectations being bounded by a second maximum expectation and a second equilibrium expectation;   means for receiving said second input signal and said first expectation and generating a modified first expectation;   means for receiving said first result signal and responsively producing a potentiation signal;   means for receiving said first modified expectation, said second expectation and said potentiation signal, modifying said second expectation as a function of said modified first expectation and said potentiation signal and responsively generating a modified second expectation;   wherein said modified second expectation is provided to said modified first expectation generating means and wherein said first modified first expectation is a function of said second input signal and said second modified expectation; and   means for receiving said first input signal and said first modified expectation and producing said second result signal as a function of said second input signal and said modified first expectation;     means for receiving said first and second result signals and providing said result signals to the neural network.   
     
     
       70. A processing element, as set forth in claim 69, wherein said first result signal is determined by:   r.sup.1 =k·W.sup.01 ·e.sup.-λτ,     where,   r 1  =the first result signal,   k=a first constant,   W 01  =the modified input expectation,   λ=a second constant, and   τ=a predetermined time differential.   
     
     
       71. A processing element, as set forth in claim 69, wherein said first result signal is updated at discrete time intervals of Δt and is determined by:   r.sup.1 (t)=k·W.sup.01, if X.sup.1 >0,       or       r.sup.1 (t)=r(t-Δt)-λ·r(t-Δt),     otherwise   where   X 1  =the first input signal,   t=the current time,   r 1  =the first result signal,   k=a first constant,   W 01  =the modified input expectation, and   λ=a second constant.   
     
     
       72. A processing element, as set forth in claim 69, wherein said first decrease term is proportional to the difference between said modified input expectation and said first equilibrium expectation and in accordance with a first decay constant. 
     
     
       73. A processing element, as set forth in claim 69, wherein said first increase term is a function of the difference between said modified input expectation and said first maximum expectation and a first rise constant. 
     
     
       74. A processing element, as set forth in claim 69, wherein said first synaptic processor includes means for modifying said first equilibrium expectation, wherein the net modification to said first equilibrium expectation has a second increase term and a second decrease term, said second increase term being a function of said input expectation and a second rise constant, and said second decrease term being a function of a second decay constant. 
     
     
       75. A processing element, as set forth in claim 69, wherein said modified input expectation is determined by: ##EQU27## where, W 01  =the modified input expectation, X 1  =the input signal,   α 0  = a first rise constant,   δ 01  =a first decay constant,   W M1  =the maximum expectation, and   W p   01  =the previous modified input expectation.   
     
     
       76. A processing element, as set forth in claim 69, wherein said modified input expectation is determined by: ##EQU28## where, W 01  =the modified input expectation, X 1  =the input signal,   α 0  = a first rise constant,   δ 01  =a first decay constant, and   W p   01  =the previous modified input expectation.   
     
     
       77. A processing element, as set forth in claim 69, wherein said second maximum, second equilibrium, and said first and second expectations have the relationships:   W.sup.M2 >W.sup.02 >W.sup.12 >W.sup.E2,     where,   W M2  =the second maximum expectation,   W 02  =the modified first expectation,   W 12  =the modified second expectation, and   W E2  =the equilibrium expectation.   
     
     
       78. A processing element, as set forth in claim 69, wherein said second result signal, r 2 , is determined by:   r.sup.2 =k·W.sup.02 ·e.sup.-λτ,     where,   r 2  =the second result signal,   k=a first constant,   W 02  =the modified first expectation,   λ=a second constant, and   τ=a predetermined time differential.   
     
     
       79. A processing element, as set forth in claim 69, wherein said second result signal is updated at discrete time intervals of Δt and is determined by:   r.sup.2 (t)=k·W.sup.02, if X.sup.2 >0,       or       r.sup.2 (t)=r(t-Δt)-λ·r(t-Δt),     otherwise,   where,   X 2  =the second input signal,   t==the current time,   r 2  =the second result signal,   k=a first constant,   W 02  =the modified first expectation, and   λ=a second constant.   
     
     
       80. A processing element, as set forth in claim 69, wherein said modified first expectation is a function of the magnitude of said first input signal. 
     
     
       81. A processing element, as set forth in claim 69, wherein said first expectation modifying means includes means for determining a net modification and adding said net modification to said first expectation, wherein said net modification includes a third increase term and a third decrease term, said third increase term being a function of said first expectation and a third rise constant, and said third decrease term being a function of said first expectation and a third decay constant. 
     
     
       82. A processing element, as set forth in claim 69, wherein said second expectation modifying means includes means for determining a net modification and adding said net modification to said second expectation, wherein said net modification includes a fourth increase term and a fourth decrease term, said fourth increase term being a function of said modified first expectation and a fourth rise constant, and said fourth decrease term being a function of said modified first expectation and a fourth decay constant. 
     
     
       83. A processing element, as set forth in claim 69, wherein said first expectation modifying means includes means for determining a first net modification and adding said first net modification to said first expectation, wherein said first net modification includes a third increase term and a third decrease term, said third increase term being a function of said first expectation and a third rise constant, and said third decrease term being a function of said first expectation, and a third decay constant, and wherein said second expectation modifying means includes means for determining a second net modification to said second expectation and adding said second net modification to said second expectation, wherein said second net modification includes a fourth increase term and a fourth decrease term, said fourth increase term being a function of said modified first expectation and a fourth rise constant, and said fourth decrease term being a function of said modified first expectation and a second decay constant. 
     
     
       84. A processing element, as set forth in claim 69, including means for modifying said second equilibrium expectation, wherein the net modification to said second equilibrium expectation has a fifth increase term and a fifth decrease term, said fifth increase term being a function of said second expectation and a fifth rise constant, and said fifth decrease term being a function of a fifth decay constant. 
     
     
       85. A processing element, as set forth in claim 69, wherein said modified first expectation is determined by: ##EQU29## where, W 02  =the modified first expectation, W 12  =the modified second expectation,   X 2  =the second input signal,   α 0  = a third rise constant,   δ 02  =a third decay constant, and   W p   02  =the previous modified first expectation.   
     
     
       86. A processing element, as set forth in claim 69, wherein said modified first expectation is determined by: ##EQU30## where, W 02  =the modified first expectation, W 12  =the modified second expectation,   X 2  =the second input signal,   α 0  = a third rise constant,   δ 02  =a third decay constant, and   W p   02  =the previous modified first expectation.   
     
     
       87. A processing element, as set forth in claim 69, wherein said modified second expectation, W 12 , is determined by: ##EQU31## where, W 12  =the modified second expectation, P=the potentiation signal,   α 1  = a fourth rise constant,   δ 12  =a fourth decay constant, and   W p   12  =the previous modified second.   
     
     
       88. A processing element for use in a neural network, comprising: means for producing a threshold signal;   at least two input synaptic processors for producing respective result signals in response to respective input signals, each said input synaptic processor including: means for initializing first and second expectations, said first and second expectations being bounded by a maximum expectation and an equilibrium expectation;   means for receiving said input signal and said first expectation and generating a modified first expectation;   means for receiving said modified first expectation and said second expectation, modifying said second expectation as a function of said modified first expectation, and responsively generating a modified second expectation;   wherein said modified second expectation is provided to said modified first expectation generating means and wherein said modified first expectation is a function of said modified second expectation and input signal; and   means for receiving said modified first expectation and producing said result signal as a function of said modified first expectation; and     means for receiving said result signals, summing said result signals, and responsively producing a sigma signal; and   means for receiving said sigma signal and said threshold signal and responsively producing an output signal.   
     
     
       89. A processing element, as set forth in claim 88, wherein said expectations have the relationships:   W.sup.M >W.sup.0 >W.sup.1 >W.sup.E,     where,   W M  =the maximum expectation,   W 0  =the modified first expectation,   W 1  =the modified second expectation, and   W E  =the equilibrium expectation.   
     
     
       90. A processing element, as set forth in claim 88, wherein said first expectation modifying means includes means for determining a first net modification and adding said first net modification to said first expectation, wherein said first net modification has a first increase term and a first decrease term, said first increase term being a function of said first expectation and a first rise constant and said first decrease term being a function of said first expectation and a first decay constant, and wherein said second expectation modifying means includes means for determining a second net modification to said second expectation and adding said second net modification to said second expectation, wherein said second net modification includes a second increase term and a second decrease term, said second increase term being a function of said second expectation and a second rise constant, and said second decrease term being a function of said second expectation and a second decay constant. 
     
     
       91. A processing element, as set forth in claim 88, including means for modifying said equilibrium expectation, wherein the net modification to said equilibrium expectation includes an increase term and a decrease term, said increase term being a function of said second expectation and a rise constant and said decrease term being a function of a second decay constant. 
     
     
       92. A processing element, as set forth in claim 88, wherein said modified first expectation is determined by: ##EQU32## where, W 0  =the modified first expectation, W 1  =the modified second expectation,   X=the input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant,   W M  =the maximum expectation, and   W p   0  =the previous modified first expectation.   
     
     
       93. A processing element, as set forth in claim 88, wherein said modified first expectation is determined by: ##EQU33## where, W 0  =the modified first expectation, W 1  =the modified second expectation,   X=the input signal,   α.sup. = a first rise constant,   δ 0  =a first decay constant, and   W p   0  =the previous modified first expectation.   
     
     
       94. A processing element, as set forth in claim 88, wherein said modified second expectation is determined by: ##EQU34## where, W 1  =the modified second expectation, α.sup. = a second rise constant,   δ 0  =a second decay constant,   W E  =the equilibrium expectation, and   W p   1  =the previous modified second expectation.   
     
     
       95. A processing element, as set forth in claim 88, wherein said threshold signal producing means includes an output processor for receiving said output signal and responsively producing a threshold modifying signal. 
     
     
       96. A neural network implemented by application specific integrated chips, the neural network composed of a plurality of synaptic processors, each synaptic processor defined by a series of state variables; comprising: an input processor for receiving external signals and passing said external signals to the neural network;   a processor for updating the state variables of the synaptic processors;   a memory device for storing the state variables;   an output processor for receiving signals from the neural network and passing said signals to the outside world;   wherein said neural network is implemented via programming of said processor; and wherein said processor includes, for each synaptic processor: means for initializing first and second expectations, said first and second expectations being bounded by a maximum expectation and an equilibrium expectation;   means for receiving said input signal and said first expectation and generating a modified first expectation;   means for receiving said second expectation and said modified first expectation, modifying said second expectation as a function of said modified first expectation, and responsively generating a modified second expectation;   wherein said modified second expectation is provided to said modified first expectation generating means and wherein said modified first expectation is a function of said modified second expectation and said input signal;   means for receiving said modified first expectation and producing said result signal as a function of said first expectation; and   means for receiving said result signal and providing said result signal to the neural network.