1. Supervised Hebbian Learning

2. Hebb’s Postulate
“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.”
D. O. Hebb, 1949 B A

3. Linear Associator
Training Set:

4. Hebb Rule Simplified Form: Supervised Form: Matrix Form:
Presynaptic Signal
Postsynaptic Signal
Simplified Form:
Supervised Form:
Matrix Form:

5. Batch Operation Matrix Form: W t p P P p T t (Zero Initial Weights) =W t 1 2 Q p T P = P p 1 2 Q = T t 1 2 Q =

6. Performance Analysis Case I, input patterns are orthogonal.q k =
Therefore the network output equals the target:
Case II, input patterns are normalized, but not orthogonal.
Error

7. Example Weight Matrix (Hebb Rule): Tests: Banana Apple
Normalized Prototype Patterns
Weight Matrix (Hebb Rule):
Tests:
Banana
Apple

8. Pseudoinverse Rule - (1)
Performance Index:
Matrix Form: T t 1 2 Q = P p 1 2 Q = || E 2 e i j å =

9. Pseudoinverse Rule - (2)
Minimize:
If an inverse exists for P, F(W) can be made zero:
When an inverse does not exist F(W) can be minimized
using the pseudoinverse:

10. Relationship to the Hebb RuleW T P =
Pseudoinverse Rule
If the prototype patterns are orthonormal:

11. Example

12. Autoassociative Memory

13. Noisy Patterns (7 pixels)
Tests
50% Occluded
67% Occluded
Noisy Patterns (7 pixels)

14. Variations of Hebbian Learning
Basic Rule:
Learning Rate:
Smoothing:
Delta Rule:
Unsupervised: