1. Covariation Learning and Auto-Associative Memory
Chapter 4, Tutorial on Neural Systems Modeling, Anastasio
Learning Objectives:
After mastering the material in this lesson, you should be able to:
explain how the principle “neuron that fire together, wire together” can by used in neural circuits to store memories
compute, by hand, the change in synaptic weights in a neuronal network due to different stimulus patterns
use computer code to simulate memory formation and recall in a neuronal network

2. A simple model of habituation in Aplysia
Input x contacts y with weight v
Tutorial on Neural Systems Modeling, Anastasio, p. 7

3. Donald Hebb
Neurons that fire together, wire together.

4. Auto-associator neural network
Strengthened
connections
between activated
neurons
Pattern of activity
from sensory input
(e.g., hearing
a song)
--note that the connections that do NOT form are just as important as the connections that DO form
--called an “auto-associative neural network” because it learns to associate different neurons with each other
Entire pattern
recovered
Partial pattern
presented (e.g.,
first few notes from
a song)

5. 4 different learning rules
--for “post-synaptic,” you ‘punish’ the connection if the post-synaptic neuron is active when the pre-synaptic neuron is not
“punish” when
post-syn neuron
is active when
pre-syn is not
Not biological—
but useful

7. Summing over multiple patterns
Pattern index
Hebb
Hopfield
Pre-synaptic
Post-synaptic i
Pre-syn neuron index
Post-syn neuron index
Anastasio, p. 103

8. Implementing the Hopfield rule
in a simple network
Hopfield i
p1 = [1 1 0]
p2 = [0 0 1]
How do the weights
change when these
two patterns are presented?

9. Set P = [1 1 0; 0 0 1; 1 0 1] What is the resulting matrix HP
Set P = [1 1 0; 0 0 1; 1 0 1] What is the resulting matrix HP? Can you explain the change?
HP measures the covariance between two neurons. The more
they “do the same thing,” the more positive their connections become.
The more they “do the opposite things,” the more negative their
connections become. If they do the same thing half the time and the
opposite thing half the time, their connection strength will be zero.

10. Run autoConnectivity.m for P = [ ; ] Compare the results for the four different learning rules.

11. Results for four different learning rules