Capacity of auto-associative networks Learning objectives: explore the number of “memories” that can be stored within a network of neurons, using the model of auto-associative networks explain why even though Hopfield networks work well in artificial applications, this learning rule is not used in the brain

“Has it ever struck you…that life is all memory, except for the one present moment that goes by you so quickly you hardly catch it going? It’s really all memory…except for each passing moment.” ~ Tennessee Williams

Donald Hebb Neurons that fire together, wire together.

Memory fundamentals

Terje Lomo and Tim Bliss’s investigation of the rat hippocampus --Lomo and Bliss’s investigation focused on the perforant pathway --they stimulated the axons in the perforant pathway at high frequency (~100 Hz), then measured the post-synaptic response elicited by a single pre-synaptic stimulation

This was due to the formation of extra post-synaptic High-frequency stimulation of the perforant path resulted in increased amplitude of excitatory post-synaptic potentials After stimulation Long-term potentiation This was due to the formation of extra post-synaptic receptors AND more glutamate release.

Increased post-synaptic response lasts for hours or longer

Place cells http://hargreaves.swong.webfactional.com/place.htm

Place cells Place field of this particular place cell

Mice lose the ability to maintain stable place fields when LTP is inhibited.

Memory “fun facts”

Henry Molaison Suffered brain damage in a bicycle accident at age 7, had temporal lobe epilepsy Had his entire hippocampus removed at age 27, which largely cured his epilepsy… …but it left him unable to form new memories! He could remember what happened before the operation, but could not form memories for facts or events afterward He could remember information for a few minutes, but could not transfer that information into long-term memory

Memento

HM could form new long-term implicit memories --basal ganglia are thought to be the primary place in the brain where procedural memories are stored --cerebellum is also important Principles of Neural Science, Kandel, p. 1446

Different forms of memory Principles of Neural Science, Kandel, p. 1447

Howard Engel Mystery author Stroke left him unable to read Re-learned how to read by tracing the shapes of words on the roof of his mouth with his tongue, and associating the movement with the spoken word!

Modeling the CA3 as an auto-associative network Section 4.7, Tutorial on Neural Systems Modeling, Anastasio

Unbiological

If the Hopfield rule works so well, why doesn’t the brain use it If the Hopfield rule works so well, why doesn’t the brain use it? We’ll explore this in the lab

What if the change in weights doesn’t have to be 1?

Different learning rules are better for different pattern sparsities As the patterns become more sparse, the optimal learning rule is the Hebbian rule.

Neuronal activity in the hippocampus is extremely sparse

CA3 features recurrent connections --see pp. 1512-1513 of Kandel for importance of recurrent connections in CA3 to pattern completion When LTP is knocked out at these specific synapses, mice perform worse in specific memory tasks.

Further experimental investigation is still required to confirm that the CA3 does function as an auto-associative neural network.

The hippocampus helps consolidate memory traces in the cortex --the hippocampus initially serves as a hub that links different cortical regions --the hippocampus then repeatedly activates these different cortical regions, and plasticity mechanisms result in connections forming between the different regions (“neurons that fire together, wire together”) --eventually the connections from the hippocampus to the cortex degrade, leaving just a set of strengthened connections in the cortex; this constitutes the memory trace having been completely transferred to the cortex “The Organization of Recent and Remote Memories,” PW Franklin and B Bontempi, Nature Rev. Neurosi.

Memory traces are thought to be transferred from the hippocampus to the cortex during sleep --this transfer from the hippocampus to the cortex is thought to occur during sleep --during a phase of sleep called slow wave sleep, sequences similar to those observed during waking are “replayed” in both the hippocampus and the cortex --this is thought to be the basis for the transfer of memory traces from the hippo. To the cortex “The memory function of sleep,” S Diekelman and J Born, Nature Rev. Neurosci.

To the lab!