Lecture 3: Learning and Memory Prof.dr. Jaap Murre University of Maastricht University of Amsterdam

Overview We will study Hebbian learning and the formation of categories We will do some basic memory experiments Examine various forms of memory We will try to locate memory in the brain and relate brain lesions to amnesia We will also briefly explore executive functions in the frontal lobes We will look at memory improvement

With Hebbian learning, two learning methods are possible With unsupervised learning there is no teacher: the network tries to discern regularities in the input patterns With supervised learning an input is associated with an output –If the input and output are the same, we speak of auto-associative learning –If they are different it is called hetero- associative learning

Supervised learning with Hopfield (1982) network Bipolar activations –-1 or 1 Symmetric weights (no self weights) –w ij = w ji Asynchronous update rule –Select one neuron randomly and update it Simple threshold rule for updating

Energy of a Hopfield network Energy E = - ½  i,j w ji a i a j  E = - ½  i (w ji a i + w ij a i )  a j = -  i w ji a i  a j Net input to node j is  i w ji a i = net j Thus, we can write  E = - net j  a j

The energy minimization question can also be turned around Given a i and a j, how should we set the weight w ji = w ji so that the energy is minimized?  E = - ½   w ji a i a j, so that –when a i a j = 1,  w ji must be positive –when a i a j = -1,  w ji must be negative For example,  w ji =  a i a j, where  is a learning constant

Hebb and Hopfield When used with Hopfield type activation rules, the Hebb learning rule places patterns at attractors If a network has n nodes, 0.15n random patterns can be reliably stored by such a system For complete retrieval it is typically necessary to present the network with over 90% of the original pattern

We will look at an example of competitive learning Competitive learning is a form of unsupervised learning

Example of competitive learning: Stimulus ‘at’ is presented ato 12

Example of competitive learning: Competition starts at category level ato 12

Example of competitive learning: Competition resolves ato 12

Example of competitive learning: Hebbian learning takes place ato 12 Category node 2 now represents ‘at’

Presenting ‘to’ leads to activation of category node 1 ato 12

ato 12

ato 12

ato 12

Category 1 is established through Hebbian learning as well ato 12 Category node 1 now represents ‘to’

Before we continue... Everybody on the right of the classroom, please, close their eyes until the following words have been presented The others, pay attention to the following 10 words. You will be asked to remember them later Don’t write them down!

table

car

tree

computer

monkey

paper

scissors

tennis

dessert

bread

Now for the other half... Everybody on the left of the classroom, please, close their eyes until the following words have been presented The others, pay attention to the following 10 words. You will be asked to remember them later Don’t write them down!

table

car

tree

computer

monkey

paper

assassin

tennis

dessert

bread

Memory and attention are strongly intertwined Paying attention can be seen as holding in memory Attention is required for rehearsal The longer an item is attended (held in memory), the higher the chance it will be remembered later

Desimone’s study of V4 * neurons * V4 is visual cortex before inferotemporal cortex (IT)

Brown-Peterson task Try to remember three letters, e.g., XJC When given a number (e.g., 307), start counting backward in threes (307, 304, 301, 298, …) When the Write! text appears, write down the letters you remember This has to be done at least several times to obtain the effect

Ready!

RGP

875

Write!

Ready!

ZQN

317

Write!

Ready!

HWB

504

Write!

Typical results of the Brown- Peterson task The results typically show very low memory performance The reason is that rehearsal of the letters is prevented by the counting task

Before we continue, Write down all the words you remember from the presentation Make sure you do not verbalize them at this moment We will verify the result in a minute, but first we have the following two puzzles

Fragment completion Try to complete the following English word fragments You have 30 seconds Each dot (.) stands for a letter Don’t verbalize! (So, we can obtain a better sample)

s..ss..s

.ss.ss..

Stop!

The correct answers were scissors assassin

The presented words were: Left half –table –car –tree –computer –monkey –paper –scissors –tennis –dessert –bread Right half –table –car –tree –computer –monkey –paper –assassin –tennis –dessert –bread

Short-term memory and working memory

Typical memory recall curve

Atkinson and Shiffrin model (1968)

This model had some limitations Ba, ta, pa, ta, pa, ba is much more difficult to remember than ba, bu, bi, bu, bi, ba Hence, there are phonological effects in short-term memory

Working memory model by Alan Baddeley and Graham Hitch (1974)

Executive functions What controls the memory retrieval process? How does the control process work? What determines which areas of brain are ‘allowed’ to active in the first place?

The elusive frontal cortex

Brodmann’s map

Anatomy of prefrontal cortex Strong lateral connectivity via stellates in Layer IV No direct connections to motor outputs Certain cells fire strongly and selectively during the delay period of a task in relation to certain aspects of the taks (e.g., position), especially in area 46 surrounding the principal sulcus

Goldman-Rakic studies of Piaget’s A  B Paradigm Infants persist in reaching for a target even if they have observed it being hidden in another place and older infants will do this if the delay is large enough (2-5 s at months) Still older infants will not do this Monkeys with dorsolateral prefrontal lesions show similar behavior (delays > 2 s)

Similarities with ‘prefrontal’ patients Prefrontal patients show perseveration on the Wisconsin card sorting test There is evidence that also in adult humans such behavior is mainly caused by lesions to the dorsolateral prefrontal cortex

There is also a working memory aspect to the task The subject must keep in mind where the hiding place was, which may involve a kind of ‘working memory’ lasting several seconds In other experiments Patricia Goldman-Rakic has implicated area 46 as performing a type of working memory function Alan Baddeley is not convinced that this type of working memory is similar to his own concept

Long-term memory and amnesia

Larry Squire’s taxonomy of long- term memory

Forgetting There is currently no theory that explains why we forget Forgetting seems to follow rather strict rules, but even these have not been fully explored It is postulated that very well rehearsed knowledge will never be forgotten (Harry Barrick’s ‘permastore’)

10,000 to 100,000 connections per neuron

Anatomy of a neuron

Memory is stored in the connections (synapses) between neurons

Main storage sites of memories in brain Neocortex Hippocampus

Personen, dieren, en voorwerpen in de temporaalschors

Ook met breinscans worden dergelijke locaties aangetroffen

Position of the hippocampus

The neocortex Whale (5 x human) Human

1000 PC hard disks The neocortex contains about 10 billion Every neuron connects to 10,000 others That amounts to 100,000 billion connections that each can store about 1 byte The neocortex thus has the equivalent capacity of at least a 1000 hard disks of 100 giga bytes

TraceLink model A connectionist model of memory consolidation and amnesia

TraceLink model: structure

System 1: Trace system Function: Substrate for bulk storage of memories, ‘association machine’ Corresponds roughly to neocortex

System 2: Link system Function: Initial ‘scaffold’ for episodes Corresponds roughly to hippocampus and certain temporal and perhaps frontal areas

System 3: Modulatory system Function: Control of plasticity Involves at least parts of the hippocampus, amygdala, fornix, and certain nuclei in the basal forebrain and in the brain stem

Stages in episodic learning

Retrograde amnesia Primary cause: loss of links Ribot gradients Shrinkage

Anterograde amnesia Primary cause: loss of modulatory system Secondary cause: loss of links Preserved implicit memory

Semantic dementia The term was adopted recently to describe a new form of dementia, notably by Julie Snowden et al. (1989, 1994) and by John Hodges et al. (1992, 1994) Semantic dementia is almost a mirror- image of amnesia

Neuropsychology of semantic dementia Progressive loss of semantic knowledge Word-finding problems Comprehension difficulties No problems with new learning Lesions mainly located in the infero-lateral temporal cortex but (early in the disease) with sparing of the hippocampus

Severe loss of trace connections Stage-2 learning proceeds as normal Stage 3 learning strongly impaired Non-rehearsed memories will be lost No consolidation in semantic dementia

Semantic dementia in TraceLink Primary cause: loss of trace-trace connections Stage-3 (and 4) memories cannot be formed: no consolidation The preservation of new memories will be dependent on constant rehearsal For a review, see Murre, J. M. J., Graham, K. S., & Hodges, J. R. (2001). Semantic dementia: Relevance to connectionist models of long-term memory. Brain, 124,

Nadel and Moscovitch (1997): Trace Replication Theory They reject the ‘Standard Theory’ of consolidation Hippocampus always remains involved Hippocampal representations increase in strength with time For review and assessment, see Meeter, M., & J. M. J. Murre (2004). Consolidation of long-term memory: Evidence and alternatives. Psychological Bulletin, in press.

Sleep and Dreaming When memory may be consolidated

“We dream in order to forget” Or do we? Theory by Francis Crick and Graeme Mitchison (1983) Main problem: Overloading of memory Solution: Reverse learning leads to removal of ‘obsessions’

Dreaming and memory consolidation When should this reverse learning take place? During REM sleep –Normal input is deactivated –Semi-random activations from the brain stem –REM sleep may have lively hallucinations

Consolidation may also strengthen memory This may occur during deep sleep (as opposed to REM sleep) Both hypothetical processes may work together to achieve an increase in the clarity of representations in the cortex

Relevant data by Matt Wilson and Bruce McNaughton (1994) 120 neurons in rat hippocampus PRE: Slow-wave sleep before being in the experimental environment (cage) RUN: During experimental environment POST: Slow-wave sleep after having been in the experimental environment

Wilson en McNaughton Data PRE: Slow-wave sleep before being in the experimental environment (cage) RUN: During experimental environment POST: Slow-wave sleep after having been in the experimental environment

Experiment by Robert Stickgold Difficult visual discrimination problem Several hours of practice One group goes home Other group stays in the lab and skips a night of sleep

Improvement without further training due to sleep Normal sleep Skipped first night sleep

Connectionist implementation of the TraceLink model With Martijn Meeter

How the simulations work: One simulated ‘day’ A new memory is learned A period of ‘simulated dreaming’ follows –Artificial neurons are activated randomly –This random activity causes ‘recall’ of a memory –The recalled memory is strengthened in the neocortex

Frequency of consolidation of patterns over time

A simulation with TraceLink

Strongly and weakly encoded patterns Mixture of weak, middle and strong patterns Strong patterns had a higher learning parameter (cf. longer learning time)

A ‘Darwinian’ competition? Over time, the consolidation process squeezes out the weak patterns

Memory Improvement Not automatic but possibly effective

Memory Improvement Strengthening of existing memory Not suitable for anterograde amnesia –Memory book/-electronic agenda –Errorless learning (Baddeley and Wilson)

The two pillars of effective memory Elaboration or Making words more memorable Rehearsal or Going back to what you are about to forget

Elaboration: Making words more memorable Partition (break it up!) Link –use anything that comes into your mind Imagine –visualize (bizarre) –hear, feel, smell, etc. –verbalize

Elaboration of names Pat Galveston Pat Galve-ston Pot Gulf Stone An enormous pot with a ‘sea’ (gulf) inside and with a massive rock in it Al Kane (no need to break up) Eel Cane An eel slithering down a cane

Rehearsal: Go back to what you are about to forget The more rehearsal, the better the memory retention Rehearse at progressively longer intervals: expanding rehearsal

Expanding rehearsal Example schedule –immediately after the lecture (lesson, meeting, experience, …) –the next day –three days later –a week later –a month later –after half a year

A very useful memory trick Uses the journey technique Best used with lists of objects or names In your mind, walk along a familiar route Mentally, ‘place’ the objects at locations along the route Elaborate upon the locations

Example of the journey technique In front of my building The revolving entrance door The lobby Waiting for the elevator In the elevator Buy cat food Call Lucy Order chair Ted Bill John

Practice

Further hints on the journey technique Combine different journeys to remember long lists Always use the same locations This allows reference by number (e.g., 7-th on the list)

A trick to remember numbers One is a bun Two is a shoe Three is a knee Four is a door Five is a hive Six is sticks Seven is heaven Eight is a gate Nine is wine Ten is a hen

This is a peg technique Combine with the journey technique to remember long numbers, e.g., –bee hive in front of the building –lots of bottles of wine in the door –lobby has turned into heaven –wine is presented while waiting for the elevator –elevator is full of shoes, etc.

Where to go from here More sophisticated memory techniques –Remember numbers with the phonetic mnemonic –Mental filing For a detailed description and trainers: