1. Lecture 3: Learning and Memory Prof.dr. Jaap Murre University of Maastricht University of Amsterdam jaap@murre.com http://neuromod.uva.nl

2. Overview We will study Hebbian learning and the formation of categories We will do some basic memory experiments Examine various forms of memory We will try to locate memory in the brain and relate brain lesions to amnesia We will also briefly explore executive functions in the frontal lobes We will look at memory improvement

3. With Hebbian learning, two learning methods are possible With unsupervised learning there is no teacher: the network tries to discern regularities in the input patterns With supervised learning an input is associated with an output –If the input and output are the same, we speak of auto-associative learning –If they are different it is called hetero- associative learning

4. Supervised learning with Hopfield (1982) network Bipolar activations –-1 or 1 Symmetric weights (no self weights) –w ij = w ji Asynchronous update rule –Select one neuron randomly and update it Simple threshold rule for updating

5. Energy of a Hopfield network Energy E = - ½  i,j w ji a i a j  E = - ½  i (w ji a i + w ij a i )  a j = -  i w ji a i  a j Net input to node j is  i w ji a i = net j Thus, we can write  E = - net j  a j

6. The energy minimization question can also be turned around Given a i and a j, how should we set the weight w ji = w ji so that the energy is minimized?  E = - ½   w ji a i a j, so that –when a i a j = 1,  w ji must be positive –when a i a j = -1,  w ji must be negative For example,  w ji =  a i a j, where  is a learning constant

7. Hebb and Hopfield When used with Hopfield type activation rules, the Hebb learning rule places patterns at attractors If a network has n nodes, 0.15n random patterns can be reliably stored by such a system For complete retrieval it is typically necessary to present the network with over 90% of the original pattern

8. We will look at an example of competitive learning Competitive learning is a form of unsupervised learning

9. Example of competitive learning: Stimulus ‘at’ is presented ato 12

10. Example of competitive learning: Competition starts at category level ato 12

11. Example of competitive learning: Competition resolves ato 12

12. Example of competitive learning: Hebbian learning takes place ato 12 Category node 2 now represents ‘at’

13. Presenting ‘to’ leads to activation of category node 1 ato 12

14. ato 12

15. ato 12

16. ato 12

17. Category 1 is established through Hebbian learning as well ato 12 Category node 1 now represents ‘to’

18. Before we continue... Everybody on the right of the classroom, please, close their eyes until the following words have been presented The others, pay attention to the following 10 words. You will be asked to remember them later Don’t write them down!

19. table

20. car

21. tree

22. computer

23. monkey

24. paper

25. scissors

26. tennis

27. dessert

28. bread

30. Now for the other half... Everybody on the left of the classroom, please, close their eyes until the following words have been presented The others, pay attention to the following 10 words. You will be asked to remember them later Don’t write them down!

31. table

32. car

33. tree

34. computer

35. monkey

36. paper

37. assassin

38. tennis

39. dessert

40. bread

42. Memory and attention are strongly intertwined Paying attention can be seen as holding in memory Attention is required for rehearsal The longer an item is attended (held in memory), the higher the chance it will be remembered later

43. Desimone’s study of V4 * neurons * V4 is visual cortex before inferotemporal cortex (IT)

44. Brown-Peterson task Try to remember three letters, e.g., XJC When given a number (e.g., 307), start counting backward in threes (307, 304, 301, 298, …) When the Write! text appears, write down the letters you remember This has to be done at least several times to obtain the effect

45. Ready!

46. RGP

47. 875

48. Write!

49. Ready!

50. ZQN

51. 317

52. Write!

53. Ready!

54. HWB

55. 504

56. Write!

57. Typical results of the Brown- Peterson task The results typically show very low memory performance The reason is that rehearsal of the letters is prevented by the counting task

58. Before we continue, Write down all the words you remember from the presentation Make sure you do not verbalize them at this moment We will verify the result in a minute, but first we have the following two puzzles

59. Fragment completion Try to complete the following English word fragments You have 30 seconds Each dot (.) stands for a letter Don’t verbalize! (So, we can obtain a better sample)

60. s..ss..s

61. .ss.ss..

62. Stop!

63. The correct answers were scissors assassin

64. The presented words were: Left half –table –car –tree –computer –monkey –paper –scissors –tennis –dessert –bread Right half –table –car –tree –computer –monkey –paper –assassin –tennis –dessert –bread

65. Short-term memory and working memory

66. Typical memory recall curve

67. Atkinson and Shiffrin model (1968)

68. This model had some limitations Ba, ta, pa, ta, pa, ba is much more difficult to remember than ba, bu, bi, bu, bi, ba Hence, there are phonological effects in short-term memory

69. Working memory model by Alan Baddeley and Graham Hitch (1974)

70. Executive functions What controls the memory retrieval process? How does the control process work? What determines which areas of brain are ‘allowed’ to active in the first place?

71. The elusive frontal cortex

72. Brodmann’s map

73. Anatomy of prefrontal cortex Strong lateral connectivity via stellates in Layer IV No direct connections to motor outputs Certain cells fire strongly and selectively during the delay period of a task in relation to certain aspects of the taks (e.g., position), especially in area 46 surrounding the principal sulcus

74. Goldman-Rakic studies of Piaget’s A  B Paradigm Infants persist in reaching for a target even if they have observed it being hidden in another place and older infants will do this if the delay is large enough (2-5 s at 7.5-9 months) Still older infants will not do this Monkeys with dorsolateral prefrontal lesions show similar behavior (delays > 2 s)

75. Similarities with ‘prefrontal’ patients Prefrontal patients show perseveration on the Wisconsin card sorting test There is evidence that also in adult humans such behavior is mainly caused by lesions to the dorsolateral prefrontal cortex

76. There is also a working memory aspect to the task The subject must keep in mind where the hiding place was, which may involve a kind of ‘working memory’ lasting several seconds In other experiments Patricia Goldman-Rakic has implicated area 46 as performing a type of working memory function Alan Baddeley is not convinced that this type of working memory is similar to his own concept

77. Long-term memory and amnesia

78. Larry Squire’s taxonomy of long- term memory

79. Forgetting There is currently no theory that explains why we forget Forgetting seems to follow rather strict rules, but even these have not been fully explored It is postulated that very well rehearsed knowledge will never be forgotten (Harry Barrick’s ‘permastore’)

80. 10,000 to 100,000 connections per neuron

81. Anatomy of a neuron

82. Memory is stored in the connections (synapses) between neurons

83. Main storage sites of memories in brain Neocortex Hippocampus

84. Personen, dieren, en voorwerpen in de temporaalschors

85. Ook met breinscans worden dergelijke locaties aangetroffen

86. Position of the hippocampus

87. The neocortex Whale (5 x human) Human

88. 1000 PC hard disks The neocortex contains about 10 billion Every neuron connects to 10,000 others That amounts to 100,000 billion connections that each can store about 1 byte The neocortex thus has the equivalent capacity of at least a 1000 hard disks of 100 giga bytes

89. TraceLink model A connectionist model of memory consolidation and amnesia

90. TraceLink model: structure

91. System 1: Trace system Function: Substrate for bulk storage of memories, ‘association machine’ Corresponds roughly to neocortex

92. System 2: Link system Function: Initial ‘scaffold’ for episodes Corresponds roughly to hippocampus and certain temporal and perhaps frontal areas

93. System 3: Modulatory system Function: Control of plasticity Involves at least parts of the hippocampus, amygdala, fornix, and certain nuclei in the basal forebrain and in the brain stem

94. Stages in episodic learning

95. Retrograde amnesia Primary cause: loss of links Ribot gradients Shrinkage

96. Anterograde amnesia Primary cause: loss of modulatory system Secondary cause: loss of links Preserved implicit memory

97. Semantic dementia The term was adopted recently to describe a new form of dementia, notably by Julie Snowden et al. (1989, 1994) and by John Hodges et al. (1992, 1994) Semantic dementia is almost a mirror- image of amnesia

98. Neuropsychology of semantic dementia Progressive loss of semantic knowledge Word-finding problems Comprehension difficulties No problems with new learning Lesions mainly located in the infero-lateral temporal cortex but (early in the disease) with sparing of the hippocampus

99. Severe loss of trace connections Stage-2 learning proceeds as normal Stage 3 learning strongly impaired Non-rehearsed memories will be lost No consolidation in semantic dementia

100. Semantic dementia in TraceLink Primary cause: loss of trace-trace connections Stage-3 (and 4) memories cannot be formed: no consolidation The preservation of new memories will be dependent on constant rehearsal For a review, see Murre, J. M. J., Graham, K. S., & Hodges, J. R. (2001). Semantic dementia: Relevance to connectionist models of long-term memory. Brain, 124, 647-675.

101. Nadel and Moscovitch (1997): Trace Replication Theory They reject the ‘Standard Theory’ of consolidation Hippocampus always remains involved Hippocampal representations increase in strength with time For review and assessment, see Meeter, M., & J. M. J. Murre (2004). Consolidation of long-term memory: Evidence and alternatives. Psychological Bulletin, in press.

102. Sleep and Dreaming When memory may be consolidated

103. “We dream in order to forget” Or do we? Theory by Francis Crick and Graeme Mitchison (1983) Main problem: Overloading of memory Solution: Reverse learning leads to removal of ‘obsessions’

104. Dreaming and memory consolidation When should this reverse learning take place? During REM sleep –Normal input is deactivated –Semi-random activations from the brain stem –REM sleep may have lively hallucinations

105. Consolidation may also strengthen memory This may occur during deep sleep (as opposed to REM sleep) Both hypothetical processes may work together to achieve an increase in the clarity of representations in the cortex

106. Relevant data by Matt Wilson and Bruce McNaughton (1994) 120 neurons in rat hippocampus PRE: Slow-wave sleep before being in the experimental environment (cage) RUN: During experimental environment POST: Slow-wave sleep after having been in the experimental environment

107. Wilson en McNaughton Data PRE: Slow-wave sleep before being in the experimental environment (cage) RUN: During experimental environment POST: Slow-wave sleep after having been in the experimental environment

108. Experiment by Robert Stickgold Difficult visual discrimination problem Several hours of practice One group goes home Other group stays in the lab and skips a night of sleep

109. Improvement without further training due to sleep Normal sleep Skipped first night sleep

110. Connectionist implementation of the TraceLink model With Martijn Meeter

111. How the simulations work: One simulated ‘day’ A new memory is learned A period of ‘simulated dreaming’ follows –Artificial neurons are activated randomly –This random activity causes ‘recall’ of a memory –The recalled memory is strengthened in the neocortex

112. Frequency of consolidation of patterns over time

113. A simulation with TraceLink

114. Strongly and weakly encoded patterns Mixture of weak, middle and strong patterns Strong patterns had a higher learning parameter (cf. longer learning time)

115. A ‘Darwinian’ competition? Over time, the consolidation process squeezes out the weak patterns

116. Memory Improvement Not automatic but possibly effective

117. Memory Improvement Strengthening of existing memory Not suitable for anterograde amnesia –Memory book/-electronic agenda –Errorless learning (Baddeley and Wilson)

118. The two pillars of effective memory Elaboration or Making words more memorable Rehearsal or Going back to what you are about to forget

119. Elaboration: Making words more memorable Partition (break it up!) Link –use anything that comes into your mind Imagine –visualize (bizarre) –hear, feel, smell, etc. –verbalize

120. Elaboration of names Pat Galveston Pat Galve-ston Pot Gulf Stone An enormous pot with a ‘sea’ (gulf) inside and with a massive rock in it Al Kane (no need to break up) Eel Cane An eel slithering down a cane

121. Rehearsal: Go back to what you are about to forget The more rehearsal, the better the memory retention Rehearse at progressively longer intervals: expanding rehearsal

122. Expanding rehearsal Example schedule –immediately after the lecture (lesson, meeting, experience, …) –the next day –three days later –a week later –a month later –after half a year

123. A very useful memory trick Uses the journey technique Best used with lists of objects or names In your mind, walk along a familiar route Mentally, ‘place’ the objects at locations along the route Elaborate upon the locations

124. Example of the journey technique In front of my building The revolving entrance door The lobby Waiting for the elevator In the elevator Buy cat food Call Lucy Order chair E-mail Ted Bill John

125. Practice

126. Further hints on the journey technique Combine different journeys to remember long lists Always use the same locations This allows reference by number (e.g., 7-th on the list)

127. A trick to remember numbers One is a bun Two is a shoe Three is a knee Four is a door Five is a hive Six is sticks Seven is heaven Eight is a gate Nine is wine Ten is a hen

128. This is a peg technique Combine with the journey technique to remember long numbers, e.g., 597928641 –bee hive in front of the building –lots of bottles of wine in the door –lobby has turned into heaven –wine is presented while waiting for the elevator –elevator is full of shoes, etc.

129. Where to go from here More sophisticated memory techniques –Remember numbers with the phonetic mnemonic –Mental filing For a detailed description and trainers: www.murre.com