1. Neural Networks with Short-Term Synaptic Dynamics (Leiden, May ) Misha Tsodyks, Weizmann Institute
Mathematical Models of Short-Term Synaptic plasticity
Computations in Neural Networks with Short-Term Synaptic Plasticity
Experiments: Henry Markram, Eli Nelken
Theory: Klaus Pawelzik, Alex Loebel

2. Multineuron Recording

4. Principles of Time-Dependency of Release
Synaptic
Facilitation
Synaptic
Depression
Three Laws of Release
Limited Synaptic Resources
Release Dependent Depression
Release Independent Facilitation
AP onset
Facilitation
Four Parameters of Release
Absolute strength
Probability of release
Depression time constant
Facilitation time constant
Depression

5. A Phenomenological Approach to Dynamic Synaptic Transmission
4 Key Synaptic Parameters
Absolute strength
Probability of release
Depression time constant
Facilitation time constant

7. Phenomenological model of synaptic depression (deterministic)

8. Stochastic transmission.

9. Binomial model of synaptic release
Loebel et al, submitted

10. The fitting process (deterministic model)

11. Testing the Model
experiment
experiment
model
model

12. Frequency-dependence of post-synaptic response

13. Frequency-dependence of post-synaptic response

14. Frequency-dependence of post-synaptic response

15. Frequency-dependence of post-synaptic response
Tsodyks et al, PNAS 1997

16. The 1/f Law of Release

17. Redistribution of Synaptic Efficacy

18. Frequency Dependence of Synaptic Modification

19. RSE

20. Shifting the Distribution of Release Probabilities

21. Population signal
Tsodyks et al, Neural Computation 1998

22. Population signal

23. Population signal
‘High-pass filtering’ of the input rate

24. Steady-state signal

25. Tsodyks et al PNAS 1997

26. Synchronous change in pre-synaptic rates

27. Synchronous change in pre-synaptic rates

28. Synchronous change in pre-synaptic rates
Abbott et al Science 1997

31. Modeling synaptic facilitation (deterministic)
Markram, Wang & Tsodyks PNAS 1998

32. Facilitation

33. Frequency-dependence of post-synaptic response

34. Frequency-dependence of post-synaptic response

35. Frequency-dependence of post-synaptic response

37. Population signal
Tsodyks et al 1998

38. Steady-state signal

39. Signaling with Facilitating Synapses
SYNAPSES AS CONTEXTUAL
FILTERS

40. A Small Circuit

41. Recurrent networks with synaptic depression
Tsodyks et al, J. Neurosci. 2000
Loebel & Tsodyks JCNS 2002
Loebel, Nelken & Tsodyks, Frontiers in Neurosci. 2007

43. Simulation of Network Activity

44. Simulation of Network Activity

45. Origin of Population Spikes

46. Network response to stimulation

47. Experimental evidence for population spikes
DeWeese & Zador 2006

48. (no inhibition, uniform connections, rate equations)
Simplified model
(no inhibition, uniform connections, rate equations) i J J

49. The rate model equations
There are two sets of equations representing the excitatory units firing rate, E , and their depression factor, R :

51. Tone
The tonic stimuli is represented by a constant shift of the {e}’s, that, when large enough, causes the network to spike and reach a new steady state

52. DeWeese, …, Zador 2003

54. Noise amplitude
(Nelken et al, Nature 1999)

55. Extended model
Loebel, Nelken & Tsodyks, 2007

56. Forward suppression
Rotman et al, 2001

57. Network response to complex stimuli

58. Network response to complex stimuli

59. Neural Network Models of Working Memory Misha Tsodyks Weizmann Institute of Science
Barak Blumenfeld, Son Preminger & Dov Sagi
Gianluigi Mongillo & Omri Barak

60. Delayed memory experiments – sample to match
Miyashita et al, Nature 1988

61. Persistent activity (Fuster ’73)
Miyashita et al, Nature 1988

62. Neural network models of associative memory (Hopfield ’82)
Memories are represented as attractors (stable states) of network dynamics.
Attractor = internal representation (memory) of a stimulus in the form of stable network activity pattern
Synaptic modifications => Changes in attractor landscape = long-term changes in memory
Convergence to an attractor = recall of item from long-term memory into a working form

63. Associative memory model
Hopfield Model (1982):
Neurons:
Memory patterns:
Synaptic connections:
Network dynamics:

64. Associative memory model
Hopfield Model (1982):
Neurons:
Memory patterns:
Synaptic modifications:

65. Associative memory model
Hopfield Model (1982):
Neurons:
Memory patterns:
Synaptic modifications:

66. Associative memory model
Hopfield Model (1982):
Neurons:
Memory patterns:
Synaptic modifications:
(for each ) if
Amit, Guetfriend and Sompolinsky et al 1985

67. Network dynamics: Convergence to an attractor
Overlap
Memory #

68. Network dynamics: Convergence to an attractor
Overlap
Memory #

69. Context-dependent representations: gradual change in the pattern
Intro – morph sequence memory is a dynamics system that is constantly updated based on experience to capture changes in the environment--- morph example. the representation depends not only on the context in which things are acquired but also on the internal context – what other objects or stimuli are represented in the same system together. Visual stimuli that we are exposed to can change the representation of objects that are already stored in the system. Also suggested by associative memory models

70. Long-term memory for faces
Friends
Non Friends … …
Preminger, Sagi & Tsodyks, Vision Research 2007

71. Experiment – Terminology
Basic Friend or Non-Friend task (FNF task)
Face images of faces are flashed for 200 ms
for each image the subject is asked whether the image is a friend image (learned in advance) or not.
50% of images are friends, 50% non-friends, in random order; each friend is shown the same number of times. No feedback is given ? ? ?
F/NF
F/NF
F/NF
200ms
200ms
200ms
200ms
200ms
200ms
200ms
200ms
200ms

72. Morph sequence Source (friend) Target (unfamiliar) 1 … 20 … 40 … 60 …80 …
100
Source
(friend)
Target
(unfamiliar)

73. Morph effect Subject HL -------- (blue-green spectrum) days 1-18
Number of ‘Friend’ responses
Bin number
Preminger, Sagi & Tsodyks, Vision Research 2007

74. Morphed patterns – Hopfield model
Continuous set of patterns

75. Morphed patterns – Hopfield model
Continuous set of patterns
Blumenfeld et al, Neuron 2006

76. Network dynamics: Convergence to the common attractor
Overlap

77. Network dynamics: Convergence to the common attractor
Overlap

78. Working memory in biologically-realistic networks (D. Amit ‘90)

79. Working memory in biologically-realistic networks (D. Amit ‘90)

80. Working memory in biologically-realistic networks (D. Amit ‘90)
Brunel & Wang, J. Comp. Neurosci 2001

81. Working memory in biologically-realistic networks
Brunel & Wang, J. Comp. Neurosci 2001

82. Multi-stability in recurrent networks

83. Multi-stability in recurrent networks
Firing rate (HZ)
Synaptic strength (J)

84. Persistent activity is time-dependent.
Rainer & Miller EJN 2002

85. A Phenomenological Approach to Dynamic Synaptic Transmission
4 Key Synaptic Parameters
Absolute strength
Probability of release
Depression time constant
Facilitation time constant
2 Synaptic Variables
Resources available (x)
Release probability (u)
Markram, Wang & Tsodyks PNAS 1998

86. Synaptic diversity in the pre-frontal cortex
Wang, Markram et al, Nature Neuroscience 2006

87. New idea
To hold short-term memories in synapses too, with pre-synaptic calcium level.
Use spiking activity only when the memory is needed for processing, and/or to refresh the synapses (smth like ‘rehearsal’).
Mongillo, Barak & Tsodyks 2008

88. Single homogeneous excitatory population – rate model

89. Gain function

90. Bifurcation diagram

91. Bi-stability in networks with synaptic facilitation

92. Attractors in networks with synaptic facilitation

93. Integrate and fire network simulations

94. Integrate and fire network simulations

95. Integrate and fire network simulations

96. Robustness and multi-item memory

97. Robustness and multi-item memory

98. Random overlapping populations

99. Predictions
Increased pre-synaptic calcium concentration during the delay period, disassociated from increased firing rate
Resistance to temporal cessation of spiking
Synchronous spiking activity during the delay period
Slow oscillations during the delay period (theta rhythm?).

100. Correlated spiking activity during delay period
Constantinidis et al, J. Neurosci. 2001

101. Theta oscillations and working memory
Lee et al, Neuron 2005

102. Son Preminger
(WIS, Rehovot)
Dov Sagi
(WIS, Rehovot)
Barak Blumenfeld
(WIS, Rehovot)
Omri Barak
(WIS, Rehovot)
Gianluigi Mongillo
(ENS, Paris)