1. Sparse Coding and Hebbian Learning Sen Song 2017/11/9

2. Types of learning task Unsupervised learning Supervised learning
Create an internal representation of the input e.g. form clusters; extract features
How do we know if a representation is good?
Supervised learning
Learn to predict output when given input vector
Who provides the correct answer?
Reinforcement learning
Learn action to maximize payoff
Not much information in a payoff signal
Payoff is often delayed

3. Hebbian Learning

4. Hebbian learning
When an axon of cell A excites cell B and repeatedly or persistently takes part in firing it, some growth processes or metabolic change takes place in one or both cells so that A‘s efficiency ... is increased.
Donald Hebb (1949) A B A t B

5. Grandmother Cell - local coding theory
The grandmother cell is a hypothetical neuron that represents a complex but specific concept or object. It activates when a person "sees, hears, or otherwise sensibly discriminates” a specific entity, such as his or her grandmother.
The term was coined around 1969 by Jerry Lettvin. A similar concept was proposed two years earlier by Jerzy Konorski of gnostic neuron.
Konorski’s gnostic fields and neurons

6. Distributed (population) coding
Young’s formulation predicted that the collective or distributed response pattern of these three retinal receptors could be used to represent the wavelength (or colour) of any light stimulus in the visual spectrum unambiguously.
Despite lacking any insight into the structure of the retina or the brain,Young’s ingenious formulation gave rise to the concept of distributed neural coding. In this scheme, the electrical activity of large and spatially distributed populations of neurons — rather than single cells — is responsible for representing the attributes of incoming
sensory stimuli, or for generating the motor commands required for the production of a voluntary act.
Miguel A. L. Nicolelis 2003, Nature Reviews Neuroscience

7. Sparse Code

8. Sparse Coding
The sparse code is when each item is encoded by the strong activation of a relatively small set of neurons. For each item to be encoded, this is a different subset of all available neurons.

9. 神经元发放率呈长尾分布

10. Sparse Coding Algorithms
Given a potentially large set of input patterns, sparse coding algorithms (e.g. Sparse Autoencoder) attempt to automatically find a small number of representative patterns which, when combined in the right proportions, reproduce the original input patterns. The sparse coding for the input then consists of those representative patterns.
For example, the very large set of English sentences can be encoded by a small number of symbols (i.e. letters, numbers, punctuation, and spaces) combined in a particular order for a particular sentence, and so a sparse coding for English would be those symbols.

12. Hebb Rule
Hebb suggested that such synaptic modification could produce neuronal assemblies that reflect the relationships experienced during
training.
For example, consider applying this rule to neurons that fire together during training
due to an association between a stimulus and a response. These neurons
would develop strong interconnections, and subsequent activation
of some of them by the stimulus could produce the synaptic drive needed
to activate the remaining neurons and generate the associated response.

13. Hebb Rule
Linear neuron
Hebb rule
Similar to LTP (but not quite…)

14. Hebbian Learning
Mathematics

15. Hebb Rule Average Hebb rule= correlation rule
Q: correlation matrix of u

16. Hebb Rule Main problem with Hebb rule: it’s unstable… Two solutions:
Bounded weights
Normalization of either the activity of the postsynaptic cells or the weights.

17. Basic Hebbian Rule is unstable
So the length of the weight vector grows continuously.
We need to impose saturation, but still basic Hebb rule fails to induce
competition between different synapes.

18. Synaptic Competition
Synapses are modifed independently under a Hebbian rule, which can have deleterious consequences. For example, all of the synaptic weights may be driven to their maximum allowed values wmax, causing the postsynaptic neuron to lose selectivity to different patterns of input.
The development of input selectivity typically requires competition between different synapses, so that some are forced to weaken when others become strong.

19. Bounded Weights
Increasing synaptic strength in response to activity is a positive feedback process. The activity that modifies synapses is reinforced by Hebbian plasticity, which leads to more activity and further modification. Without appropriate adjustments of the synaptic plasticity rules or the imposition of constraints, Hebbian modification tends to produce uncontrolled growth of synaptic strengths.
The easiest way to control synaptic strengthening is to impose an upper limit on the value that a synaptic weight can take. Such an upper limit is supported by LTP experiments. It also makes sense to prevent weights from changing sign, because the plasticity processes we are modeling cannot change an excitatory synapse into an inhibitory synapse or vice versa.
We therefore impose the constraint, which we call a saturation constraint, synaptic saturation that all excitatory synaptic weights must lie between 0 and a maximum value, which is a constant. The simplest implementation of saturation is to set any weight that would cross a saturation bound due to
application of a plasticity rule to the limiting value.

20. Covariance Rule Introducing LTD into the rule
Hebb rule with threshold= covariance rule
C: covariance matrix of u
Note that <(v-< v >)(u-< u >)> would be unrealistic because it predicts LTP when both u and v are low
However, still unstable!

21. BCM rule Hebb rule with sliding threshold
Finally a stable and competitive rule!
must grow more rapidly than as the output activity grows large.
BCM rule implements competition because when a synaptic weight grows, it raises by v2, making more difficult for other weights to grow.

22. Another way to stablize Hebbian Rule Synaptic Normalization
Holding the total postsynaptic weights constant

23. Weight Normalization Multiplicative Normalization:
Norm of the weights converge to 1/a

24. Homeostatic Plasticity

25. Homeostatic Plasticity

26. Weight Normalization
Subtractive Normalization:

27. Hebb Rule Convergence properties: Use an eigenvector decomposition:
where em are the eigenvectors of Q

28. Hebb Rule e1 e2
l1>l2

29. Hebb Rule
Equations decouple because em are the eigenvectors of Q

30. Hebb Rule

31. Hebb Rule
The weights line up with first eigenvector and the postsynaptic activity, v, converges toward the projection of u onto the first eigenvector (unstable PCA)

32. Hebb Rule Non zero mean distribution: correlation vs covariance
The Principal Component carries the maximal variance
Naïve Hebbian Rule finds the PC of the correlation matrix.
Covariance Rule finds the PC of the covariance matrix.

33. Hebb Rule Limiting weights growth affects the final state
0.2
0.4
0.6
0.8 1
First eigenvector: [1,-1]
0.8 x a m w / 2 w w / w 1
max

34. Hebb Rule Normalization also affects the final state.
Ex: multiplicative normalization. In this case, Hebb rule extracts the first eigenvector but keeps the norm constant (stable PCA).

35. Hebb Rule Normalization also affects the final state.
Ex: subtractive normalization.

36. Hebb Rule

37. Hebb Rule The constrain does not affect the other eigenvector:
The weights converge to the second eigenvector (the weights need to be bounded to guarantee stability…)

38. Ocular Dominance Column Formation

39. Ocular Dominance
Ocular dominance refers to the tendency of input to neurons in the adult primary visual cortex of many mammalian species to favor one eye over the other. This is especially true for neurons in layer 4, which receive extensive innervation from the LGN.
Neurons dominated by one eye or the other occupy different patches of cortex, and areas with left- or right-eye ocular dominance alternate across the cortex in fairly regular bands known as ocular dominance stripes.

40. Visual Pathway – Towards Cortical Selectivities
Visual Cortex
Receptive fields are:
Binocular
Orientation Selective
Area 17
LGN
Receptive fields are:
I have chosen to use the visual cortex as a model system. It is a good system since there is a lot of experimental data about the VC plasticity and because it is easy to directly control the inputs to the visual cortex.
Now Describe visual pathway
Stress monocular LGN with no orientation selectivity + radially symmetric.
Monocular
Radially Symmetric
Retina
light electrical signals

41. Right Left Right Left Tuning curves Ocular Dom. Distr.
180
360 90
270
Response (spikes/sec)
Response difference indicative of ocularity
Here give song and Dance
Orientation Selectivity

42. Orientation Selectivity
Binocular Deprivation
Normal
Adult
Response (spikes/sec)
Response (spikes/sec)
Adult
It has been established that the maturation of orientation selectivity is experience dependent.
In cats at birth some cells show broadly tuned orientation selectivity. As the animal matures in a
Natural environment it’s cells become more orientation selective (Show images). If an animal is deprived of
A patterned environment it will not develop orientation selectivity and even loose whatever orientation selectivity
It had at eye opening.
angle
angle
Eye-opening
Eye-opening

43. Monocular Deprivation
Normal
Left
Right
Right
Response (spikes/sec)
Left
angle
angle
Cells in visual cortex show varying degrees of ocular dominance. Cells can be classified by their degree of ocular dominance.
Point to OD and explain. If an animal is monocularly deprived by lid suture it alters the OD histogram – as seen in first slide etc.
Show OD histogram. 20 30
% of cells 15
Ocular dom, group 10
Left dom
Right dom
group
group

44. Ocular dominance Stripes

45. Modelling Ocular Dominance – Single Cell
Left ul wl v
Eye input wr ur
Right
We need to generate a situation where through Hebbian learning one
synapse will grow while the other should drop to zero.
Called: Synaptic competition

46. Ocular Dominance Column
One unit with one input from right and left eyes
s: same eye
d: different eyes

47. Ocular Dominance Column
The eigenvectors are:

48. Ocular Dominance Column
Since qd is likely to be positive, qs+qd>qs-qd. As a result, the weights will converge toward the first eigenvector which mixes the right and left eye equally. No ocular dominance...

49. Ocular Dominance Column
To get ocular dominance we need subtractive normalization.

50. Ocular Dominance Column
Note that the weights will be proportional to e2 or –e2 (i.e. the right and left eye are equally likely to dominate at the end). Which one wins depends on the initial conditions.
Check that

51. Spike Timing Dependent Plasticity: Temporal Hebbian Learning
What did Hebb really say?
Synaptic
change %
Pre
tPre
Post
tPost
Pre precedes Post:
Long-term
Potentiation
Acausal
Pre follows Post:
Long-term
Depression
Pre
tPre
Post
tPost
Causal
(possibly)
Weight-change curve (Bi&Poo, 2001)

52. Different Learning Curves
(Note: X-axis is pre-post,
We will use: post - pre,
which seems more natural)

53. Single Cell Model
800 Excitatory
200 Inhibitory

54. STDP is a stable learning rule

55. Distribution of synaptic strength

56. Weight dependency

57. Effect of learning rule

58. Correlation between presynaptic and postsynaptic neuron

59. Effect of Input Correlation

60. Advance in spike time

61. STDP disfavors reciprocal connections

62. STDP is a Competitive Learning Rule
Subtractive or Multiplicative?

63. STDP is a Competitive Learning Rule
Subtractive or Multiplicative?

64. Recent Paper in Science from Mriganka Sur Lab
Pre-before-post pairing at specific synapses, via visual stimuli presented at a target location closely followed by channelrhodopsin-2 (ChR2)–driven spiking of an individual neuron, would induce Hebbian potentiation of excitatory synapses responding to the target stimulus and consequently shift the receptive field at the soma
There are various common methods for inducing bidirectional synaptic plasticity.
The most traditional one is by using extracellular stimulation at different frequencies. High frequency produces LTP whereas low frequency may produce LTD.
Another protocol is often called paring. Here the postsynaptic cell is voltage clamped to a certain postsynaptic voltage and at the same time a low frequency presynaptic stimuli is delivered. For small depolarization to ~-50 mv LTD is induced and depolarization to –10 produces LTP.
A third recently popular protocol is spike time dependent plasticity STDP – here a presynaptic stimuli is delivered either closely before of after a postsynaptic AP. Typically if the pre comes before the post LTP is induced and if post comes before pre LTD is produced.
Locally coordinated synaptic plasticity of visual cortex neurons in vivo
Science  22 Jun 2018:

65. LTP spines rapidly increased in volume immediately after pairing, and this was followed by a moderate increase over the next 2 hours. In contrast, sLTD spines showed an initial small decrease in volume that was amplified over the next 2 hours until the average volume change for sLTD and sLTP spines became approximately balanced. The density of sLTD spines was significantly correlated with, and was greater than, the density of sLTP spines in individual dendrites (Fig. 2E). sLTD spine density was significantly larger at short sLTP-sLTD distances (19), indicating that sLTD spines were preferentially located around sLTP spines (Fig. 2F)

66. Spine receptive fields were heterogeneously distributed along dendritic stretches (fig. S8). We hypothesized that sLTP spines should have their receptive field centers overlapping the visual target because of Hebbian plasticity, whereas nearby sLTD spines would have receptive field centers located away from the target because of heterosynaptic, potentially cooperative, plasticity. Spines with receptive fields overlapping the target stimulus indeed increased in volume (Fig. 3, D and E), whereas neighboring spines with receptive fields away from the target were reduced (Fig. 3, D and F).

67. Arc, the protein encoded by the immediate gene Arc, is involved in AMPAR endocytosis (29). Arc preferentially interacts with the inactive β isoform of CaMKII and acts as an inverse tag of plasticity (30) that could potentially mediate heterosynaptic depression in dendritic segments

68. Multiple Postsynaptic Neurons
There is a stable fixed point

69. Solving the Equation Three senarios:
Plastic feedforward and fixed recurrent synapses
Plastic feedforward and recurrent synapses
Fixed feedforward and plastic recurrent synapses

70. Ocular Dominance Stripes
Plastic feedforward and fixed recurrent synapses

71. STDP leads to development of selectivity

72. Formation of Selective Column

73. Synaptic Strengths at Various Stages of Development

74. Formation of Multiple Layers

75. Refinement of Maps

76. De novo Formation of Maps with all-to-all recurrent inhibitory connection

77. Remapping after lesion

78. STDP is a Competitive Learning Rule
Subtractive or Multiplicative?
Additional Hetereosynaptic LTD Mechanisms: mGluR, endocannabinoids

79. Learning at Recurrent Synapses Symmetric STDP in Hippocampus
Hippocampal neurons tend to fire in bursts.
In autoassociative network models, storage and recall are more robust with symmetric than with asymmetric STDP rules.
Thus, a specialized STDP induction rule allows reliable storage and recall of information in the hippocampal CA3 network.
Symmetric spike timing-dependent plasticity at CA3–CA3 synapses optimizes storage and
recall in autoassociative networks
Peter Jonas Nature Comm 2016

80. Reward Modulated STDP
Retroactive modulation of spike timing-dependent plasticity by dopamine
Time scale of several seconds, depend on cAMP pathway
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning
Rules
Wulfram Gerstner Front. Neural Circuits, 19 January 2016