1. Learning crossmodal spatial transformations through STDP Gerhard Neumann Seminar B, SS 06

2. Overview Network Model Hebbian Learning and STDP Properties of STDP Cortical Maps Learning Spatial Transformations Papers: [Song99] : S. Song, L. Abbott, Competitive Hebbian Learning through Spike-Timing-Dependent Plasticity [Song00] : S. Song, L. Abbott, Cortical Development and Remapping through Spike Timing-Dependent Plasticity [Davison06]: A. Davison and Y.Fregnac, „Learning Crossmodal spatial transformations through spike-timing-dependent plasticity.

3. Leaky Integrate + Fire Model Membrane Potential V j of neuron j: Input consists of: Background noise Excitatory Input (added): Inhibatory Input (substracted):

4. Neuron Model Excitatory Input: Incremented by following each spike Inhibatory Input: incremented by by every spike Simplified Version: Direct change in synaptic current More Complex Version Conductance Based IF Neurons used by Abbott Basically the same results

5. Hebbian Learning Donald Hebb: When an axon of cell A is near enough to excite cell B or repeatedly or consistently takes part in firing it, some growth or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased. Correlation based learning Not a stable rule: Weight normalization needed No Competition Usually we need a „global competition signal“  Not biologically realistic Only for feed forward networks: No recurrent connections possible

6. Spike-timing-dependent plasticity Synaptic plasticity is sensitive to the temporal oder of the presynaptic and postsynaptic spike Long Term Potentiation (LTP) If pre synaptic spike before post synaptic spike Correlated Input Long Term Depression (LTD) If post synaptic spike before pre synaptic spike Random Input Experiments with culture of rat Hippocampal cells

7. STDP: Time Window Time Window: ~ 20 ms Hard Bounds or Soft Bounds Model w… either models the conductance for the synaptic input or directly the change in the synaptic current Area of Depression must be larger than area of potentiation Typical Values for: : 20 ms : 20 ms – 100 ms

8. STDP: Other Models Other Models: Symmetric STDP: Short Time Intervalls: LTP Long Time Intervalls: LTD Inverse STDP Reversed LTP/LTD Also recurrent loops are possible Surpresses recurrent loops leading to a stable network Mean Input to a Neuron should only be sufficient to charge the membrane to a point below or only slightly above the treshold Postsynaptic Neuron fires primarily in response to statistical fluctuations in the input

9. Basic Experiments with STDP For a single Post Synaptic Neuron STDP tends to segragate synaptic weights into strong and weak groups (~ 50 %). Competitive Nature of STDP

10. Basic Experiments with STDP Effects of Different Correlation Times  - Dots…  - Triangles…  also works for larger Correlation times

11. Network Model: 1-D stimulus E.g. location of a touch stimulus Encoded in 1000 Input Neurons Grid over the input stimulus Firing rate: Gaussian Firing Curve  Maximum at the prefered stimulus location of the cell Population Coding Use periodic boundary conditions 200 network neurons Sparse connectivity to the input neurons (20 %, random) Learning procedure Input: Brief presentations of the stimulus at a random input location Lasts ~ 20 ms (exponential distribution)

12. Experiments: Without any recurrent connections: Each Neuron develops a random input selectivity => Nearby input neurons are correlated Strengthing of synapses in one group of corr. inputs Surpresses other input (competitive)

13. Experiments Add all-to-all recurrent excitatory connections to the output neurons Initiliaze weights with zero Selectivity and Column Structure, all neurons are selective in the same neighborhood Recurrent Connections are quite weak after convergence

14. Experiments Seeding the network Network neurons 81 – 120 were given initial input weights for stimulus locations from 401 to 600 Recurrent synapses are strengthened before ff synapses Only one strongly correlated group of network neurons, many correlated groups of input neurons Seeded network neurons begin to drive unseeded network Synapse from input neurons 401-600 to unseeded network become strong Recurrent synapses are weakened again FF synapses compete with recurrent synapses because of shorter latency. Seeded network units can be seen as sort of teacher signal

15. Experiments Seeding the network

16. Cortical Map Until now: Competitive nature of STDP leads to a winner take it all situation Single column structure forms We want: A Continuous Map Restrict the spread of selectivity from one neuron to another Limit the recurrent excitatory connections between network neurons to local neighborhoods Add an initial seed to the FF connections

17. Experiment : Refinement of Maps Seeded Cortical Map Initialize FF connections with a coarse map Map is tightened and refined.

18. Experiment : Unseeded Case Without initial seeding, a single column structure forms. Map can also arise from random initial conditions if inhibitatory connections are introduced All to all uniform connections of fixed strength between network neurons Different neurons in the network tend to develop different location preferences Local excitatory connections favor similar preferences => formation of a smoothly changing cortical map

19. Experiment : Unseeded Case A Map like structure is formed The Map can be arranged in either direction at any point in the network (random initial conditions)

20. Learning Spatial Transformations Learn the transformation of a 1 or 2 DoF simulated arm from proprioceptive input to the visual location of the hand (end effector) Network structure: Three populations, each consisting of a one (or 2) dimensional array of cells Input Population: Proprioceptive Stimulus Training Population: Visual Stimulus (Location) Output Population: Prediction Connectivity: Input -> Output: All to All, learned with STDP Training -> Output: topographical and local, fixed

21. Learning Procedure A random location is choosen, the training reference calculated Use for the input population, for the training population E.g. Again use Gaussian Firing Curve

22. Experiment: 1 DoF arm: Evolution of weights S-Shaped Band becomes visible in the matrix of synaptic weights

23. Prediction: Training Layer removed, sweep through the input space Weight pattern from the input to the output layer remains stable Good approximation found, extreme values are underestimated

24. Experiment: Other Functions Linear FunctionsNon-Linear Functions

25. Experiment: 2 DoF Also for the 2-D case, longer learning time

26. Influence of the population tuning curve shape Poor accuracy of trigonometric functions near the extremes Multiple input values giving a single output value Input values compete with one another => Make width of the gaussian population tuning curve smaller

27. Influence of spatio temporal learning Drawbacks of the learning procedure: Assumes that all point in space is visited with equal frequency Tested with normal distribution Still works, slower convergence Requires dwell times (time the input stays in a particular location) Has to be approximately the same as the STDP time constants Ignores travel time between input locations, input location changed smoothly Works if motion is fast enough Inputs must fire together to produce potentiation, but then quickly stop firing to avoid depression

28. Influence of the correlation time Increase or use symmetric STDP

29. Influence of signal latency Latency difference between the input and the training signal For negative latencies (training before input), the negative weight pattern can emerge

30. Influence of Training Pathways Precise range and strength of the connections to the training population is not critical Plasticity also for the training connections Pattern is stable for prespecified initial connections Similar to Map refinement from Song

31. Learning the training pathways Start with random training weights Recurrent (local) excitatory and inhibitory connections are needed Similar to Map Learning with random initial conditions from Song Connections develop with an arbitrary phase Tends to wander as training continues

32. Experiments Influence of plastic rules: Symmetric: More robust to temporal structure Asymmetric: Less dependent on the population timing curve Soft weight Boundaries: Modulation of the weight pattern by correlation is weaker, but still learnt Influence of network size and connectivity Behavior of the network is the same by adjusting other parameters

33. Summary STDP: Biologically Inspired Correlation Based learning Competitive rule which strengthens connections to correlated input, weakens connections to random input Population coding + STDP: Can be used to calculate cortical mappings Can also learn (simple) spatial transformations Quite complex model Learning can also be done by a simple offline-linear regression Sensitive to a lot of parameters