2. Vidna kognicija III Danko Nikolić

3. Teme Neurofiziološki kodovi prijenosa i obrade informacija u vidnom sustavu Dva kôda za percepciju svjetline Problem povezivanja dijelova vidne scene u cjelinu (tzv. binding problem) Uloga pažnje u pohranjivanju informacija u radno pamćenje Uloga radnog pamćenja za formiranje dugoročnog vidnog pamćenja Mehanizmi sinestezijskih asocijacija

4. New lecture Small delays in synchronization

5. → 150 ms → 250 ms

6. 20 ms per stage! 1 spike per neuron! The speed of vision Thorpe & Fabre-Thorpe (2001) 20-40 ms 30-50 ms 40-50 ms 50-70 ms 70-90 ms 80-100 ms

7. What can one spike tell us?

11. Kuffler (1953) Discharge patterns and functional organisation of vertebrate retina, Journal of Neurophysiology Increase in stimulus intensity Stimulus onset

12. “I don’t think Steve much liked our abstract”.

13. Van Rulen and Thorpe (2001) Rate Coding Versus Temporal Order Coding:What the Retinal Ganglion Cells Tell the Visual Cortex, Neural Computation, 13 A simulation test

14. Spike timing in sensory receptors

15. Onset latencies in somatosensation Johansson & Birznieks (2004) D1 D2 D3 D4 D5 Stimulus onset

16. Measuring small delays Fitting a function and taking its maximum value for the estimate. Cosine fit

17. Phase offsets can be measured with sub- millisecond precision Schneider and Nikolić, Journal of Neuroscience Methods (2006).

19. Large networks 200 μm

20. Relative firing time [ms]

21. Extraction of the firing sequence Schneider, Havenith and Nikolić, Neural Computation (2006) Relative firing time [ms] Nikolić, Journal of Comp. Neuroscience (2007) Schneider and Nikolić, Journal of Neuroscience Methods (2006)

22. Non-parametric detection of temporal order Nikolić, Journal of Comp. Neuroscience (2007).

23. Example: Stimulus dependence 1)2)

26. Firing sequences change dynamically

30. Spike timing in single neurons: Synaptic integration Rall (1964)

31. Spike timing in single neurons: Synaptic integration Rall (1964)

32. Spike timing in single neurons: Synaptic integration Rall (1964)

33. Spike timing in single neurons: Synaptic integration Rall (1964)

34. Spike timing in single neurons: Synaptic integration Rall (1964)

35. Spike timing in single neurons: Synaptic integration Rall (1964)

36. Spike timing in single neurons: Synaptic integration Rall (1964)

37. Spike timing in single neurons: Synaptic integration Euler & Denk (2004) Stiefel & Sejnowski (2007)

39. Conclusion: Firing sequences Short time delays can serves as a code for carrying stimulus-related information that is as reliable as is the neuronal firing rate. Stronger synchronization increases the reliability of the code.

40. Good timing is everything

41. Binding problem

42. Perceptual integration and organization

45. Hierarchical coding by extraction of feature combinations Grandmother cell

46. Problems: combinatorial explosion and novel combinations Combinatorial explosion: There are many different grandmothers and each can be seen from many different perspectives. Novel combinations: Some grandmothers are seen for the first time. There is no chance to learn all the possible combination of features that make a grandmother.

47. Perceptual organization through synchronization of action potentials

48. Split bar experiment (Gray et. al, 1989)

54. Perceptual organization through synchronization of action potentials

55. Synchrony at different scales

56. Conflicting bar experiment Engel, A.K., Koenig, P.& Singer, W. (1991); Kreiter, A.K. & Singer, W. (1996).

57. Mechanisms: Tangential connections

58. Higher brain areas and awake states An important role of attention Infero-temporal cortex and recognition of faces - In early visual areas - In higher visual areas V4, MT. Mechanisms of synchronization Mechanisms of detection

59. Attention and rates in V4 Modulation of rate firing rate responses in V4 Moran & Desimone, 1985

60. - Fries et al., 2001 - Investigated strength of synchrony - Spike-triggered averages Attention and synchrony in V4 Delay period Stimulus period

61. Infero-temporal cortex FaceNon-face - Hirabayashi & Miyashita - Perception of faces - Synchronization is stronger when faces are perceived.

62. Attention in early visual areas Roelfsema et al., 2004 Non-modulation of synchrony in V1

63. Mechanisms For large part unknown To a high degree theoretical answers Models, simulations Three types of mechanism are considered: – bottom-up – lateral interactions – top-down

64. Bottom-up Common input Input is not shared

65. Lateral interactions Tangential connections

66. Top-down Lower visual areaHigher visual area Feedback connections

68. The brain as a liquid- state machine

69. A thinking ocean is the main character in a famous science fiction novel by Stanislav Lem But how can a liquid possibly process information ?

70. Common mistake: Trying to understand the brain from the perspective of the organization of a digital computer Amsterdam-07 IBM Blue-Gene supercomputer

71. Two obvious differences in the organization of computers and brains Amsterdam-07 A computer requires a program (it implements a Turing machine). A brain does not have a program: Instead it has to rely on learning. It may be viewed as a genetically encoded network of learning-agents.

72. Amsterdam-07 A literal interpretation of liquid computing Fernando and Sojakka: Pattern recognition in a bucket: A real liquid brain, ECAL 2003: “This paper demonstrates that the waves produced on the surface of water can be used as a medium for a “Liquid State Machine”. We made a bucket of water, vibrated it with lego motors, filmed the waves with a webcam and put it through a perceptron on matlab and got it to solve the XOR problem and do speech recognition.” „one“ „zero“

73. Why did we call this style of computing liquid computing ? A cortical circuit is viewed here as a special case of a Liquid State Machine

74. 1 15 -5 23 31 102 0 1+15 1/-5*23 Log(15*31) Sin(102)+Log(… … … … do it many, many times. readoutliquidinput

75. time inputliquidreadout

76. primary visual cortex Parallel recordings by multiple Michigan probes. Up to 48 channels. Cat visual cortex, area 17 Anesthesia

78. main result

79. persistence of information

80. temporal superposition XOR:

81. code invariance

82. correlations matter

83. precise spike timing SD - jitter [ms]

84. a subject to noise

85. conclusions input non-linear map readout t - memory: information about previously shown images is available for a prolonged period of time. - superposition: information about previously and currently shown stimuli is available simultaneously. - non-linearity: information about non-linear transformations of input properties can be extracted by linear classifiers. - rates and timing: information is coded partially in neuronal firing rates and partially in the precise timing of neuronal spiking activity. - 2 nd order correlation: the advantage of using additional non-linear classification methods was limited to the use of pair-wise correlations between neurons.

86. Neuronal Avalanches in Vivo

87. European avalanche-size table 1 - Slough Small snow slide that cannot bury a person. length <50 m volume <100 m³ 2 - Small Stops within the slope. length <100 m volume <1,000 m³ 3 - Medium Runs to the bottom of the slope. length <1,000 m volume <10,000 m³ 4 – Large Runs over flat areas, may reach the valley bottom. length >1,000 m volume >10,000 m³

88. Distribution of Avalanche-Sizes P (size) ~ size α log P (n) ~ α log (n) = y ~ k x α

89. Power law in complex systems: Earthquakes, forest fires, evolution of species Size of US cities, citations of papers Does the brain generate neuronal avalanches with power law statistics ? →How do we imagine such “neuronal” avalanches? →How do we observe them in the brain?

90. Neuronal Avalanche of Spikes Size = 24 spikesLifetime = 14 ms

91. Methods Recording: → Michigan Probes Recording sites: → 3 cats: Area 17 → 1 cat: Area 17 & Area 21

92. Spontaneous activity under anesthesia 7 datasets → Duration: 100 – 500 s Units → Per dataset: 105 - 158 → Single units: 68 - 118 → Multi units: 26 - 50

93. Definition of Avalanches Δ t (ms) 6 ms

94. Power Law Cat: Col05 – Probe1 Δ t: 4 ms

95. Power law is independent of Δ t Δt was varied between 1 and 10 ms →Number of extracted avalanches decreases with larger Δt →Number of larger avalanches increases relative to smaller ones Power law remains stable irrespective of Δt ! Exponent of power law increases with Δt

96. Non - Power Law Cat: Col11 – Probe 1 Δt = 4 ms Dependence on Δt → 2 cases Exponential-like distribution remains robust With small Δt → power law / with large Δt → exponential function

97. Exponent Cat: Col05 – Probe1 Δ t (ms) → Exponent increases with larger bin-sizes

98. Exponent – Δ t avg Δt avg = mean of time intervals between spikes → statistical approach for optimal Δ t → Separation and concatenation of avalanches is minimized

99. Exponent ( Δ t avg ) ~ -1.8

100. Lifetime Cat: Col05 – Probe 1 Lifetime distribution of avalanches does not follow a power law in any of the probes, irrespective of Δt.

101. Conclusions Neuronal avalanches defined by spikes → Power law in some size-distributions → Exponent ~ -1.8 → No Power law in other size - distributions → No Power law in lifetime distributions Interpretation → Self-organized criticality (SOC) → Critical branching processes