Vidna kognicija III Danko Nikolić

Teme Neurofiziološki kodovi prijenosa i obrade informacija u vidnom sustavu Dva kôda za percepciju svjetline Problem povezivanja dijelova vidne scene u cjelinu (tzv. binding problem) Uloga pažnje u pohranjivanju informacija u radno pamćenje Uloga radnog pamćenja za formiranje dugoročnog vidnog pamćenja Mehanizmi sinestezijskih asocijacija

New lecture Small delays in synchronization

→ 150 ms → 250 ms

20 ms per stage! 1 spike per neuron! The speed of vision Thorpe & Fabre-Thorpe (2001) ms ms ms ms ms ms

What can one spike tell us?

Kuffler (1953) Discharge patterns and functional organisation of vertebrate retina, Journal of Neurophysiology Increase in stimulus intensity Stimulus onset

“I don’t think Steve much liked our abstract”.

Van Rulen and Thorpe (2001) Rate Coding Versus Temporal Order Coding:What the Retinal Ganglion Cells Tell the Visual Cortex, Neural Computation, 13 A simulation test

Spike timing in sensory receptors

Onset latencies in somatosensation Johansson & Birznieks (2004) D1 D2 D3 D4 D5 Stimulus onset

Measuring small delays Fitting a function and taking its maximum value for the estimate. Cosine fit

Phase offsets can be measured with sub- millisecond precision Schneider and Nikolić, Journal of Neuroscience Methods (2006).

Large networks 200 μm

Relative firing time [ms]

Extraction of the firing sequence Schneider, Havenith and Nikolić, Neural Computation (2006) Relative firing time [ms] Nikolić, Journal of Comp. Neuroscience (2007) Schneider and Nikolić, Journal of Neuroscience Methods (2006)

Non-parametric detection of temporal order Nikolić, Journal of Comp. Neuroscience (2007).

Example: Stimulus dependence 1)2)

Firing sequences change dynamically

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Rall (1964)

Spike timing in single neurons: Synaptic integration Euler & Denk (2004) Stiefel & Sejnowski (2007)

Conclusion: Firing sequences Short time delays can serves as a code for carrying stimulus-related information that is as reliable as is the neuronal firing rate. Stronger synchronization increases the reliability of the code.

Good timing is everything

Binding problem

Perceptual integration and organization

Hierarchical coding by extraction of feature combinations Grandmother cell

Problems: combinatorial explosion and novel combinations Combinatorial explosion: There are many different grandmothers and each can be seen from many different perspectives. Novel combinations: Some grandmothers are seen for the first time. There is no chance to learn all the possible combination of features that make a grandmother.

Perceptual organization through synchronization of action potentials

Split bar experiment (Gray et. al, 1989)

Perceptual organization through synchronization of action potentials

Synchrony at different scales

Conflicting bar experiment Engel, A.K., Koenig, P.& Singer, W. (1991); Kreiter, A.K. & Singer, W. (1996).

Mechanisms: Tangential connections

Higher brain areas and awake states An important role of attention Infero-temporal cortex and recognition of faces - In early visual areas - In higher visual areas V4, MT. Mechanisms of synchronization Mechanisms of detection

Attention and rates in V4 Modulation of rate firing rate responses in V4 Moran & Desimone, 1985

- Fries et al., Investigated strength of synchrony - Spike-triggered averages Attention and synchrony in V4 Delay period Stimulus period

Infero-temporal cortex FaceNon-face - Hirabayashi & Miyashita - Perception of faces - Synchronization is stronger when faces are perceived.

Attention in early visual areas Roelfsema et al., 2004 Non-modulation of synchrony in V1

Mechanisms For large part unknown To a high degree theoretical answers Models, simulations Three types of mechanism are considered: – bottom-up – lateral interactions – top-down

Bottom-up Common input Input is not shared

Lateral interactions Tangential connections

Top-down Lower visual areaHigher visual area Feedback connections

The brain as a liquid- state machine

A thinking ocean is the main character in a famous science fiction novel by Stanislav Lem But how can a liquid possibly process information ?

Common mistake: Trying to understand the brain from the perspective of the organization of a digital computer Amsterdam-07 IBM Blue-Gene supercomputer

Two obvious differences in the organization of computers and brains Amsterdam-07 A computer requires a program (it implements a Turing machine). A brain does not have a program: Instead it has to rely on learning. It may be viewed as a genetically encoded network of learning-agents.

Amsterdam-07 A literal interpretation of liquid computing Fernando and Sojakka: Pattern recognition in a bucket: A real liquid brain, ECAL 2003: “This paper demonstrates that the waves produced on the surface of water can be used as a medium for a “Liquid State Machine”. We made a bucket of water, vibrated it with lego motors, filmed the waves with a webcam and put it through a perceptron on matlab and got it to solve the XOR problem and do speech recognition.” „one“ „zero“

Why did we call this style of computing liquid computing ? A cortical circuit is viewed here as a special case of a Liquid State Machine

/-5*23 Log(15*31) Sin(102)+Log(… … … … do it many, many times. readoutliquidinput

time inputliquidreadout

primary visual cortex Parallel recordings by multiple Michigan probes. Up to 48 channels. Cat visual cortex, area 17 Anesthesia

main result

persistence of information

temporal superposition XOR:

code invariance

correlations matter

precise spike timing SD - jitter [ms]

a subject to noise

conclusions input non-linear map readout t - memory: information about previously shown images is available for a prolonged period of time. - superposition: information about previously and currently shown stimuli is available simultaneously. - non-linearity: information about non-linear transformations of input properties can be extracted by linear classifiers. - rates and timing: information is coded partially in neuronal firing rates and partially in the precise timing of neuronal spiking activity. - 2 nd order correlation: the advantage of using additional non-linear classification methods was limited to the use of pair-wise correlations between neurons.

Neuronal Avalanches in Vivo

European avalanche-size table 1 - Slough Small snow slide that cannot bury a person. length <50 m volume <100 m³ 2 - Small Stops within the slope. length <100 m volume <1,000 m³ 3 - Medium Runs to the bottom of the slope. length <1,000 m volume <10,000 m³ 4 – Large Runs over flat areas, may reach the valley bottom. length >1,000 m volume >10,000 m³

Distribution of Avalanche-Sizes P (size) ~ size α log P (n) ~ α log (n) = y ~ k x α

Power law in complex systems: Earthquakes, forest fires, evolution of species Size of US cities, citations of papers Does the brain generate neuronal avalanches with power law statistics ? →How do we imagine such “neuronal” avalanches? →How do we observe them in the brain?

Neuronal Avalanche of Spikes Size = 24 spikesLifetime = 14 ms

Methods Recording: → Michigan Probes Recording sites: → 3 cats: Area 17 → 1 cat: Area 17 & Area 21

Spontaneous activity under anesthesia 7 datasets → Duration: 100 – 500 s Units → Per dataset: → Single units: → Multi units:

Definition of Avalanches Δ t (ms) 6 ms

Power Law Cat: Col05 – Probe1 Δ t: 4 ms

Power law is independent of Δ t Δt was varied between 1 and 10 ms →Number of extracted avalanches decreases with larger Δt →Number of larger avalanches increases relative to smaller ones Power law remains stable irrespective of Δt ! Exponent of power law increases with Δt

Non - Power Law Cat: Col11 – Probe 1 Δt = 4 ms Dependence on Δt → 2 cases Exponential-like distribution remains robust With small Δt → power law / with large Δt → exponential function

Exponent Cat: Col05 – Probe1 Δ t (ms) → Exponent increases with larger bin-sizes

Exponent – Δ t avg Δt avg = mean of time intervals between spikes → statistical approach for optimal Δ t → Separation and concatenation of avalanches is minimized

Exponent ( Δ t avg ) ~ -1.8

Lifetime Cat: Col05 – Probe 1 Lifetime distribution of avalanches does not follow a power law in any of the probes, irrespective of Δt.

Conclusions Neuronal avalanches defined by spikes → Power law in some size-distributions → Exponent ~ -1.8 → No Power law in other size - distributions → No Power law in lifetime distributions Interpretation → Self-organized criticality (SOC) → Critical branching processes