1. Wellcome Trust Centre for Neuroimaging University College London
Fifth Conference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications April 4-7, 2009 | Budapest, Hungary
A-0115
The MMN and perception
K Friston
Wellcome Trust Centre for Neuroimaging University College London
This talk will try to frame the MMN in terms of perceptual learning and inference. We start with a statistical formulation of Helmholtz’s ideas about neural energy to furnish a model of perception that can explain a remarkable range of neurobiological facts. Using constructs from statistical physics it can be shown that the problems of inferring what caused our sensory inputs and learning causal regularities in the sensorium can be resolved using exactly the same principles. Furthermore, inference and learning can proceed in a biologically plausible fashion. The ensuing scheme rests on empirical Bayes and hierarchical models of how sensory information is generated. The use of hierarchical models enables the brain to construct prior expectations in a dynamic and context-sensitive fashion. This scheme provides a principled way to understand many aspects of the brain’s organization and responses. Using this framework we can simulate evoked responses to repeated stimuli and examine the within and between-trial changes (reflecting inference and learning respectively). These simulations suggest that learning in lower levels of the sensory hierarchy may explain early effects (c.f., N1 enhancement), whereas changes in higher levels may underlie later MMN components proper.

2. Overview Inference and learning under the free energy principle
Hierarchical Bayesian inference
Bird songs (inference)
Perceptual categorisation
Prediction and omission
Bird songs (learning)
Repetition suppression
The mismatch negativity

3. The free-energy principle
Sensation
External states
Internal states
environment
Action
agent - m
Action to minimise a bound on surprise
Perception to optimise the bound

4. Neuronal implementation
Perception and inference
Learning and memory
Activity-dependent plasticity
Synaptic activity
Synaptic efficacy
Functional specialization
Attentional gain
Enabling of plasticity
Synaptic adaptation
Attention and salience

5. Forward prediction error
Neuronal hierarchies
Forward prediction error SG L4 IG
Backward predictions
Synaptic efficacy
Synaptic adaptation
Synaptic activity

6. Overview Inference and learning under the free energy principle
Hierarchical Bayesian inference
Bird songs (inference)
Perceptual categorisation
Prediction and omission
Bird songs (learning)
Repetition suppression
The mismatch negativity

7. Synthetic song-birds Syrinx Higher vocal centre Sonogram
0.2
0.4
0.6
0.8
2000
2500
3000
3500
4000
4500
5000
time (seconds)

8. Forward prediction error
Hierarchical recognition 10 20 30 40 50 60 -5 5 15
prediction and error
time
Backward predictions 10 20 30 40 50 60
-10 -5 5 15
causes
time (bins)
percept
0.2
0.4
0.6
0.8
2000
2500
3000
3500
4000
4500
5000
time (seconds)
Forward prediction error 10 20 30 40 50 60 -5 5 15
hidden states
time

9. Categorizing sequences 90% confidence regions
Frequency (Hz)
Song A
0.2
0.4
0.6
0.8
2000
3000
4000
5000
time (seconds)
Song B
Song C
0.2
0.4
0.6
0.8 1
-20
-10 10 20 30 40 50
Categorizing sequences
90% confidence regions
Song A
Song B
Song C
time (seconds) 10 15 20 25 30 35 1
1.5 2
2.5 3
3.5
Song A
Song B
Song C

10. Overview Inference and learning under the free energy principle
Hierarchical Bayesian inference
Bird songs (inference)
Perceptual categorisation
Prediction and omission
Bird songs (learning)
Repetition suppression
The mismatch negativity

11. Sequences of sequences
Neuronal hierarchy
Syrinx
sonogram
Frequency (KHz)
0.5 1
1.5
Time (sec)

12. Hierarchical perception20 40 60 80
100
120
-20
-10 10 30 50
prediction and error
Hierarchical perception
5000
percept
4500
4000
Prediction error encoded by superficial pyramidal cells that generate ERPs
3500
3000
2500
2000
0.5 1
1.5
time (seconds)
time
5000
stimulus
4500
4000
3500
3000
2500 20 40 60 80
100
120
-10 10 30 50
causes
time 20 40 60 80
100
120
-20
-10 10 30 50
hidden states
time
hidden states
2000
0.5 1
1.5 50
time (seconds) 40 30 20 10
-10
-20 20 40 60 80
100
120
time

13. omission and violation of predictions
Stimulus but no percept
Percept but no stimulus

14. Overview Inference and learning under the free energy principle
Hierarchical Bayesian inference
Bird songs (inference)
Perceptual categorisation
Prediction and omission
Bird songs (learning)
Repetition suppression
The mismatch negativity

15. Repetition suppression and the MMN
The MMN is an enhanced negativity seen in response to any change (deviant) compared to the standard response.
Main effect of faces
Suppression of inferotemporal responses to repeated faces
Henson et al 2000

16. Hierarchical learning Synaptic adaptation Synaptic efficacy10 20 30 40 50 60
-0.2
-0.1
0.1
0.2
0.3
0.4
0.5
causes
time
Time (sec)
Frequency (Hz)
0.1
0.2
0.3
0.4
2000
2500
3000
3500
4000
4500
5000 10 20 30 40 50 60 -5 5 15 25 35
prediction and error
time 10 20 30 40 50 60 -2 -1 1 2
hidden states
time

17. Simulating ERPs to repeated chirps
hidden states
percept
prediction error 5
5000 10
Simulating ERPs to repeated chirps
4000
Frequency (Hz)
LFP (micro-volts)
3000
-10 -5
2000
0.2
0.4
0.1
0.2
0.3
100
200
300 5 10
4000
Frequency (Hz)
LFP (micro-volts)
-10 -5
2000
0.2
0.4
0.1
0.2
0.3
100
200
300 5 10
4000
Frequency (Hz)
LFP (micro-volts)
Perceptual inference: suppressing error over peristimulus time
Perceptual learning: suppression over repetitions
-10 -5
2000
0.2
0.4
0.1
0.2
0.3
100
200
300 5
4000 10
Frequency (Hz)
LFP (micro-volts)
-10 -5
2000
0.2
0.4
0.1
0.2
0.3
100
200
300 5 10
4000
Frequency (Hz)
LFP (micro-volts)
-10 -5
2000
0.2
0.4
0.1
0.2
0.3
100
200
300 5
4000 10
Frequency (Hz)
LFP (micro-volts)
-10 -5
2000
0.2
0.4
0.1
0.2
0.3
100
200
300
Time (sec)
Time (sec)
peristimulus time (ms)

18. Synthetic MMN Synaptic efficacy Synaptic gain First presentation3 6
Synthetic MMN
2.5 5 2 4
changes in parameters
1.5
hyperparameters 3 1 2
0.5 1 1 2 3 4 5 1 2 3 4 5
presentation
presentation 10 10 10 10 10
primary level prediction error
First presentation
(before learning)
-10
-10
-10
-10
-10
Last presentation
(after learning)
100
200
300
100
200
300
100
200
300
100
200
300
100
200
300
0.2
0.2
0.2
0.2
0.2
secondary level
-0.2
-0.2
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.4
-0.4
100
200
300
100
200
300
100
200
300
100
200
300
100
200
300
primary level (N1/P1)
secondary level (MMN) 20
0.4 15
0.2 10 5
Difference waveform
Difference waveform
-0.2 -5
-0.4
-10
-15
-0.6
100
200
300
400
100
200
300
400
peristimulus time (ms)
peristimulus time (ms)

19. Intrinsic connections
Synaptic efficacy
Synaptic gain
Extrinsic connections
Intrinsic connections 3 6
200
200
180
180
2.5 5
160
160 2 4
140
140
120
120
changes in parameters
1.5
hyperparameters 3
100
100 80 80 1 2 60 60
0.5 1 40 40 20 20 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
presentation
presentation
presentation
presentation A1
STG
subcortical input
repetition effects
Synthetic and real ERPs

20. Summary
A free energy principle can account for several aspects of action and perception
The architecture of cortical systems speak to hierarchical generative models
Estimation of hierarchical dynamic models corresponds to a generalised deconvolution of inputs to disclose their causes
This deconvolution can be implemented in a neuronally plausible fashion by constructing a dynamic system that self-organises when exposed to inputs to suppress its free-energy