1. Continuity between perception and imagination in the predictive coding framework
Nicolas Alzetta
CoNGA: Cognition and Neuroscience Group of Antwerp
Dec 14th 2016

2. Origins of predictive coding in cognitive science
Hermann Helmholtz:
How can we escape from the world of the sensations of our nervous systems into the world of real things ?
We are guided by the answers nature delivers when we query it, using unconscious perceptual inference based on our prior learning.
Connectionism & machine learning: models of unsupervised learning that relied heavily on recurrence and prediction
Bayesianism: the brain as inference machine that infers causes from sensory input
Data compression: eg. JPEGs: the computational cost of coding color of every pixel is too high.

3. The Bayesian brain
Nervous system maintains probabilistic models which are updated by sensory information according to (an approximation of) Bayes’ theorem

4. Main ideas of PC
The brain continuously makes generative models (hypotheses) at multiple levels of the cortical hierarchy. These statistical models predict the activity from lower areas and are sent down the hierarchy through recurrent connections (top down).
Each model is compared to the activation at the level below. If there is a mismatch, the so-called prediction error is sent up through feedforward connections (bottom-up).
From: Stefanics, et al. 2015

5. Main ideas of PC (2)
At each level of the hierarchy, the generative model is continuously updated on the basis of the prediction error. The aim of the process is to minimize prediction error, in which case the model has optimal accuracy. This is also described as ‘minimizing surprise’.
An other way of minimizing the prediction error is not by updating the model at the higher level, but by action in order to change the input signal.
The conscious percept is the prediction (hypothesis) with the highest overall posterior probability and hence the lowest the prediction error.
Depending on context, input might be less precise (low signal to noise ratio). In that case, the impact of the prediction error on the generative model is smaller. Attention on this framework is the optimization precision.

6. Main ideas of PC (3)
Depending on context (low signal to noise ratio, high uncertainty), input might be less precise. In that case, the impact of the prediction error on the updating of the generative model is smaller.
Attention, on this picture, serves to optimize the precision of the prediction error signal. It ‘turns up’ or ‘lowers the volume’ of signal.

7. Explanatory strengths
Offers a solution to (among other things):
Binding problem
Binocular rivalry
Contextual effects

8. Weaknesses Relatively little neurological evidence
But: Friston (2009) argues that the sources of forward connections are the superficial pyramidal cell population and the sources of backward connections are the deep pyramidal cells
Too general and therefore unfalsifiable.
Not all attentional effects can be explained using precision optimization.

9. Mental imagery
According to PC, mental imagery consists simply of generative models without error messages coming from the lowest levels of the hierarchy to update them. There is no fundamental divide between perceiving and imagining.
Empirical support:
Reddy et al. study using ‘brain reading’: reconstruct properties of a stimulus from fMRI data, by extracting multivoxel response patterns from which you infer (decode) properties of the stimulus that caused them.
First subjects viewed objects. A decoding algorithm was developed that could successfully classify properties of the objects (food, face, tool, …).
The same was done when subjects imagined the objects. A different decoding algorithm was successfully developed.

10. Mental imagery
Then, both algorithms were swapped, without significantly impacting the classification results. This indicates a deep similarity between neuronal activation patterns in both perception and imagery.
This could only be done in ventral temporal cortex. Not in V1 & V2 where decoding was only possible in perception cases.
The claim I wish to defend (…), is thus that animals able to perceive a complex external world of interacting causes using the characteristic resources of prediction-driven learning will be animals capable of the endogenous generation of sensory like states. (Clark 2016)
A good imaginer is a good perceiver ? Is that why artists see more ?

11. Questions?
Sources:
Friston, K., and Kiebel, S. (2009). Predictive coding under the free-energy principle. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 1211–1221.
Stefanics, G., Kremláček, J., and Czigler, I. (2014). Visual mismatch negativity: a predictive coding view. Front. Hum. Neurosci. 8:666, 1–19. doi: /fnhum