Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Generation Recognition Introduction time

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Curve fitting without Bayes DataOrdinary least squares

Curve fitting without Bayes DataOrdinary least squares

Curve fitting without Bayes Ordinary least squares Bases (explanatory variables)Sum of squared errors Data and model fit Bases (explanatory variables)Sum of squared errors

Curve fitting without Bayes Data and model fitOrdinary least squares Bases (explanatory variables)Sum of squared errors

Curve fitting without Bayes Data and model fitOrdinary least squares Bases (explanatory variables)Sum of squared errors

Curve fitting without Bayes Data and model fit Bases (explanatory variables)Sum of squared errors Over-fitting: model fits noise Inadequate cost function: blind to overly complex models Solution: incorporate uncertainty in model parameters Ordinary least squares

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Bayesian Paradigm: priors and likelihood Model:

Bayesian Paradigm: priors and likelihood Model: Prior:

Sample curves from prior (before observing any data) Mean curve Bayesian Paradigm: priors and likelihood Model: Prior:

Bayesian Paradigm: priors and likelihood Model: Prior: Likelihood:

Bayesian Paradigm: priors and likelihood Model: Prior: Likelihood:

Bayesian Paradigm: priors and likelihood Model: Prior: Likelihood:

Bayesian Paradigm: posterior Model: Prior: Likelihood: Bayes Rule: Posterior:

Bayesian Paradigm: posterior Model: Prior: Likelihood: Bayes Rule: Posterior:

Bayesian Paradigm: posterior Model: Prior: Likelihood: Bayes Rule: Posterior:

Bayesian Paradigm: model selection Cost function Bayes Rule: normalizing constant Model evidence:

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Hierarchical models recognition time space generation

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Variational methods: approximate inference True posterior Initial guess and iteratively improve to approximate true posterior L F KL fixed Can compute Maximize minimize KL Difference btw approx. and true posterior But cannot compute as do not know But how?

Variational methods: approximate inference accuracy complexity If you assume posterior factorises then F can be maximised by letting where

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

AR coeff (correlated noise) prior precision of AR coeff VB estimate of W ML estimate of W aMRI smoothed W (RFT) fMRI time series analysis with spatial priors observations GLM coeff prior precision of GLM coeff prior precision of data noise Penny et al 2005 degree of smoothnessSpatial precision matrix

AR coeff (correlated noise) prior precision of AR coeff VB estimate of W ML estimate of W aMRI smoothed W (RFT) fMRI time series analysis with spatial priors observations GLM coeff prior precision of GLM coeff prior precision of data noise Penny et al 2005

Mean (Cbeta_*.img) Std dev (SDbeta_*.img) PPM (spmP_*.img) activation threshold Posterior density q(w n ) Probability mass p n fMRI time series analysis with spatial priors: posterior probability maps probability of getting an effect, given the data mean: size of effect covariance: uncertainty Display only voxels that exceed e.g. 95%

fMRI time series analysis with spatial priors: single subject - auditory dataset Active > Rest Active != Rest Overlay of effect sizes at voxels where SPM is 99% sure that the effect size is greater than 2% of the global mean Overlay of 2 statistics: This shows voxels where the activation is different between active and rest conditions, whether positive or negative

fMRI time series analysis with spatial priors: group data – Bayesian model selection Compute log-evidence for each model/subject model 1 model K subject 1 subject N Log-evidence maps

fMRI time series analysis with spatial priors: group data – Bayesian model selection BMS maps PPM EPM model k Joao et al, 2009 (submitted) Compute log-evidence for each model/subject model 1 model K subject 1 subject N Log-evidence maps Probability that model k generated data

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Distributed Source model MEG/EEG Source Reconstruction Forward model (generation) Inversion (recognition) - under-determined system - priors required Mattout et al, 2006 [n x t][n x p] [n x t] [p x t] n : number of sensors p : number of dipoles t : number of time samples

Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

Neural state equation: Electric/magnetic forward model: neural activity EEG MEG LFP Neural model: 1 state variable per region bilinear state equation no propagation delays Neural model: 8 state variables per region nonlinear state equation propagation delays fMRIERPs inputs Hemodynamic forward model: neural activity BOLD Dynamic Causal Modelling: generative model for fMRI and ERPs

Thank-you