Multimodal Brain Imaging Will D. Penny FIL, London Guillaume Flandin, CEA, Paris Nelson Trujillo-Barreto, CNC, Havana

Experimental Manipulation Neuronal Activity MEG,EEG Optical Imaging PET fMRI Single/multi-unit recordings Spatial convolution via Maxwell’s equations Temporal convolution via Hemodynamic/Balloon models FORWARD MODELS Sensorimotor Memory Language Emotion Social cognition

Experimental Manipulation Neuronal Activity MEG,EEG fMRI Spatial deconvolution via beamformers Temporal deconvolution via model fitting/inversion INVERSION 1. Spatio-temporal deconvolution 2. Probabilistic treatment

Overview Spatio-temporal deconvolution for M/EEG Spatio-temporal deconvolution for fMRI Towards models for multimodal imaging

Spatio-temporal deconvolution for M/EEG Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal. Puts M/EEG analysis into same framework as PET/fMRI analysis. Work with Nelson. Described in chapter of new SPM book.

Generative Model: Hyperpriors:

Variational Bayes: Mean-Field Approximation Repeat Update source estimates, q(j) Update regression coefficients, q(w) Update spatial precisions, q(  ) Update temporal precisions, q( ) Update sensor precisions, q(  ) Until change in F is small L F KL

Mean-Field Approximation: Approximated posteriors:

Corr(R3,R4)=0.47

o + 500ms Low Symmetry Low Asymmetry High Symmetry High Asymmetry Phase 1 Time 600ms + 700ms + o 2456ms + FaFa + SbSb UbUb + SaSa Henson R. et al., Cerebral Cortex, 2005

B8 A1 Faces minus Scrambled Faces 170ms post-stimulus

B8A1 Faces Scrambled Faces

Daubechies Cubic Splines Wavelets

28 Basis Functions 30 Basis Functions Daubechies-4

ERP Faces ERP Scrambled

t = 170 ms

Faces – Scrambled faces: Difference of absolute values

Spatio-temporal deconvolution for fMRI Temporal evolution is described by GLM in the usual way. Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets. Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases. Embed this in a probabilistic model. Work with Guillaume. To appear in Neuroimage very soon.

Spatial Model eg. Wavelets

Mixture prior on wavelet coefficients (1)Wavelet switches: d=1 if coefficient is ON. Occurs with probability  (2)If switch is on, draw z from the fat Gaussian.

Probabilistic Generative Model fMRI data General Linear Model Wavelet coefficients Temporal Model Spatial Model Wavelet switches Switch priors

Compare to (i) GMRF prior used in M/EEG and (ii) no prior

Inversion using wavelet priors is faster than using standard EEG priors

Results on face fMRI data

Towards multimodal imaging Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms) EEG is compromised -> artifact removal Testing the `heuristic’ Start work on specifying generative models Ongoing work with Felix Blankenburg and James Kilner

fMRI results

We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier MRI Gradient artefact removal from EEG

Ballistocardiogram removal Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

Ballistocardiogram removal

The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that increases in average EEG frequency predict BOLD activation. g(w) = spectral density Testing the heuristic

RMSF for Marta’s data at Cz

Log of Bayes factor for Heuristic versus Null

Log of Bayes factor for Heuristic versus Alpha

Tentative probabilistic generative model

THANK-YOU FOR YOUR ATTENTION !