1. Multimodal Brain Imaging Will D. Penny FIL, London Guillaume Flandin, CEA, Paris Nelson Trujillo-Barreto, CNC, Havana

2. Experimental Manipulation Neuronal Activity MEG,EEG Optical Imaging PET fMRI Single/multi-unit recordings Spatial convolution via Maxwell’s equations Temporal convolution via Hemodynamic/Balloon models FORWARD MODELS Sensorimotor Memory Language Emotion Social cognition

3. Experimental Manipulation Neuronal Activity MEG,EEG fMRI Spatial deconvolution via beamformers Temporal deconvolution via model fitting/inversion INVERSION 1. Spatio-temporal deconvolution 2. Probabilistic treatment

4. Overview Spatio-temporal deconvolution for M/EEG Spatio-temporal deconvolution for fMRI Towards models for multimodal imaging

5. Spatio-temporal deconvolution for M/EEG Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal. Puts M/EEG analysis into same framework as PET/fMRI analysis. Work with Nelson. Described in chapter of new SPM book.

7. Generative Model: Hyperpriors:

8. Variational Bayes: Mean-Field Approximation Repeat Update source estimates, q(j) Update regression coefficients, q(w) Update spatial precisions, q(  ) Update temporal precisions, q( ) Update sensor precisions, q(  ) Until change in F is small L F KL

9. Mean-Field Approximation: Approximated posteriors:

11. Corr(R3,R4)=0.47

14. o + 500ms Low Symmetry Low Asymmetry High Symmetry High Asymmetry Phase 1 Time 600ms + 700ms + o 2456ms + FaFa + SbSb UbUb + SaSa Henson R. et al., Cerebral Cortex, 2005

15. B8 A1 Faces minus Scrambled Faces 170ms post-stimulus

16. B8A1 Faces Scrambled Faces

17. Daubechies Cubic Splines Wavelets

18. 28 Basis Functions 30 Basis Functions Daubechies-4

19. ERP Faces ERP Scrambled

20. t = 170 ms

21. Faces – Scrambled faces: Difference of absolute values

22. Spatio-temporal deconvolution for fMRI Temporal evolution is described by GLM in the usual way. Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets. Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases. Embed this in a probabilistic model. Work with Guillaume. To appear in Neuroimage very soon.

23. Spatial Model eg. Wavelets

24. Mixture prior on wavelet coefficients (1)Wavelet switches: d=1 if coefficient is ON. Occurs with probability  (2)If switch is on, draw z from the fat Gaussian.

25. Probabilistic Generative Model fMRI data General Linear Model Wavelet coefficients Temporal Model Spatial Model Wavelet switches Switch priors

27. Compare to (i) GMRF prior used in M/EEG and (ii) no prior

28. Inversion using wavelet priors is faster than using standard EEG priors

29. Results on face fMRI data

30. Towards multimodal imaging Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms) EEG is compromised -> artifact removal Testing the `heuristic’ Start work on specifying generative models Ongoing work with Felix Blankenburg and James Kilner

31. fMRI results

33. We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier MRI Gradient artefact removal from EEG

34. Ballistocardiogram removal Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

35. Ballistocardiogram removal

37. The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that increases in average EEG frequency predict BOLD activation. g(w) = spectral density Testing the heuristic

38. RMSF for Marta’s data at Cz

39. Log of Bayes factor for Heuristic versus Null

40. Log of Bayes factor for Heuristic versus Alpha

41. Tentative probabilistic generative model

42. THANK-YOU FOR YOUR ATTENTION !