1. Detecting connectivity between images: MS lesions, cortical thickness, and the 'bubbles' task in an fMRI experiment Keith Worsley, Math + Stats, Arnaud Charil, Montreal Neurological Institute, McGill Philippe Schyns, Fraser Smith, Psychology, Glasgow Jonathan Taylor, Stanford and Université de Montréal

2. What is ‘bubbles’?

3. Nature (2005)

4. Subject is shown one of 40 faces chosen at random … Happy Sad Fearful Neutral

5. … but face is only revealed through random ‘bubbles’ First trial: “Sad” expression Subject is asked the expression: “Neutral” Response: Incorrect Sad 75 random bubble centres Smoothed by a Gaussian ‘bubble’ What the subject sees

6. Your turn … Trial 2 Subject response: “Fearful” CORRECT

7. Your turn … Trial 3 Subject response: “Happy” INCORRECT (Fearful)

8. Your turn … Trial 4 Subject response: “Happy” CORRECT

9. Your turn … Trial 5 Subject response: “Fearful” CORRECT

10. Your turn … Trial 6 Subject response: “Sad” CORRECT

11. Your turn … Trial 7 Subject response: “Happy” CORRECT

12. Your turn … Trial 8 Subject response: “Neutral” CORRECT

13. Your turn … Trial 9 Subject response: “Happy” CORRECT

14. Your turn … Trial 3000 Subject response: “Happy” INCORRECT (Fearful)

15. Bubbles analysis E.g. Fearful (3000/4=750 trials): Trial 1 + 2 + 3 + 4 + 5 + 6 + 7 + … + 750 = Sum Correct trials Proportion of correct bubbles =(sum correct bubbles) /(sum all bubbles) Thresholded at proportion of correct trials=0.68, scaled to [0,1] Use this as a bubble mask

16. Results Mask average face But are these features real or just noise? Need statistics … Happy Sad Fearful Neutral

17. Statistical analysis Correlate bubbles with response (correct = 1, incorrect = 0), separately for each expression Equivalent to 2-sample Z-statistic for correct vs. incorrect bubbles, e.g. Fearful: Very similar to the proportion of correct bubbles: Response 0 1 1 0 1 1 1 … 1 Trial 1 2 3 4 5 6 7 … 750 Z~N(0,1) statistic

18. Both depend on average correct bubbles, rest is ~ constant Comparison Proportion correct bubbles = Average correct bubbles / (average all bubbles * 4) Z=(Average correct bubbles -average incorrect bubbles) / pooled sd

19. Results Thresholded at Z=1.64 (P=0.05) Multiple comparisons correction? Need random field theory … Average face Happy Sad Fearful Neutral Z~N(0,1) statistic

20. Euler Characteristic = #blobs - #holes Excursion set {Z > threshold} for neutral face Heuristic: At high thresholds t, the holes disappear, EC ~ 1 or 0, E(EC) ~ P(max Z > t). Exact expression for E(EC) for all thresholds, E(EC) ~ P(max Z > t) is extremely accurate. EC = 0 0 -7 -11 13 14 9 1 0

21. The details …

22. 2 S Tube(S,r) r

23. 3

24. A B

26. 6 S Tube Λ (S,r) r Λ is small Λ is big

27. S S s1s1 s2s2 s3s3 U(s1)U(s1) U(s2)U(s2) U(s3)U(s3) Tube 2ν

28. R Tube(R,r) r N 2 (0,I) Z1Z1 Z2Z2

29. Tube(R,r) t-r t z Tube(R,r) R z1z1 z2z2 z3z3 R R r

32. Summary

34. Random field theory results For searching in D (=2) dimensions, P-value of max Z is (Adler, 1981; W, 1995): P(max Z > z) ~ E( Euler characteristic of thresholded set ) = Resels × Euler characteristic density (+ boundary) Resels (=Lipschitz-Killing curvature/c) is Image area / (bubble FWHM) 2 = 146.2 Euler characteristic density(×c) is (4 log(2)) D/2 z D-1 exp(-z 2 /2) / (2π) (D+1)/2 See forthcoming book Adler, Taylor (2007)

35. Results, corrected for search Thresholded at Z=3.92 (P=0.05) Average face Happy Sad Fearful Neutral Z~N(0,1) statistic

36. Bubbles task in fMRI scanner Correlate bubbles with BOLD at every voxel: Calculate Z for each pair (bubble pixel, fMRI voxel) – a 5D “image” of Z statistics … Trial 1 2 3 4 5 6 7 … 3000 fMRI

37. Discussion: thresholding Thresholding in advance is vital, since we cannot store all the ~1 billion 5D Z values Resels=(image resels = 146.2) × (fMRI resels = 1057.2) for P=0.05, threshold is Z = 6.22 (approx) The threshold based on Gaussian RFT can be improved using new non-Gaussian RFT based on saddle-point approximations (Chamandy et al., 2006) Model the bubbles as a smoothed Poisson point process The improved thresholds are slightly lower, so more activation is detected Only keep 5D local maxima Z(pixel, voxel) > Z(pixel, 6 neighbours of voxel) > Z(4 neighbours of pixel, voxel)

38. Discussion: modeling The random response is Y=1 (correct) or 0 (incorrect), or Y=fMRI The regressors are X j =bubble mask at pixel j, j=1 … 240x380=91200 (!) Logistic regression or ordinary regression: logit(E(Y)) or E(Y) = b 0 +X 1 b 1 +…+X 91200 b 91200 But there are only n=3000 observations (trials) … Instead, since regressors are independent, fit them one at a time: logit(E(Y)) or E(Y) = b 0 +X j b j However the regressors (bubbles) are random with a simple known distribution, so turn the problem around and condition on Y: E(X j ) = c 0 +Yc j Equivalent to conditional logistic regression (Cox, 1962) which gives exact inference for b 1 conditional on sufficient statistics for b 0 Cox also suggested using saddle-point approximations to improve accuracy of inference … Interactions? logit(E(Y)) or E(Y)=b 0 +X 1 b 1 +…+X 91200 b 91200 +X 1 X 2 b 1,2 + …

39. MS lesions and cortical thickness Idea: MS lesions interrupt neuronal signals, causing thinning in down-stream cortex Data: n = 425 mild MS patients Lesion density, smoothed 10mm Cortical thickness, smoothed 20mm Find connectivity i.e. find voxels in 3D, nodes in 2D with high correlation(lesion density, cortical thickness) Look for high negative correlations …

40. Average lesion volume Average cortical thickness n=425 subjects, correlation = -0.568

41. Thresholding? Cross correlation random field Correlation between 2 fields at 2 different locations, searched over all pairs of locations one in R (D dimensions), one in S (E dimensions) sample size n MS lesion data: P=0.05, c=0.325 Cao & Worsley, Annals of Applied Probability (1999)

42. Normalization LD=lesion density, CT=cortical thickness Simple correlation: Cor( LD, CT ) Subtracting global mean thickness: Cor( LD, CT – av surf (CT) ) And removing overall lesion effect: Cor( LD – av WM (LD), CT – av surf (CT) )

43. threshold Histogram ‘Conditional’ histogram: scaled to same max at each distance

44. Science (2004)

45. fMRI activation detected by correlation between subjects at the same voxel The average nonselective time course across all activated regions obtained during the first 10 min of the movie for all five subjects. Red line represents the across subject average time course. There is a striking degree of synchronization among different individuals watching the same movie. Voxel-by-voxel intersubject correlation between the source subject (ZO) and the target subject (SN). Correlation maps are shown on unfolded left and right hemispheres (LH and RH, respectively). Color indicates the significance level of the intersubject correlation in each voxel. Black dotted lines denote borders of retinotopic visual areas V1, V2, V3, VP, V3A, V4/V8, and estimated border of auditory cortex (A1).The face-, object-, and building-related borders (red, blue, and green rings, respectively) are also superimposed on the map. Note the substantial extent of intersubject correlations and the extension of the correlations beyond visual and auditory cortices.

46. What are the subjects watching during high activation? Faces …

47. … buildings …

48. … hands

49. Thresholding? Homologous correlation random field Correlation between 2 equally smooth fields at the same location, searched over all locations in R (in D dimensions) P-values are larger than for the usual correlation field (correlation between a field and a scalar) E.g. resels=1000, df=100, threshold=5, usual P=0.051, homologous P=0.139 Cao & Worsley, Annals of Applied Probability (1999)

50. Detecting Connectivity between Images: the 'Bubbles' Task in fMRI Keith Worsley, McGill Phillipe Schyns, Fraser Smith, Glasgow

51. Subject is shown one of 40 faces chosen at random … Happy Sad Fearful Neutral … but face is only revealed through random ‘bubbles’ E.g. first trial: “Sad” expression: Subject is asked the expression: “Neutral” Response: Incorrect=0 Sad 75 random bubble centres Smoothed by a Gaussian ‘bubble’ What the subject sees