1. Wellcome Dept. of Imaging Neuroscience University College London
Hierarchical Models
Stefan Kiebel
Wellcome Dept. of Imaging Neuroscience
University College London

2. Overview Why hierarchical models? The model Estimation
Expectation-Maximization
Summary Statistics approach
Variance components
Examples

3. Overview Why hierarchical models? Why hierarchical models? The model
Estimation
Expectation-Maximization
Summary Statistics approach
Variance components
Examples

4. Data Time fMRI, single subject EEG/MEG, single subject
fMRI, multi-subject
ERP/ERF, multi-subject
Hierarchical model for all imaging data!

5. Reminder: voxel by voxel
model
specification
parameter
estimation
Time
hypothesis
statistic
Time
Intensity
single voxel
time series
SPM

6. Overview Why hierarchical models? The model The model Estimation
Expectation-Maximization
Summary Statistics approach
Variance components
Examples

7. General Linear Model = +
Model is specified by
Design matrix X
Assumptions about e
N: number of scans
p: number of regressors

8. Linear hierarchical model
Multiple variance components at each level
Hierarchical model
At each level, distribution of parameters is given by level above.
What we don’t know: distribution of parameters and variance parameters.

9. Example: Two level model= + = +
Second level
First level

10. Algorithmic Equivalence
Parametric
Empirical
Bayes (PEB)
Hierarchical
model
EM = PEB = ReML
Single-level
model
Restricted
Maximum
Likelihood
(ReML)

11. Overview Why hierarchical models? The model Estimation Estimation
Expectation-Maximization
Summary Statistics approach
Variance components
Examples

12. Estimation EM-algorithm E-step M-step Assume, at voxel j:
Friston et al. 2002, Neuroimage

13. And in practice? Most 2-level models are just too big to compute.
And even if, it takes a long time!
Moreover, sometimes we‘re only interested in one specific effect and don‘t want to model all the data.
Is there a fast approximation?

14. Summary Statistics approach
First level
Second level
Data Design Matrix Contrast Images
SPM(t)
One-sample
2nd level

15. Validity of approach
The summary stats approach is exact under i.i.d. assumptions at the 2nd level, if for each session:
Within-session covariance the same
First-level design the same
One contrast per session
All other cases: Summary stats approach seems to be robust against typical violations.

16. Mixed-effects Summary statistics EM approach Step 1 Step 2
Friston et al. (2004) Mixed effects and fMRI studies, Neuroimage

17. Overview Why hierarchical models? The model Estimation
Expectation-Maximization
Summary Statistics approach
Variance components
Variance components
Examples

18. Sphericity
Scans
‚sphericity‘ means:
i.e.
Scans

19. 2nd level: Non-sphericity
Error
covariance
Errors are independent
but not identical
Errors are not independent
and not identical

20. 1st level fMRI: serial correlations
with
autoregressive process of order 1 (AR-1)
autocovariance-
function
COMPUTATIONAL EFFICIENCY
To test a hypothesis a lot of images (on the order of 104) are accessed. Acceptable for interpolated channel data, but probably not for source reconstructed 3D-data.
For fMRI, this procedure is ok, because there are not really many hypotheses
(HRF up or HRF down + differences/interactions)
For EEG, there are many more hypotheses to test, because there are many potential different hypotheses
Some hypotheses cannot be tested with conventional model: any contrasts with lots of different regressors at 2nd level!!

21. Restricted Maximum Likelihood
observed
ReML
estimated
COMPUTATIONAL EFFICIENCY
To test a hypothesis a lot of images (on the order of 104) are accessed. Acceptable for interpolated channel data, but probably not for source reconstructed 3D-data.
For fMRI, this procedure is ok, because there are not really many hypotheses
(HRF up or HRF down + differences/interactions)
For EEG, there are many more hypotheses to test, because there are many potential different hypotheses
Some hypotheses cannot be tested with conventional model: any contrasts with lots of different regressors at 2nd level!!

22. Estimation in SPM5 ReML (pooled estimate) Ordinary least-squares
Maximum Likelihood
optional in SPM5
one pass through data
statistic using effective degrees of freedom
2 passes (first pass for selection of voxels)
more accurate estimate of V

23. 2nd level: Non-sphericity
Errors can be
non-independent and non-identical
when…
Several parameters per subject, e.g. repeated measures design
Complete characterization of the hemodynamic response,
e.g. taking more than 1 HRF parameter to 2nd level
Example data set (R Henson) on:

24. Overview Why hierarchical models? The model Estimation
Expectation-Maximization
Summary Statistics approach
Variance components
Examples
Examples

25. Example I Stimuli: Subjects: Scanning:
Auditory Presentation (SOA = 4 secs) of
(i) words and (ii) words spoken backwards
Stimuli:
e.g.
“Book”
and
“Koob”
(i) 12 control subjects
(ii) 11 blind subjects
Subjects:
Scanning:
fMRI, 250 scans per subject, block design
U. Noppeney et al.

26. Population differences
1st level:
Controls
Blinds
2nd level:

27. Auditory Presentation (SOA = 4 secs) of words
Example II
Stimuli:
Auditory Presentation (SOA = 4 secs) of words
Motion
Sound
Visual
Action
“jump”
“click”
“pink”
“turn”
Subjects:
(i) 12 control subjects
fMRI, 250 scans per
subject, block design
Scanning:
What regions are affected
by the semantic content of
the words?
Question:
U. Noppeney et al.

28. Repeated measures Anova
1st level:
Motion
Sound
Visual
Action ? = ? = ? =
2nd level:

29. Example 3: 2-level ERP model
Single subject
Multiple subjects
Subject 1
Trial type 1
. . .
. . .
Subject j
Trial type i
. . .
. . .
Subject m
Trial type n

30. Test for difference in N170
ERP-specific contrasts:
Average between 150 and 190 ms
Example: difference
between trial types
2nd level contrast
-1 1
. . . = = +
Identity
matrix
second level
first level

31. Some practical points
RFX: If not using multi-dimensional contrasts at 2nd level (F-tests), use a series of 1-sample t-tests at the 2nd level.
Use mixed-effects model only, if seriously in doubt about validity of summary statistics approach.
If using variance components at 2nd level: Always check your variance components (explore design).

32. Conclusion
Linear hierarchical models are general enough for typical multi-subject imaging data (PET, fMRI, EEG/MEG).
Summary statistics are robust approximation to mixed-effects analysis.
To minimize number of variance components to be estimated at 2nd level, compute relevant contrasts at 1st level and use simple test at 2nd level.