1. Hierarchical Models and
Variance Components
Will Penny
Wellcome Department of Imaging Neuroscience,
University College London, UK
Cyclotron Research Centre,
University of Liege, April 2003

2. Outline Random Effects Analysis General Framework
Summary statistic approach 2nd level)
General Framework
Multiple variance components and Hierarchical models
Multiple variance components
F-tests and level
Modelling fMRI serial level
Hierarchical models for Bayesian Inference
SPMs versus PPMs

3. Outline Random Effects Analysis General Framework
Summary statistic approach 2nd level)
General Framework
Multiple variance components and Hierarchical models
Multiple variance components
F-tests and level
Modelling fMRI serial level
Hierarchical models for Bayesian Inference
SPMs versus PPMs

4. Random Effects Analysis:Summary-Statistic Approach
1st Level nd Level
Data Design Matrix Contrast Images 1 ^
SPM(t)
1 ^ 2 ^
2 ^
11 ^
11 ^  ^
One-sample
level
12 ^
12 ^

5. Validity of approach ^ ^ Gold Standard approach is EM – see later –
estimates population mean effect as MEANEM
the variance of this estimate as VAREM
For N subjects, n scans per subject and equal within-subject variance
we have
VAREM = Var-between/N + Var-within/Nn
In this case, the SS approach gives the same results, on average:
Avg[a] = MEANEM
Avg[Var(a)] =VAREM
In other cases, with N~12, and typical ratios of between-subject to within-subject variance found in fMRI, the SS approach will give very similar results to EM. ^ ^

6. Two populations Estimated population means Contrast images Two-sample
level

7. Outline Random Effects Analysis General Framework
Summary statistic approach 2nd level)
General Framework
Multiple variance components and Hierarchical models
Multiple variance components
F-tests and level
Modelling fMRI serial level
Hierarchical models for Bayesian Inference
SPMs versus PPMs

8. The General Linear Model
y = X  + e
N  N  L L  N  1
Error covariance N
2 Basic Assumptions
Identity
Independence N
We assume ‘sphericity’

9. Multiple variance components
y = X  + e
N  N  L L  N  1
Error covariance N
Errors can now have
different variances and
there can be correlations N
We allow for ‘nonsphericity’

10. Non-Sphericity Errors are independent but not identical
Errors are not independent
and not identical
Error Covariance

11. General Framework Multiple variance components Hierarchical Models
at each level
With hierarchical models we can define priors and make
Bayesian inferences.
If we know the variance components we can compute
the distributions over the parameters at each level.

12. Estimation EM algorithm Friston, K. et al. (2002), Neuroimage ( ) å y
E-Step ( ) y C X T 1 - = e q h
M-Step r
for i and j { } { Q tr J g i j ij k å + l
Friston, K. et al. (2002), Neuroimage

13. Pooling assumption Decompose error covariance at each voxel, i, into
a voxel specific term, r(i), and voxel-wide terms.
The hyperparameters are now estimated once for the
whole brain.

14. Algorithm Equivalence
Parametric
Empirical
Bayes (PEB)
Hierarchical
model
EM=PEB=ReML
Restricted
Maximimum
Likelihood
(ReML)
Single-level
model

15. Outline Random Effects Analysis General Framework
Summary statistic approach 2nd level)
General Framework
Multiple variance components and Hierarchical models
Multiple variance components
F-tests and level
Modelling fMRI serial level
Hierarchical models for Bayesian Inference
SPMs versus PPMs

16. Non-Sphericity Errors are independent but not identical
Errors are not independent
and not identical
Error Covariance

17. Non-Sphericity Error can be Independent but Non-Identical when…
1) One parameter but from different groups
e.g. patients and control groups
2) One parameter but design matrices differ across subjects
e.g. subsequent memory effect

18. Non-Sphericity Error can be Non-Independent and Non-Identical when…
1) Several parameters per subject
e.g. Repeated Measurement design
2) Conjunction over several parameters
e.g. Common brain activity for different cognitive processes
3) Complete characterization of the hemodynamic response
e.g. F-test combining HRF, temporal derivative and dispersion regressors

19. Example I U. Noppeney et al.
Stimuli: Auditory Presentation (SOA = 4 secs) of
(i) words and (ii) words spoken backwards
Subjects: (i) 12 control subjects
(ii) 11 blind subjects
jump
touch
koob
“click”
Scanning: fMRI, 250 scans per subject, block design
Q. What are the regions that activate for real words relative to
reverse words in both blind and control groups?

20. Independent but Non-Identical Error
1st Level
Controls
Blinds
2nd Level
Controls and Blinds
Conjunction
between the
2 groups

21. Example 2 U. Noppeney et al.
Stimuli: Auditory Presentation (SOA = 4 secs) of words
motion
sound
visual
action
jump
touch
“jump”
“click”
“pink”
“turn”
“click”
Subjects: (i) 12 control subjects
Scanning: fMRI, 250 scans per subject, block design
Q. What regions are affected by the semantic content of
the words ?

22. = = = ? ? ? Non-Independent and Non-Identical Error
1st Leve visual sound hand motion ? = ? = ? =
2nd Level
F-test

23. Example III U. Noppeney et al.
Stimuli: (i) Sentences presented visually
(ii) False fonts (symbols)
Some of the sentences are syntactically primed
Scanning: fMRI, 250 scans per subject, block design
Q. Which brain regions of the “sentence reading system”
are affected by Priming?

24. Non-Independent and Non-Identical Error
1st Level Sentence > Symbols No-Priming>Priming
Orthogonal
contrasts
2nd Level
Conjunction
of 2 contrasts
Left
Anterior
Temporal

25. Example IV Modelling serial correlation in fMRI time series
Model errors for each subject
as AR(1) + white noise.

26. Outline Random Effects Analysis General Framework
Summary statistic approach 2nd level)
General Framework
Multiple variance components and Hierarchical models
Multiple variance components
F-tests and level
Modelling fMRI serial level
Hierarchical models for Bayesian Inference
SPMs versus PPMs

27. The Interface PEB OLS Parameters Parameters, and REML Hyperparameters
No Priors
Shrinkage
priors

28. Bayes Rule

29. Example 2:Univariate model
Likelihood and Prior
Posterior
Relative Precision Weighting

30. Bayesian Inference 1st level = within-voxel Likelihood Shrinkage Prior
In the absence of evidence
to the contrary parameters
will shrink to zero
2nd level = between-voxels

31. Bayesian Inference: Posterior Probability Maps
PPMs
Posterior
Likelihood
Prior
SPMs

32. SPMs and PPMs PPMs: Show activations of a given size
SPMs: show voxels with
non-zero activations

33. PPMs Advantages Disadvantages Use of shrinkage One can infer a cause
priors over voxels
is computationally
demanding
Utility of Bayesian
approach is yet
to be established
One can infer a cause
DID NOT elicit a response
SPMs conflate effect-size
and effect-variability
No multiple comparisons
problem (hence no smoothing)
P-values don’t change with
search volume

34. Summary Random Effects Analysis Multiple variance components
Summary statistic approach 2nd level)
Multiple variance components
F-tests and level
Modelling fMRI serial level
Hierarchical models for Bayesian Inference
SPMs versus PPMs
General Framework
Multiple variance components and Hierarchical models