1. NA-MIC National Alliance for Medical Image Computing Non-Parametric Statistical Permutation Tests for Local Shape Analysis Martin Styner, UNC Dimitrios Pantazis, Richard Leahy, USC LA Tom Nichols, University of Michigan Ann Arbor

2. National Alliance for Medical Image Computing http://na-mic.org TOC Motivation local shape analysis Local shape difference/distance measures Statistical significance maps Problem: Multiple correlated comparisons 1 st approach: It’s a hack! 2 nd approach: Let’s do it right! Template free - Hotelling T 2 measures Example Results Conclusions & Outlook

3. National Alliance for Medical Image Computing http://na-mic.org Motivation Shape Analysis Anatomical studies of brain structures –Changes between patient and healthy controls –Detection, Enhanced understanding, course of disease, pathology –Normal neuro-development interest in diseases with brain changes –Schizophrenia, autism, fragile-X, Alzheimer's Information additional to volume Both volumetric and shape analysis Shape analysis: where and how?

4. National Alliance for Medical Image Computing http://na-mic.org Shape Distances Shape description: –SPHARM-PDM –M-rep Normalization: –Rigid Procrustes, brain size normalized Local scalar distance –Euclidean distance –“Radius” difference –Signed vs absolute

5. National Alliance for Medical Image Computing http://na-mic.org Local Shape Analysis Distance to template Distance between subject pairs Sets of distance-maps Significance map –Statistical test at each point –Mean difference test P-values Significance threshold

6. National Alliance for Medical Image Computing http://na-mic.org Multiple Comparisons Lots of correlated statistical tests → Overly optimistic –M-rep: 2x24 tests, SPHARM: 2252 tests –Same problem with other shape descriptions and other difference analysis schemes Correction needed, overly optimistic –Test locally at given level (e.g. α = 0.05) –Globally incorrect false-positive rate Bonferroni correction, worst case, assumption: 0% correlation –Correct False-Positive rate at α/n = 0.05/4000 = 0.0000125 –Correct False-Positive rate at 1-(1- α) 1/n = 0.0000128

7. National Alliance for Medical Image Computing http://na-mic.org 1 st Approach: SnPM Statistical non-Parametric Maps in SPM (SPIE 2004) Decomposition of distance map into separate images for processing in SnPM 75% overlap necessary due to distortions Each image is tested separately in SnPM ONE BIG HACK: –6 correlated tests –Averaging in overlap

8. National Alliance for Medical Image Computing http://na-mic.org 2 nd Approach: Permutations Non-parametric permutation test using spatially summarized statistics, ISBI 2004 Correct false positive control (Type II) Summary: –Random permutations of the group labels –Metric for difference between populations –Spatial normalization for uniform spatial sensitivity –Summarize statistics across whole shape –Choose threshold in summary statistic

9. National Alliance for Medical Image Computing http://na-mic.org Statistical Problem 2 groups: a & b, #member n a, n b Each member: p-features (e.g. 4000) Test: Is the mean of each feature in the 2 populations the same? –Null hypothesis: The mean of each feature is the same –Permutations of group label leave distributions unchanged under null hypothesis –M permutations Specific test –Correct false positive rate

10. National Alliance for Medical Image Computing http://na-mic.org Non-parametric Permutation Tests Goal: significance for a vector with 4’000 correlated variables 50’000 to 100’000 permutations Extrema statistic: controls false-positive diff norm Summary Statistic Min/Max Histogram diff norm

11. National Alliance for Medical Image Computing http://na-mic.org Single Feature Example Feature f A,1 -f A,n1 vs f B,1 -f B,n1 Compute difference: T 0 =|  A -  B| Permute group label → A’ i,B’ I → T i Make Histogram of T i Histogram = pdf Sum histogram = cdf Cdf at 1-α = Threshold α

12. National Alliance for Medical Image Computing http://na-mic.org Multiple features Testing a single feature → no problem Testing multiple features together as a whole, NOT individually Summary is necessary of all features across the surface For correct Type II, use an extrema measurement –Right sided distance metrics → Maxima –Left sided distance metrics → Minima

13. National Alliance for Medical Image Computing http://na-mic.org Spatial Normalization Extremal summary is most influenced by regions with higher variance Assume 2 regions with same difference, but one has larger variance –Region with larger variance contributes more to extremal statistics and thus sensitivity in that region is higher Normalization of local statistical distributions is necessary for spatially uniform sensitivity

14. National Alliance for Medical Image Computing http://na-mic.org Spatial Normalization A) local p-values, non-parametric –Minimum, (1-α) thresh B) standard deviation, parametric –Maximum, α thresh C) q-th quantile, non-parametric –q = 68% ~  if Gaussian –Maximum, α thresh Assumptions: A > C > B Uniform sensitivity: A > C ~ B Numerical pdf: C > B > A Use A –Many permutations –High computation + space costs Extrema statistics Shape difference metric α1-α Norm  Max-stat Norm p-value Min-stat

15. National Alliance for Medical Image Computing http://na-mic.org Raw vs Corrected P-values Raw significance map: –4000 elements, 5% → 200 will be significant at 5% by pure chance, if locations are uncorrelated. Corrected significance map –Correct control of false negative –Single location significant → whole shape significant No assumption over local covariance –Overly pessimistic –There is room for improvement!

16. National Alliance for Medical Image Computing http://na-mic.org Raw vs Corrected P-values Raw p-values are comparable But visualization of raw p-value map is misleading even without statement about significance –Too optimistic, often viewed using linear colormap –P-value correction is non-linear ! Correction factor: F = Raw-P / Corr-P

17. National Alliance for Medical Image Computing http://na-mic.org Metric for Group difference Scalar Local difference: –Signed/Unsigned Euclidean distance –Thickness difference –Pairs, Template Difference of mean metric → Statistical feature T = |  A -  B | Needed: Positive scalar  + shape difference metric between populations PDM: Mean difference of Euclidean distance at a selected point Gaussian, passed Lilliefors test 0.01

18. National Alliance for Medical Image Computing http://na-mic.org Template Free Stats No need for a scalar value at each location for each subject Positive scalar difference value between populations SPHARM-PDM –So far: Signed/absolute Euclidean distance at each location to template → Scalar field analysis –New: Difference vectors to template → Vector field analysis –Better: Location vector at each location → Template free analysis → Length of difference vector between mean vectors of populations → Hotelling T 2 distance between populations = Hotelling T 2 is mean difference 2 vector weighted with the pooled Covariance matrix T 2 = (μ a – μ b ) Σ a,b (μ a –μ b ) Σ a,b = ( (n a - 1) Σ a + (n b -1) Σ b ) / (n a +n b - 2)

19. National Alliance for Medical Image Computing http://na-mic.org Hotelling T2 histogram Hotelling T 2 distance of locations (template free) →  2

20. National Alliance for Medical Image Computing http://na-mic.org Results SnPM hack vs Correct permutation tests Sample Hippocampus study: Stanley study, resp/non-resp SZ (56) vs Cnt (26) –Both M-rep & PDM Other example tests

21. National Alliance for Medical Image Computing http://na-mic.org SnPM-Hack vs Correct Stat SnPM too optimistic –relatively good agreement L 0.001 0.05 R SnPM

22. National Alliance for Medical Image Computing http://na-mic.org Hippocampus SZ Study Left Right

23. National Alliance for Medical Image Computing http://na-mic.org M-rep Shape Analysis Left Right

24. National Alliance for Medical Image Computing http://na-mic.org Vector Field Analysis T 2 location T 2 template difference Abs template distance (scalar) 0.001 0.05 Raw Significance Maps Corr Significance Maps

25. National Alliance for Medical Image Computing http://na-mic.org Conclusions of Methods Multiple comparison correction scheme for local shape analysis –Non-parametric, Permutation-based –Globally correct for false-positive across whole object –Applicable to scalar, vectors, any Euclidean space measures –Black box –Pessimistic estimate

26. National Alliance for Medical Image Computing http://na-mic.org NAMIC kit StatNonParamTestPDM –Command line tool, Win/Linux/MacOSX –E.g. StatNonParamTestPDM -out -surfList -numPerms 50000 -signLevel 0.05 -signSteps 1000 Output (for meshes) –P-value of global shape difference between the populations (mean T 2 across surface) –Mean difference map (effect size) –Hotelling T 2 map using robust T 2 formula –Raw significance map –Corrected significance map –Mean surfaces of the 2 groups

27. National Alliance for Medical Image Computing http://na-mic.org StatNonParamTestPDM Input: File with list of ITK mesh files Generic features also supported using customizable text-file input option Currently in NAMIC-Sandbox (open) Next: submission to Insight Journal MeshVisu, combination of Mesh and maps 0.011 0.2324 0.123 ….. Map Txt

28. National Alliance for Medical Image Computing http://na-mic.org That’s it folks… Questions

29. National Alliance for Medical Image Computing http://na-mic.org Corrected Analysis – Spatial Normalization Without normalization → incorrect, unless uniformity is assumed –High variability → overestimation of significance –Low variability → underestimation of significance  -normalization ~ 68% normalization No norm max stat L R