Anatomic Geometry & Deformations and Their Population Statistics (Or Making Big Problems Small) Stephen M. Pizer, Kenan Professor Medical Image Display & Analysis Group University of North Carolina website: midag.cs.unc.edu Co-authors: P. Thomas Fletcher, Sarang Joshi, Conglin Lu, and numerous others in MIDAG Stephen M. Pizer, Kenan Professor Medical Image Display & Analysis Group University of North Carolina website: midag.cs.unc.edu Co-authors: P. Thomas Fletcher, Sarang Joshi, Conglin Lu, and numerous others in MIDAG

Real-World Analysis with Images as Data Shape of Objects in Populations via representations z ä ä Uses for probability density p(z) ä ä Sampling p(z) to communicate anatomic variability in atlases ä ä Log prior in posterior optimizing deformable model segmentation ä ä Optimize log p(z|I), so log p(z) + log p(I|z) ä ä Compare two populations ä ä Medical science: localities where p(z|healthy) & p(z|diseased) differ ä ä Diagnostic: Is particular patient’s geometry diseased? p(z|healthy, I) vs. p(z|diseased, I) ä ä Uses for probability density p(z) ä ä Sampling p(z) to communicate anatomic variability in atlases ä ä Log prior in posterior optimizing deformable model segmentation ä ä Optimize log p(z|I), so log p(z) + log p(I|z) ä ä Compare two populations ä ä Medical science: localities where p(z|healthy) & p(z|diseased) differ ä ä Diagnostic: Is particular patient’s geometry diseased? p(z|healthy, I) vs. p(z|diseased, I)

Plan of Talk ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis

Needs of Geometric Representation z & Probability Representation p(z) ä p(z) limited samples, i.e., b ä Accurate p(z) estimation with limited samples, i.e., beat High Dimension Low Sample Size (HDLSS: many features, few training cases) ä ä Primitives’ positional correspondence; cases alignment ä ä Easy fit of z to each training segmentation or image ä ä Handle multi-object complexes ä ä Rich geometric representation ä ä Local twist, bend, swell? ä ä Make geometric effects intuitive ä ä Null probabilities for geometrically illegal objects ä ä Localization: Multiscale framework ä p(z) limited samples, i.e., b ä Accurate p(z) estimation with limited samples, i.e., beat High Dimension Low Sample Size (HDLSS: many features, few training cases) ä ä Primitives’ positional correspondence; cases alignment ä ä Easy fit of z to each training segmentation or image ä ä Handle multi-object complexes ä ä Rich geometric representation ä ä Local twist, bend, swell? ä ä Make geometric effects intuitive ä ä Null probabilities for geometrically illegal objects ä ä Localization: Multiscale framework

Representations z of Deformation All but landmarks look initially big ä ä Landmarks ä ä Boundary of objects (b-reps) ä ä Points spaced along boundary ä ä or Coefficients of expansion in basis functions ä ä or Function in 3D with level set as object boundary ä ä Deformation velocity seq. per voxel ä ä Medial representation of objects’ interiors (m-reps) ä ä Landmarks ä ä Boundary of objects (b-reps) ä ä Points spaced along boundary ä ä or Coefficients of expansion in basis functions ä ä or Function in 3D with level set as object boundary ä ä Deformation velocity seq. per voxel ä ä Medial representation of objects’ interiors (m-reps)

Plan of Talk ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis

Standard Method of Making Big Problems Small via Statistics: Principal Component Analysis (PCA) Linear Statistics (PCA) Mean: closest to data in square distance. Principal direction submanifold: through mean; closest to data in square distance. D   m e.g., D=T n with T   3 D   m e.g., D=T n with T   3 New features are components in each of first few principal directions Each describes a global deformation of the mean

Plan of Talk ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis

Landmarks as Representation z ä ä First historically ä ä Kendall, Bookstein, Dryden & Mardia, Joshi ä ä Landmarks defined by special properties ä ä Won’t find many accurately in 3D ä ä Alignment via Procrustes ä ä p(z) via PCA on pt. displacements ä ä Avoid foldings via PDE-based interpolation ä ä Unintuitive principal warps ä ä Global only, and spatially inaccurate between landmarks ä ä First historically ä ä Kendall, Bookstein, Dryden & Mardia, Joshi ä ä Landmarks defined by special properties ä ä Won’t find many accurately in 3D ä ä Alignment via Procrustes ä ä p(z) via PCA on pt. displacements ä ä Avoid foldings via PDE-based interpolation ä ä Unintuitive principal warps ä ä Global only, and spatially inaccurate between landmarks

B-reps as Representation z ä ä Point samples: like landmarks; popular ä ä Fit to training objects pretty easy ä ä Handles multi-object complexes ä ä HDLSS? Typically no; not enough stable principal directions ä ä Positional correspondence of primitives ä ä Expensive reparametrization to optimize p(z) tightness ä ä Only characterization of local translations of shell ä ä Weak re null probabilities for geometric illegals ä ä Basis function coefficients ä ä Achieves legality; fit correspondence imperfect ä ä Implicit, questionable positional correspondence ä ä Global, Unintuitive ä ä Level set of F(x) ä ä Point samples: like landmarks; popular ä ä Fit to training objects pretty easy ä ä Handles multi-object complexes ä ä HDLSS? Typically no; not enough stable principal directions ä ä Positional correspondence of primitives ä ä Expensive reparametrization to optimize p(z) tightness ä ä Only characterization of local translations of shell ä ä Weak re null probabilities for geometric illegals ä ä Basis function coefficients ä ä Achieves legality; fit correspondence imperfect ä ä Implicit, questionable positional correspondence ä ä Global, Unintuitive ä ä Level set of F(x)

Probability p(z) for B-reps ä ä PCA on point displacement [Cootes & Taylor] ä ä Global, i.e., no localization ä ä PCA on basis function coefficients [Gerig] ä ä Also, global ä ä PCA on point displacement [Cootes & Taylor] ä ä Global, i.e., no localization ä ä PCA on basis function coefficients [Gerig] ä ä Also, global

z is F Objective: to handle topological variability B-rep via F(x)’s level set: z is F Objective: to handle topological variability ä ä Run nonlinear diffusion to achieve deformation ä ä Fit to training cases easy via distance ä ä Correspondence? ä ä Rich characterization of geometric effects? Yes, but objects unexplicit ä ä Unintuitive ä ä Stats inadequately developed ä ä Null probabilities for geometrically illegal objects ä ä No if statistics on function ä ä Yes if statistics on PDE ä ä Localization: via spatially varying PDE parameters?? ä ä Run nonlinear diffusion to achieve deformation ä ä Fit to training cases easy via distance ä ä Correspondence? ä ä Rich characterization of geometric effects? Yes, but objects unexplicit ä ä Unintuitive ä ä Stats inadequately developed ä ä Null probabilities for geometrically illegal objects ä ä No if statistics on function ä ä Yes if statistics on PDE ä ä Localization: via spatially varying PDE parameters?? Topology change

as representation z Deformation velocity sequence for each voxel as representation z ä ä Miller, Christensen, Joshi ä ä Labels in reference move with deformation ä ä Series of local interactions ä ä Deformation energy minimization ä ä Fluid flow ä ä Hard to fit to training cases if nonlocal ä ä Alignment and mean by centroid that minimizes deformation energies [Davis & Joshi]   A B via A  then (B  ) -1 ä ä Miller, Christensen, Joshi ä ä Labels in reference move with deformation ä ä Series of local interactions ä ä Deformation energy minimization ä ä Fluid flow ä ä Hard to fit to training cases if nonlocal ä ä Alignment and mean by centroid that minimizes deformation energies [Davis & Joshi]   A B via A  then (B  ) -1

Probability p(z) for Probability p(z) for deformation velocity sequence per voxel ä ä PCA on collection of point displacements [Csernansky] (better on velocity sequence) ä ä Global ä ä Large problem of HDLSS ä ä Few PCA coefficients are stable ä ä Intuitive characterization of geometric effects: warp ä ä Very local translations, but also very local rotations, magnifications ä ä Can produce geometrically illegal objects ä ä Benefit from p(nonlinear transformations)? ä Need many-voxel tests done via permutations ä ä PCA on collection of point displacements [Csernansky] (better on velocity sequence) ä ä Global ä ä Large problem of HDLSS ä ä Few PCA coefficients are stable ä ä Intuitive characterization of geometric effects: warp ä ä Very local translations, but also very local rotations, magnifications ä ä Can produce geometrically illegal objects ä ä Benefit from p(nonlinear transformations)? ä Need many-voxel tests done via permutations cf.

Plan of Talk ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis

M-reps as Representation z Represent the Egg, not the Eggshell ä ä The eggshell: object boundary primitives ä ä The egg: object interior primitives ä ä M-reps ä ä Transformations of primitives: local displacement, local bending & twisting (rotations), local swelling/contraction ä ä Handles multifigure objects and multi-object complexes ä ä Interstitial space?? ä ä The eggshell: object boundary primitives ä ä The egg: object interior primitives ä ä M-reps ä ä Transformations of primitives: local displacement, local bending & twisting (rotations), local swelling/contraction ä ä Handles multifigure objects and multi-object complexes ä ä Interstitial space??

A deformable model of the object interior: the m-rep ä Object interior primitives: medial atoms ä Objects, figures ä Local displacement, bending/twisting, swelling: intuitive ä Neighbor geometry ä Represent interior sections ä Object interior primitives: medial atoms ä Objects, figures ä Local displacement, bending/twisting, swelling: intuitive ä Neighbor geometry ä Represent interior sections

Medial atom as a geometric transformation ä Medial atoms carry position, width, 2 orientations ä Local deformation T   3 ×  + × S 2 × S 2 ä From reference atom ä Hub translation × Spoke magnification in common × spoke 1 rotation × spoke 2 rotation ä S 2 = SO(3)/SO(2) ä Represent interior section ä M-rep is n-tuple of medial atoms ä T n, n local T’s, a symmetric space ä Medial atoms carry position, width, 2 orientations ä Local deformation T   3 ×  + × S 2 × S 2 ä From reference atom ä Hub translation × Spoke magnification in common × spoke 1 rotation × spoke 2 rotation ä S 2 = SO(3)/SO(2) ä Represent interior section ä M-rep is n-tuple of medial atoms ä T n, n local T’s, a symmetric space u v t

Preprocessing for computing p(z) for Preprocessing for computing p(z) for M-reps ä ä Fitting m-reps into training binaries ä ä Edge-constrained ä ä Irregularity penalty ä ä Yields correspondence(?) ä ä Interpenetration avoidance ä ä Alignment via minimization of sum of squared interior distances (geodesic distances -- next slide) ä ä Fitting m-reps into training binaries ä ä Edge-constrained ä ä Irregularity penalty ä ä Yields correspondence(?) ä ä Interpenetration avoidance ä ä Alignment via minimization of sum of squared interior distances (geodesic distances -- next slide)

Principal Geodesic Analysis (PGA) [Fletcher] Curved Statistics (PGA)Linear Statistics (PCA) Mean: closest to data in square distance. Principal direction submanifold: through mean; closest to data in square distance. T with T   3 ×  + × S 2 × S 2 T n with T   3 ×  + × S 2 × S 2

Advantages of Geodesic Geometry with Rotation & Magnification ä Strikingly fewer principal components than PCA (LDLSS) ä Avoids geometric illegals ä Procrustes in geodesic geometry to align interiors ä Geodesic interpolation in time & space is natural A to B to A to C to A

Statistics at Any Scale Level ä Global ä By object ä By figure (atom mesh) ä By atom (interior section) ä By voxel or boundary vertex quad-mesh neighbor relations atom level boundary level

Representation of multiple objects via residues from global variation ä Interscale residues ä E.g., global to per-object ä Provides localization ä Inter-relations between objects (or figures) ä Augmentation via highly correlated (near) atoms ä Prediction of remainder via augmenting atoms ä Interscale residues ä E.g., global to per-object ä Provides localization ä Inter-relations between objects (or figures) ä Augmentation via highly correlated (near) atoms ä Prediction of remainder via augmenting atoms

Does nonlinear stats on m-reps work? Ex: Use of p(m) for Segmentation ä Here, within interday changes within a patient ä Extraction of bladder, prostate, rectum via global, bladder, prostate, rectum sequence of posterior optimizations ä Speed: <5 min. today, ~10 seconds in new version ä Here, within interday changes within a patient ä Extraction of bladder, prostate, rectum via global, bladder, prostate, rectum sequence of posterior optimizations ä Speed: <5 min. today, ~10 seconds in new version

Plan of Talk ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis ä ä Needs of geometric object(s) representations z ä ä The menagerie of geometric object(s) representations ä ä How to make big problems in statistics small: PCA ä ä Properties of the representations and of forming probability densities on them ä ä Making the problem small via interior models with natural deformations and statistical analysis suited to interior models (PGA) ä ä Summary; Research strategy in image analysis

Summary re Estimating p(z) via stats on geometric transformations ä Needs ä Easy, correspondent fit to training segmentations ä Accuracy using limited samples ä Choices ä Representation of object interiors vs. boundaries ä Object complexes: Object by object vs. global ä Statistics of inter-object relations (canonical correlation?) ä Combine with voxel deformation to refine and extrapolate to interstitial spaces ä Physical vs. or and statistical models, Complexity vs. simplicity, Global vs. local ä Results so far: m-reps +voxel deformation hybrid best, but jury out & more representations to discover: functions & level sets? ä Needs ä Easy, correspondent fit to training segmentations ä Accuracy using limited samples ä Choices ä Representation of object interiors vs. boundaries ä Object complexes: Object by object vs. global ä Statistics of inter-object relations (canonical correlation?) ä Combine with voxel deformation to refine and extrapolate to interstitial spaces ä Physical vs. or and statistical models, Complexity vs. simplicity, Global vs. local ä Results so far: m-reps +voxel deformation hybrid best, but jury out & more representations to discover: functions & level sets?

Conclusions re Strategy in Object or Image Analysis ä How to Deal with Big Geometric Things ä By doing analyses well fit to the geometry and to the population, make the big things small ä These analyses require strengths of mathematicians, of statisticians, of computer scientists, and of users all working together ä How to Deal with Big Geometric Things ä By doing analyses well fit to the geometry and to the population, make the big things small ä These analyses require strengths of mathematicians, of statisticians, of computer scientists, and of users all working together

Want more info? ä ä This tutorial, many papers on m-reps and their statistics and applications can be found at website ä ä For other representations see references on following 2 slides ä ä This tutorial, many papers on m-reps and their statistics and applications can be found at website ä ä For other representations see references on following 2 slides

Kendall, D (1986). Size and Shape Spaces for Landmark Data in Two Dimensions. Statistical Science, 1: Bookstein, F (1991). Morphometric Tools for Landmark Data: Geometry and Biology. Cambridge University Press. Dryden, I and K Mardia (1998). Statistical Shape Analysis. John Wiley & Sons. Cootes, T and C Taylor (2001). Statistical Models of Appearance for Medical Image Analysis and Computer Vision. Proc. SPIE Medical Imaging. Grenander, U and MI Miller (1998), Computational Anatomy: An Emerging Discipline, Quarterly of Applied Math., 56: References

Csernansky, J, S Joshi, L Wang, J Haller, M Gado, J Miller, U Grenander, M Miller (1998). Hippocampal morphometry in schizophrenia via high dimensional brain mapping. Proc. Natl. Acad. Sci. USA, 95: Caselles, V, R Kimmel, G Sapiro (1997). Geodesic Active Contours. International Journal of Computer Vision, 22(1): Pizer, S, K Siddiqi, G Szekely, J Damon, S Zucker (2003). Multiscale Medial Loci and Their Properties. International Journal of Computer Vision - Special UNC-MIDAG issue, (O Faugeras, K Ikeuchi, and J Ponce, eds.), 55(2): Fletcher, P, C Lu, S Pizer, S Joshi (2004). Principal Geodesic Analysis for the Study of Nonlinear Statistics of Shape. IEEE Transactions on Medical Imaging, 23(8): References