1. Stephen Pizer Medical Image Display & Analysis Group University of North Carolina, USA with credit to T. Fletcher, A. Thall, S. Joshi, P. Yushkevich, G. Gerig Tutorial:Statistics of Object Geometry Tutorial: Statistics of Object Geometry 10 October 2002

2. Uses of Statistical Geometric Characterization ä Medical science: determine geometric ways in which pathological and normal classes differ ä Diagnostic: determine if particular patient’s geometry is in pathological or normal class ä Educational: communicate anatomic variability in atlases ä Priors for segmentation ä Monte Carlo generation of images

3. Object Representation Objectives ä Relation to other instances of the shape class ä Representing the real world ä Deformation while staying in shape class ä Discrimination by shape class ä Locality ä Relation to Euclidean space/projective Euclidean space ä Matching image data

4. Geometric aspects Invariants and correspondence ä Desire: An image space geometric representation that ä is at multiple levels of scale (locality) ä at one level of scale is based on the object ä and at lower levels based on object’s figures ä at each level recognizes invariances associated with shape ä provides positional and orientational and metric correspondence across various instances of the shape class

5. Object Representations  Atlas voxels with a displacement at each voxel :  x(x) ä Set of distinguished points {x i } with a displacement at each ä Landmarks ä Boundary points in a mesh ä With normal b = (x,n)  Loci of medial atoms: m = (x,F,r,  ) or end atom (x,F,r,  )   u v t

6. Continuous M-reps: B-splines in (x,y,z,r) [Yushkevich]

7. Building an Object Representation from Atoms a ä Sampled ä a ij ä can have inter-atom mesh (active surface) ä Parametrized  a(u,v) ä e.g., spherical harmonics, where coefficients become representation ä e.g., quadric or superquadric surfaces ä some atom components are derivatives of others

8. Object representation: Parametrized Boundaries ä Parametrized boundaries x(u,v)  n(u,v) is normalized  x /  u   x /  v ä Coefficients of decompositions  x(u,v) =  i c i f i (u,v) ä Spherical harmonics: (u,v) = latitude, longitude  Sampled point positions are linear in coefficients: Ax=c

9. Object representation: Parametrized Medial Loci ä Parametrized medial loci m(u,v) = [x,r](u,v)  n(u,v) is normalized  x /  u   x /  v   x r(u,v) = -cos (  ) b  gradient per distance on x(u,v)  b  x n

10. Sampled medial shape representation: Discrete M-rep slabs (bars) ä Meshes of medial atoms ä Objects connected as host, subfigures ä Multiple such objects, interrelated t=+1 p    x s  br   n  t=-1 t=0 p    x s  b  n      u v t

11. Interpolating Medial Atoms in a Figure ä Interpolate x, r via B-splines [ Yushkevich] ä Trimming curve via r<0 at outside control points ä Avoids corner problems of quadmesh ä Yields continuous boundary ä Via modified subdivision surface [Thall]  Approximate orthogonality at sail ends ä Interpolated atoms via boundary and distance  At ends elongation  needs also to be interpolated ä Need to use synthetic medial geometry [Damon] Medial sheet Implied boundary

12. End Atoms: (x,F,r,  ) Medial atom with one more parameter: elongation  Extremely rounded end atom in cross-section Corner atom in cross-section  =1  =1/cos(  ) Rounded end atom in cross-section  =1.4

13. Sampled medial shape representation: M-rep tube figures ä Same atoms as for slabs ä r is radius of tube ä sails are rotated about b ä Chain rather than mesh  b  x n x+ rR b,n (  )b x+rR b,n (-  )b

14. For correspondence: Object-intrinsic coordinates Geometric coordinates from m-reps ä Single figure ä Medial sheet: (u,v) [(u) in 2D] ä t: medial side   : signed r-prop’l dist from implied boundary  3-space: (u,v,t,  ) ä Implied boundary: (u,v,t)   u v t t=+1 p    x s  br   n  t=-1 t=0 p    x s  b  n   

15. Sampled medial shape representation: Linked m-rep slabs ä Linked figures ä Hinge atoms known in figural coordinates (u,v,t) of parent figure ä Other atoms known in the medial coordinates of their neighbors x+ rR b,n (  )b  b  x n x+rR b,n (-  )b

16. ä Blend in hinge regions ä w=(d 1 /r 1 - d 2 /r 2 )/T ä Blended d/r when |w| <1 and u-u 0 < T ä Implicit boundary: (u,w, t) ä Or blend by subdivision surface Figural Coordinates for Object Made From Multiple Attached Figures w

17. ä Inside objects or on boundary ä Per object ä In neighbor object’s coordinates ä Interobject space ä In neighbor object’s coordinates  Far outside boundary: (u[,v],t,  ) via distance (scale) related figural convexification ?? ä ?? Figural Coordinates for Multiple Objects

18. Heuristic Medial Correspondence Radius Original (Spline Parameter)Arclength 0.20.40.60.81 0.2 0.4 0.6 0.8 1 Coordinate Mapping

19. Continuous Analytical Features ä Can be sampled arbitrarily. ä Allow functional shape analysis ä Possible at many scales: medial, bdry, other object Medial Curvature Boundary texture scale

20. Feature-Based Correspondence on Medial Locus by Statistical Registration of Features curvature dr/ds Also works in 3D

21. What is Statistical Geometric Characterization ä Given a population of instances of an object class ä e.g., subcortical regions of normal males of age 30 ä Given a geometric representation z of a given instance ä e.g., a set of positions on the boundary of the object ä and thus the description z i of the i th instance ä A statistical characterization of the class is the probability density p(z) ä which is estimated from the instances z i

22. Benefits of Probabilistically Describing Anatomic Region Geometry ä Discrimination among geometric classes, C k ä Compare probabilities p(z | C k ) ä Comprehension of asymmetries or distinctions of classes ä Differences between means ä Difference between variabilities ä Segmentation by deformable models ä Probability of geometry p(z) provides prior ä Provides object-intrinsic coord’s in which multiscale image probabilities p(I|z) can be described ä Educational atlas with variability ä Monte Carlo generation of shapes, of diffeo- morphisms, to produce pseudo-patient test images

23. Necessary Analysis Provisions To Achieve Locality & Training Feasibility ä Multiple scales ä Allows few random variables per scale ä At each scale, a level of locality (spatial extent) associated with its random variable ä Positional correspondence ä Across instances ä Between scales Large scale Smaller scale

24. Discussion of Scale ä Spatial aspects of a geometric feature ä Its position ä Its spatial extent ä Region summarized ä Level of detail captured ä Residues from larger scales ä Distances to neighbors with which it has a statistical relationship ä Markov random field ä Cf PDM, spherical harmonics, dense Euclidean positions, landmarks, m-reps Large scale Smaller scale

25. Location Level of Detail Coarse Fine Scale Situations in Various Statistical Geometric Analysis Approaches Location Global coef for Multidetail feature,Detail residues each level of detail, E.g., spher. harm. E.g., boundary pt.E.g., object hierarchy

26. ä Coordinates at one scale must relate to parent coordinates at next larger scale ä Coordinates at one scale must be writable in neighbor’s coordinate system ä Statistically stable features at all scales must be relatable at various scale levels Principles of Object-Intrinsic Coordinates at a Scale Level

27. Figurally Relevant Spatial Scale Levels: Primitives and Neighbors ä Multi-object complex ä Individual object ä = multiple figures ä in geom. rel’n to neighbors ä in relation to complex ä Individual figure ä = mesh of medial atoms ä subfigs in relation to neighbors ä in relation to object ä Figural section ä = multiple figural sections ä each centered at medial atom ä medial atoms in relation to neigbhors ä in relation to figure ä Figural section residue, more finely spaced,.. ä => multiple boundary sections (vertices) ä Boundary section ä vertices in relation to vertex neighbors ä in relation to figural section ä Boundary section more finely spaced,...

28. ä If the total geometric representation z is at all scales or smallest scale, it is not stably trainable with attainable numbers of training cases, so multiscale ä Let z k be the geometric representation at scale level k ä Let z k i be the i th geometric primitive at scale level k ä Let N(z k i ) be the neighbors of z k i (at level k) ä Let P(z k i ) be the parent of z k i (at level k-1) ä Probability via Markov random fields ä p(z k i | P(z k i ), N(z k i ) ) ä Many trainable probabilities ä If p(z k i rel. to P(z k i ), z k i rel. to N(z k i ) ) ä Requires parametrized probabilities Multiscale Probability Leads to Trainable Probabilities

29. Multi-Scale-Level Image Analysis Geometry + Probability ä Multiscale critical for effectiveness with efficiency ä O(number of smallest scale primitives) ä Markov random field probabilistic basis ä Vs. methods working at small scale only or at global scale + small scale only

30. Multi-Scale-Level Image Analysis via M-reps ä Thesis: multi-scale-level image analysis is particularly well supported by representation built around m-reps ä Intuitive, medically relevant scale levels ä Object-based positional and orientational correspondence ä Geometrically well suited to deformation

31. Geometric Typicality Metrics Statistical Metrics

32. Statistics/Probability Aspects : Principal component analysis ä Any shape, x, can be written as x = x mean + Pb + r ä log p(x) = f(b 1, … b t,|r| 2 ) x1x1 x2x2 p1p1 xixi x mean b1b1

33. Visualizing & Measuring Global Deformation c = c mean + z 1  1 p 1 c = c mean + z 2  2 p 2  Shape Measurement  Modes of shape variation across patients ä Measurement = z amount of each mode

34. Statistics/Probability Aspects : Markov random fields ä Suppose z T = (z 1 … z n ) ä p(z i | {z j, j  i}) = p(z i | {z k : k a neighbour of i}) (i. e., assume sparse covariance matrix) ä Need only evaluate O(n) terms to optimize p(z) or p(z | image) ä Can only evaluate p(z i ), i.e., locally ä Interscale; within scale by locality

35. ä If z is at all scales or smallest scale, it is not stably trainable, so multiscale ä Let z k be the geometric rep’n at scale k ä Let z k i be the i th geometric primitive at scale k ä Let N(z k i ) be the neighbors of z k i ä Let P(z k i ) be the parent of z k i ä Let C(z k i ) be the children of z k i ä Probability via Markov random fields ä p(z k i | P(z k i ), N(z k i ), C(z k i ) ) ä Many trainable probabilities ä Requires parametrized probabilities for training Multiscale Geometry and Probability

36. ä z 1 (necessarily global): similarity transform for body section ä z 2 i : similarity transform for the i th object ä Neighbors are adjacent (perhaps abutting) objects ä z 3 i : “similarity” transform for the i th figure of its object in its parent’s figural coordinates ä Neighbors are adjacent (perhaps abutting) figures ä z 4 i : medial atom transform for the i th medial atom ä Neighbors are adjacent medial atoms ä z 5 i : medial atom transform for the i th medial atom residue at finer scale (see next slide) ä z 6 i : boundary offset along medially implied normal for the i th boundary vertex ä Neighbors are adjacent vertices ä Probability via Markov random fields ä p(z k i | P(z k i ), N(z k i ), C(z k i ) ) ä Many trainable probabilities ä Requires parametrized probabilities for training Examples with m-reps components p(z k i | P(z k i ), N(z k i ), C(z k i ) )

37. ä Geometrically  smaller scale ä Interpolate (1st order) finer spacing of atoms ä Residual atom change, i.e., local ä Probability ä At any scale, relates figurally homologous points ä Markov random field relating medial atom with ä its immediate neighbors at that scale ä its parent atom at the next larger scale and the corresponding position ä its children atoms Multiscale Geometry and Probability for a Figure coarse, global coarse resampled fine, local

38. Published Methods of Global Statistical Geometric Characterization in Medicine ä Global variability ä via principal component analysis on ä features globally, e.g., boundary points or landmarks, or ä global features, e.g., spherical harmonic coefficients for boundary ä Global difference ä via linear (or other) discriminant on features ä globally or on global features ä Globally based diagnosis ä via linear (or other) discriminant on features ä globally or on global features ä Example authors: [Bookstein][Golland] [Gerig] [Joshi] [Thompson & Toga][Taylor]

39. Published Methods of Local Statistical Geometric Characterization ä Local variability ä via principal component analysis on features globally or on global features, plus display of local properties of principal component ä Local difference ä via linear (or other) discriminant on global geometric primitives, plus display of local properties of discriminant direction ä On displacement vectors: signed, unsigned re inside/outside ä Example authors: [Gerig] [Golland] [Joshi] [Taylor] [Thompson & Toga] A Outward, p < 0.05 Inward, p < 0.05 p > 0.05 RL Displacement significance: Schizophrenic vs. control hippocampus

40. Shortcomings of Published Methods of Statistical Geometric Characterization ä Unintuitive ä Would like terms like bent, twisted, pimpled, constricted, elongated, extra figure ä Frequently nonlocal or local wrt global template ä Depends on getting correspondence to template correct ä Need where the differences are in object coordinates ä Which object, which figure, where in figure, where on boundary surface ä Requires infeasible number of training cases ä Due to too many random variables (features)

41. Overcoming Shortcomings of Methods of Statistical Geometric Characterization ä Intuitive ä Figural (medial) representation provides terms like bent, twisted, pimpled, constricted, elongated, extra figure ä Local ä Hierarchy by scale level provides appropriate level of locality ä Object & figure based hierarchy yields intuitive locality and good positional correspondences ä Which object, which figure, where in figure, where on boundary surface ä Positional correspondences across training cases & scale levels ä Trainable by feasible number of cases ä Few features in residue between scale levels ä Relative to description at next larger scale level ä Relative to neighbors at same scale level

42. Conclusions re Object Based Image Analysis ä Work at multiple levels of scale ä At each scale use representation appropriate for that scale ä At intermediate scales ä Represent medially  Sense at (implied) boundary Papers at midag.cs.unc.edu/pubs/papers

43. Extensions ä Variable topology ä jump diffusion (local shape) ä level set? ä Active Appearance Models ä shape and intensity ä ‘explaining’ the image ä iterative matching algorithm

44. Recommended Readings ä For deformable sampled boundary models: T Cootes, A Hill, CJ Taylor (1994). Use of active shape models for locating structures in medical images. Image & Vision Computing 12: 355-366. ä For deformable parametrized boundary models: Kelemen, Gerig, et al ä For m-rep based shape: Pizer, Fritsch, et al, IEEE TMI, Oct. 1999 ä For 3D deformable m-reps: Joshi, Pizer, et al, IPMI 2001 (Springer LNCS 2082); Pizer, Joshi, et al, MICCAI 2001 (Springer LNCS 2208)

45. Recommended Readings ä For Procrustes, landmark based deformation (Bookstein), shape space (Kendall): especially understandable in Dryden & Mardia, Statistical Shape Analysis ä For iterative conditional posterior, pixel primitive based shape: Grenander & Miller; Blake; Christensen et al