Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Statistics of Anatomic Geometry: Information Theory and Automatic Model Building Carole Twining Imaging Science and Biomedical Engineering (ISBE) University of Manchester, UK Contributions from: Rhodri Davies, Stephen Marsland, Tim Cootes, Vlad Petrovic, Roy Schestowitz, & Chris Taylor

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 2 Overview Recap of Point Distribution/Statistical Shape Models PDMs/SSMs ● Correspondence Problem: Shape Representation & Correspondence Correspondence & Statistics Methods for establishing correspondence ● Automatic Methods for Groupwise Shape Correspondence Manipulating Correspondence not Shape Minimum Description Length objective function Optimisation ● Extension to Images : MDL Groupwise Registration automatic models from unannotated image sets ● Model Evaluation Criteria

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 3 Point Distribution Models (PDMs) Statistical Shape Models (SSMs) Set of Shapes & Corresponding Points Shape Space PCA Model PDF

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 4 Adding Image Information Shape Space Shape & Appearance Space

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 5 ● Include image information from whole region ● Correlation between shape & texture Adding Image Information Shape Model Shape & Texture Model

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 6 Active Shape & Appearance Models ASM Search AAM Search

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry The Correspondence Problem

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 8 Shape Representation & Correspondence ● Non-Local Representations Fourier descriptors (e.g., SPHARM) Medial descriptors (e.g., MREPS) ● Local Representations Point based (e.g., PDMs/SSMs) ● Common Representation of training set => Correspondence Non-local tends to give implicit correspondence Point based gives explicit correspondence ● Why does the correspondence matter?

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 9 Correspondence & Statistics Shape Space Varying correspondence varies the shape statistics

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 10 Establishing Correspondence ● Manual landmarking ● Arbitrary parameterisations Kelemen, Hill, Baumberg & Hogg ● Shape features Wang, Brett ● Image registration models from deformation field Christensen, Joshi, Lavalle, Reuckert, Twining

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 11 Manual Methods for Correspondence ● Manual Landmarks Interpolate for dense correspondence May need to adjust ● Problems: Time-consuming Subjective Requires expert anatomical knowledge Very difficult in 3D

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 12 Arc-Length Parameterisation ●Equally-space landmarks around each shape (Baumberg & Hogg)

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 13 Shape Features ● e.g. Curvature-based methods ● Intuitive ● But: What about regions without such features? Not really groupwise, since depends on local properties of each shape Is it really the best correspondence?

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Automatic Groupwise Correspondence

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 15 Automatic Groupwise Correspondence Desirable features: ● Groupwise: Depends on whole set of shapes ● Automatic – little or no user intervention ● 2D & 3D ● Runs in reasonable time!

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 16 Automatic Groupwise Correspondence Optimisation Problem Framework: ● Method of manipulating correspondence: 2D & 3D ● Objective function: quantifies the ‘quality’ of the correspondence ● Optimization Scheme

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Manipulating Correspondence

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 18 Manipulating Correspondence ● Point-to-Point: Shape 1Shape 2 Shape Points Correspondence Points Varying correspondence varies shape! Vary correspondence but not shape!

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 19 Manipulating Correspondence ● Continuous parameterisation of shape ● Re-parameterising varies correspondence

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 20 ● Generalises to 3D ● Map surface to parameter sphere - no folds or tears ● Varying parameterisation on sphere Manipulating Correspondence ShapeSphere & Spherical Polar coordinates

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Objective Function

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 22 Objective Function ● Varying Correspondence = Varying Statistics ● Objective function based on model probability density function number of model modes compactness quality of fit to training data number of model parameters Shape Space

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 23 Shape Space MDL Objective Function ● Transmit training set as encoded binary message ● Shannon: Set of possible events {i} with probabilities {p i } Optimal codeword length for event i: -log p i ● Encode whole training set of shapes: Encoded Model: mean shape, model modes etc Reconstruct shape space and model pdf Each training shape: p i from model pdf Reconstruct all training shapes ● MDL Objective function = total length of message

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 24 MDL Objective Function ● Fit between model pdf and training data: Probabilities for training points => better the fit, shorter the message ● Too complex a model: model parameter term large ● Too few modes: Bad fit to data & large residual ● Badly chosen modes: Bad fit to data

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 25 Optimisation ● Genetic algorithm search (Davies et al, 2002) All parameters optimised simultaneously Slow, scales badly with no of examples ● More recent, multi-scale, multi-resolution approaches: better convergence fast enough for routine use scales approximately linearly with no of examples (Davies et al, IPMI 2003)

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 26 Results ● Quantitatively better results compared to SPHARM ● Differences tend to be subtle ● Comparing techniques, have to go beyond visual inspection (see section on Model Evaluation Criteria)

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry MDL Groupwise Image Registration

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 28 Image & Shape Correspondence ● Groups of Shapes: groupwise dense correspondence statistical models of shape variability analysis of variation across & between populations assist in analysing unseen examples (ASM & AAM) ● Groups of Images: groupwise dense correspondence = groupwise registration statistical models of shape & appearance as above ● MDL technique for correspondence can be applied to both (Twining et al 2005)

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 29 ● Spatial Correspondence between images Shape variation ● Warp one to another Difference is texture variation ● Repeat across group => Appearance model of image set Image Registration

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 30 Groupwise Image Registration ● MDL Objective Function Combined shape & texture model ● Define dense correspondence triangulated points on each image & interpolate ● Manipulate Correspondence ● Increase resolution of mesh & repeat

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 31 Results ● 104 2D brain slices ● Appearance Model

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Model Evaluation Criteria

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 33 Model Evaluation Criteria ● Need to go beyond visual inspection, subtle differences ● Generalisability: the ability to represent unseen shapes/images which belong to the same class as those in the training set ● Specificity: the ability to only represent images similar to those seen in the training set ● Quantitative comparison of models

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 34 General but not Specific Specificity and Generalization Specific but not General Training Set: Sample Set from model pdf: Space of Shapes/Images

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 35 Specificity Training Set Sample Set :distance on image/shape space

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 36 Generalisation Ability Sample Set Training Set

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 37 Validation ● Annotated/Registered Data ● Perturb Registration Generalisation Specificity Size of Perturbation Objective function

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 38 Evaluating Brain Appearance Models

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 39 Summary ● Manipulating Correspondence Shown to produce quantitatively better models Large-scale Optimisation problem - so far, only linear models Extension to other shape representation methods (e.g. MREPS) Topology – manipulate parameter space: simple, fixed topology Multi-part objects Differences tend to be subtle - go beyond visual inspection of results Model evaluation criteria Extension to groupwise image registration

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Questions?

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 41 Resources & References AAMs, ASMs ● [1] T. F. Cootes, G. J. Edwards, and C. J. Taylor, Active appearance models, IEEE Trans. Pattern Anal. Machine Intell., vol. 23, no. 6, pp , ● [2] T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham, Active shape models – their training and application, Computer Vision and Image Understanding, 61(1), 38-59, 1995 ● [3] T. F. Cootes, A. Hill, C. J. Taylor, and J. Haslam, The use of active shape models for locating structures in medical images, Image and Vision Computing, vol. 12, no. 6, pp , July ● [4] B. van Ginneken, A.F.Frangi, J.J.Stall, and B. ter Haar Romeny, Active shape model segmentation with optimal features, IEEE Trans. Med. Imag., vol. 21, pp , ● [5] P. Smyth, C. Taylor, and J. Adams, Vertebral shape: Automatic measurement with active shape models, Radiology, vol. 211, no. 2, pp , ● [6] N. Duta and M. Sonka, Segmentation and interpretation of MR brain images: An improved active shape model, IEEE Trans. Med. Imag., vol. 17, pp , Further references, as well as notes on the historical meanderings in the development of these techniques can be found on Tim Cootes’ website:

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 42 Resources & References MREPS ● [7] S. M. Pizer, D. Eberly, D. S. Fritsch, and B. S. Morse, Zoom-invariant vision of figural shape: The mathematics of cores, Computer Vision and Image Understanding, vol. 69, no. 1, pp , Fourier descriptors, spherical harmonics & SPHARM ● [8] C. Brechb¨uhler, G. Gerig, and O. Kubler, Parameterisation of closed surfaces for 3D shape description, Computer Vision, Graphics and Image Processing, vol. 61, pp , ● [9] A. Kelemen, G. Szekely, and G. Gerig, Elastic model-based segmentation of 3D neurological data sets, IEEE Trans. Med. Imag., vol. 18, no. 10, pp , Oct ● [10] C. Brechb¨uhler, G. Gerig, and O. K uhler, Parametrization of closed surfaces for 3D shape description, Computer Vision and Image Understanding, vol. 61, no. 2, pp , ● [11] G. Szekely, A. Kelemen, C. Brechbuhler, and G. Gerig, Segmentation of 2-D and 3-D objects from MRI volume data using constrained elastic deformations of flexible fourier contour and surface models, Medical Image Analysis, vol. 1, pp , 1996.

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 43 Resources & References Fourier descriptors, spherical harmonics & SPHARM ● [12] D. Meier and E. Fisher, Parameter space warping: Shape-based correspondence between morphologically different objects, IEEE Trans. Med. Imag., vol. 21, no. 1, pp , Jan ● [13] M. Styner, J. Liberman, and G. Gerig, Boundary and medial shape analysis of the hippocampus in schizophrenia, in Proc. International Conference on Medical Image Computing and Computer Aided Intervention (MICCAI), 2003, pp Feature-Based Shape correspondence ● [14] A. D. Brett, A. Hill, and C. J. Taylor, A method of automatic landmark generation for automated 3D PDM construction, Image and Vision Computing, vol. 18, pp , ● [15] Y. Wang, B. S. Peterson, and L. H. Staib, Shape-based 3D surface correspondence using geodesics and local geometry, in Proc. IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2000, pp ● [16] G. Subsol, J. Thirion, and N. Ayache, A scheme for automatically building three-dimensional morphometric anatomical atlases: application to a skull atlas, Medical Image Analysis, vol. 2, no. 1, pp , 1998.

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 44 Resources & References Elastic and Distortion based methods of shape correspondence ● [17] M. Kaus, V. Pekar, C. Lorenz, R. Truyen, S. Lobregt, and J. Weese, Automated 3-D PDM construction from segmented images using deformable models, IEEE Trans. Med. Imag., vol. 22, no. 8, pp , Aug ● [18] C. Shelton, Morphable surface models, International Journal of Computer Vision, vol. 38, pp , ● [19] S. Sclaroff and A. P. Pentland, Modal matching for correspondence and recognition, IEEE Trans. Pattern Anal. Machine Intell., vol. 17, no. 6, pp , ● [20] F. L. Bookstein, Landmark methods for forms without landmarks: morphometrics of group differences in outline shape, Medical Image Analysis, vol. 1, no. 3, pp , Minimum Description Length This is the information theory stuff behind MDL. ● [21] J. Rissanen, Lectures on Statistical Modeling Theory, ● [22] J. Rissanen, Stochastic Complexity in Statistical Inquiry, World Scientific Press, 1989.

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 45 Resources & References MDL for Shape Correspondence Approximate MDL Note that the freely available code distributed by Thodberg is only approximate MDL, not full state-ofthe- art MDL as used by other groups. In fact, the objective function used in these papers is equivalent to what is used to initialise other algorithms. This fact has caused a little confusion in the literature. ● [23] H. Thodberg, MDL shape and appearance models, in Proc. 18th Conference on Information Processing in Medical Imaging (IPMI), 2003, pp ● [24] H. Thodberg and H. Olafsdottir, Adding curvature to MDL shape models, in Proc. 14th British Machine Vision Conference (BMVC), vol. 2, 2003, pp ● [25] T. Heimann, I. Wolf, T. G. Williams, and H.-P. Meinzer, 3D Active Shape Models Using Gradient Descent Optimization of Description Length, IPMI MDL for 2D Shape This uses the initial genetic algorithm search, which was later improved upon. ● [26] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor, A minimum description length approach to statistical shape modelling, IEEE Trans. Med. Imag., vol. 21, no. 5, pp , May ● [27] R. H. Davies, C. J. Twining, P. D. Allen, T. F. Cootes, and C. J. Taylor, Building optimal 2D statistical shape models, Image and Vision Computing, vol. 21, pp , 2003.

Combining the strengths of UMIST and The Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry Slide 46 Resources & References MDL for 3D Shape ● [28] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor, 3D statistical shape models using direct optimisation of description length, in Proc. 7th European Conference on Computer Vision (ECCV), 2002, pp MDL for Image Registration ● [29] C. J. Twining, T. Cootes, S. Marsland, V. Petrovic, R. Schestowitz, and C. J. Taylor, A Unified Information-Theoretic Approach to Groupwise Non-Rigid Registration and Model Building, Presented at IPMI 2005 ● [30] C. J. Twining, S. Marsland, and C. J. Taylor, Groupwise Non-Rigid Registration: The Minimum Description Length Approach, In Proceedings of BMVC ● [31] C.J. Twining and S. Marsland, A Unified Information-Theoretic Approach to the Correspondence Problem in Image Registration, International Conference on Pattern Recognition (ICPR), Cambridge, U.K