Technology Project: Shape-Based Retrieval of 3D Craniofacial Data PI: Linda Shapiro, Ph.D. Key Personnel: James Brinkley, M.D., Ph.D. Michael Cunningham, M.D., Ph.D. Collaborators: Carrie Heike, M.D. and Tim Cox, Ph.D. Postdoc: Katarzyna Wilamowska, Ph.D. Postdoc: Indriyati Atmosukarto, Ph.D. RA: Shulin Yang, MS RA: Jia Wu, MS RA: Sara Rolfe, MS Undergrad RA: Michael Lam

Progress on Specific Aims Aim 1: Software Tools for Quantification of Craniofacial Anatomy –New method for learning to compute the plane of symmetry for human faces (paper accepted for the ACM Conference on Bioinformatics, Biology, and Biomedicine) –Landmark-free framework for the detection and description of shape differences in chicken embryos (paper submitted to the IEEE Conference on Engineering in Medicine and Biology) Aim 2: Similarity Measures –Classification and interest-region localization on craniosynostosis skulls (paper accepted for the ACM Conference on Bioinformatics, Biology, and Biomedicine) 2

Aim 3: Organization and Retrieval –Subject database being set up at Seattle Children’s Hospital –De-identified subject database being set up at University of Washington including useful attributes for retrieval (age, gender, race, reason for scan, diagnosis) and pointers to image data files Aim 4: Retrieval System –Modules for 2D Azimuth-Elevation Histogram, Local Features, 2D Longitude-Latitude Signature Map, Pose Normalization, and Automatic Cranial Image Generation delivered to the HUB. –Reference manual has been delivered to the HUB. –Graphical user interface that can use these modules is in progress. –Retrieval system will be built (probably in year 4) to use both the database from Aim 3 and the completed feature extraction and similarity modules. 3

Learning to Compute the Plane of Symmetry for Human Faces 4 We have started to work with 3D mesh data from subjects who have clefts. Faces are no longer expected to be nearly symmetric. Standard pose normalization is not guaranteed to work. Instead, we have developed a method for computing the plane of symmetry using regions about landmarks that are learned from training data.

Methodology 5 1. Use training data head meshes on which experts have marked landmarks 2.Train component detection classifiers to recognize regions (components) surrounding these landmarks using the curvature of the mesh points 3.Using the known plane of symmetry, train component goodness classifiers to determine which detected components are good for computing the plane of symmetry: components that lie on the plane of symmetry component pairs that lie an equal distance from the plane of symmetry

6 4.On independent test data Apply component detection classifiers to find components Apply component goodness classifiers to select those to be used for determining the plane of symmetry 5.Apply the RANSAC algorithm to fit the plane of symmetry to the center points of single components and points halfway between centers of pairs of components, while throwing out outliers. Single Components Pairs Good Computed Components Symmetry Plane

Landmark-Free Framework for the Detection and Description of Shape Differences in Embryos Identify surface in problematic optical projection tomography (OPT) images. Describe changes in shape during embryo development without the use of landmarks. Differentiate normal shape changes from those due to cleft lip/palate defect. 7 Problematic Tomographic Data Reconstructed 3D contour GOALS

Overview of Methodology 8 Low-level feature extraction Mid-level feature extraction Image 2 Image 1 Group similar features ROIs Deformable Registration Flow vectors represent change in shape. The features extracted from the flow vectors are vector magnitude difference from surface normal difference from reference vector local similarity measure local entropy

Feature Cluster Examples 9 Embryo1 Embryo 2 Flow Vectors flow vector flow vector distance flow vector distance magnitude clusters from reference clusters from normal clusters (red = high) (groups of similar angles) (red = similar to normal) Brain Eye Midface Brain Eye Midface

Classification and Interest Region Localization for Craniosynostosis Skulls 10 In prior work, we developed the Cranial Image (CI) representation of skull shape, a matrix of point distances. Under FaceBase support, we developed an automatic procedure for computing the CI, providing a general tool for analysis of craniofacial shape. Our tool allows users to select: how many planes on which to detect points where these planes should be located how many points per plane With 10 planes and 100 points per plane, the CI is too big for many classification/description tasks.

11 We used several forms of machine learning to both classify and quantify head shape and to reduce the number of point pairs required. logistic regression L1-regularized logistic regression fused lasso clustering lasso These machine-learning methods identify the most useful point pairs for classification provide a probability value for each classification that can be used for quantification. Coronal Metopic Sagittal

12 Misclassification rates are shown for each method. Our new clustering lasso method is best overall. The method also allows us to determine the most useful point pairs for classification of each class vs. the other two.

Pairs of Points Useful for Classification 13

The Graphical User Interface in Progress 14