1. 7/12/2004 JHU/IACL Jerry L. Prince Image Analysis and Communications Laboratory Dept. of Electrical and Computer Engineering Johns Hopkins University Cortical Surface Segmentation and Topology

2. 7/12/2004 JHU/IACL Acknowledgments Chenyang Xu Dzung Pham Xiao Han Duygu Tosun Bai Ying Daphne Yu Kirsten Behnke Xiaodong Tao Susan Resnick Mike Kraut Maryam Rettmann Christos Davatzikos Nick Bryan Aaron Carass Ulisses Braga-Neto Funding sources: NSF, NIH/NINDS, NIH/NIA

3. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

4. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

5. 7/12/2004 JHU/IACL Brain Cortex Reconstruction Magnetic Resonance Images (MRI) Cortical Surface

6. 7/12/2004 JHU/IACL Study geometry of cortex –relation to function –changes in aging and disease Use in function mapping –EEG/MEG/PET signals –localization on surface instead of volume Surgical planning –Automatic labels –geometric plan Why Cortex Reconstruction? Extracranial Tissue Cerebrospinal Fluid (CSF) Gray Matter (GM) Outer Pial Surface Central Surface Inner WM/GM Surface White Matter (WM)

7. 7/12/2004 JHU/IACL Nested Surfaces Inner Central Outer

8. 7/12/2004 JHU/IACL Some Difficulties Highly convoluted cortical folds Highly convoluted cortical folds Image noise Image noise Image intensity inhomogeneity Image intensity inhomogeneity Partial volume effect Partial volume effect

9. 7/12/2004 JHU/IACL Some Requirements Topology correctness Valid 2D manifold X X

10. 7/12/2004 JHU/IACL Four Steps 1.Fuzzy classification 2.Nested surface segmentation 3.Spherical mapping and partial inflation 4.Sulcal segmentation

11. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

12. 7/12/2004 JHU/IACL Preprocessing

13. 7/12/2004 JHU/IACL Fuzzy Segmentation [Pham & Prince TMI 1999] Gray matterWhite matterCerebrospinal fluid Yields continuous-valued fuzzy membership functions, with values in the range of [0, 1]

14. 7/12/2004 JHU/IACL Published Algorithms AFCM: Adaptive fuzzy c-means –smooth gain field; fuzzy clusters; yields pseudo partial volume segmentation AGEM: Adaptive generalized Expectation Maximization –smooth gain field; MRF label smoothness; posterior density is “fuzzy segmentation FANTASM –Fuzzy segmentation with smooth membership functions and gain field Pham and Prince

15. 7/12/2004 JHU/IACL Membership Improvements White Matter –Modifications to fill interior, remove extraneous surfaces, remove connectivity errors, and correct topology Gray Matter –Modification to provide evidence of CSF in tight sulci

16. 7/12/2004 JHU/IACL WM Isosurface Approximates WM/GM boundary Problems: –undesired surfaces –connectivity errors –handles

17. 7/12/2004 JHU/IACL Autofill WM isosurface should represent the GM/WM interface of the cortex only isosurface of WM segmentation before filling isosurface of WM segmentation after filling

18. 7/12/2004 JHU/IACL Autofill WM Volume

19. 7/12/2004 JHU/IACL WM Isosurface Principle 0.5 of WM membership approximates WM/GM interface 0.5 of WM+GM membership approximates GM/CSF interface 0.5 WMGMCSF

20. 7/12/2004 JHU/IACL Marching Cubes Isosurface Consider values on corners of voxel Label as –above isovalue –below isovalue Determine position of triangular mesh surface passing through voxel Linear interpolation > 0.5 < 0.5 Voxel values

21. 7/12/2004 JHU/IACL Connectivity Errors Multiple meshes – select the largest mesh Touching vertices, edges, and faces –isovalue choice, or –adjust pixel values by epsilon Ambiguous faces and cubes –use saddle point methods, or –use connectivity consistent MC algorithm Most isosurface algorithms use rules that lead to connectivity errors

22. 7/12/2004 JHU/IACL Ambiguous Faces Two possible tilings:

23. 7/12/2004 JHU/IACL Ambiguous Cubes Two possible tilings:

24. 7/12/2004 JHU/IACL Digital Connectivity Consistent pairs: (foreground,background) → (6,18), (6,26), (18,6), (26,6) 6-connectivity 18-connectivity 26-connectivity

25. 7/12/2004 JHU/IACL Connectivity Consistent MC Algorithm (black,white) (18,6)  choose b, f (26,6)  choose b, e Ambiguous Face Ambiguous Cube (6,18)  choose c, f (6,26)  choose c, f

26. 7/12/2004 JHU/IACL Remaining Problem: Handles multiple surfaces shared vertices shared edges shared faces connectivity errors handles Taken from actual white matter

27. 7/12/2004 JHU/IACL Removes Handles by Editing WM Fill the backgroundCut the foreground OR

28. 7/12/2004 JHU/IACL Euler Number –Euler number of a triangular mesh: –A simple closed surface is topologically equivalent to a sphere iff –genus is handle tunnel A surface handle Illustration Handles: easy to detect by computing the Euler number of the surface mesh Euler number provides no information about the location of the handles

29. 7/12/2004 JHU/IACL GTCA Flow Diagram Recycling Illustration of our topology correction filter

30. 7/12/2004 JHU/IACL 1 2 3 4 5 6 7 8 Morphological Opening structuring element “body” “residue”

31. 7/12/2004 JHU/IACL After Opening Divides object into two components: –“body” –“residue” Build graph? Throw out residue pieces? NO! –residue are often very large, but thin sheets –opening may create holes that did not exist before

32. 7/12/2004 JHU/IACL Conditional Topological Expansion Grow body by adding “nice” points from residue: prohibits creation of handles; allows filling of holes

33. 7/12/2004 JHU/IACL Build a Graph 1 2 3 4 5 6 7 1 2 3 4 5 6 7 connected components connectivity

34. 7/12/2004 JHU/IACL Detect and Remove Cycles Find a cycle using depth- first search Find the smallest residue connected component in the cycle and remove it Repeat until no more cycles remain 1 2 3 4 5 6 7 X X

35. 7/12/2004 JHU/IACL Restore Residue Add remaining residue connected components back to body Run conditional topological expansion again. –restores some points that were discarded prior to graph construction.

36. 7/12/2004 JHU/IACL Success? Compute isosurface of binary volume Compute Euler number –If less than 2; repeat on background Compute Euler number again –If less than 2; repeat with larger structuring element, and so on… Is isosurface algorithm consistent with digital topology? –wrong algorithm  connectivity paradoxes

37. 7/12/2004 JHU/IACL Topology Correction: Result Before Topology CorrectionAfter Topology Correction ¹ WM ^

38. 7/12/2004 JHU/IACL Results: Quantitative Ratio of voxels changed to original genus is around 2 Genus of resulting volume. Number of voxels changed in volume.

39. 7/12/2004 JHU/IACL GM/WM Interface Topologically correct No self intersections Sub-voxel resolution Close to –WM/GM surface –GM central surface –pial surface Represented by –triangle mesh, or –level set function

40. 7/12/2004 JHU/IACL Gray Matter Isosurface Misses tight sulci

41. 7/12/2004 JHU/IACL Partial Volume Effect Imaging GM CSF partial volume averaging WM GMCSF WM Gyri Sulci

42. 7/12/2004 JHU/IACL Weighted Distance Skeleton Distance function from the GM/WM interface Compute its Laplacian and normalize to

43. 7/12/2004 JHU/IACL Anatomically Consistent Enhancement (ACE) if Outside ^ ^

44. 7/12/2004 JHU/IACL ACE Result Original GMACE GM

45. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

46. 7/12/2004 JHU/IACL Deformable Surface Model Want to move the initial WM/GM mesh

47. 7/12/2004 JHU/IACL Nested Deformable Surfaces Pial Surface Inner Surface Central Surface TGDM-3 Initial WM Isosurface TGDM-2TGDM-1

48. 7/12/2004 JHU/IACL Parametric deformable models (PDMs) ─ Represent curves or surfaces through explicit parameterization ─ e.g. curves tessellated with nodes, surfaces tessellated with triangles Geometric deformable models (GDMs) – Implicit implementation – uses level set numerical method Deformable Models

49. 7/12/2004 JHU/IACL Parametric Deformable Models p = location on contour [Kass, Witkin, & Terzopolous, 1987] Curves/surfaces that deform with a speed law derived from image information and prior knowledge about object shape (e.g. boundary smoothness and continuity)Curves/surfaces that deform with a speed law derived from image information and prior knowledge about object shape (e.g. boundary smoothness and continuity)

50. 7/12/2004 JHU/IACL x y One Extra Dimension z x y Level Set Method [Osher and Sethian 1988]

51. 7/12/2004 JHU/IACL Advantages of GDMs Produce closed, non-self-intersecting contours Independent of contour parameterization Easy to implement: numerical solution of PDEs on regular computational grid Stable computation

52. 7/12/2004 JHU/IACL Parametric to Geometric [Osher & Sethian 1988] Level Set PDE: Contour Deformation:

53. 7/12/2004 JHU/IACL Topology Behavior of Deformable Contour Models Parametric  self intersection problem Geometric  cannot control topology TGDM (ours)  preserves topology ParametricGeometric TGDM

54. 7/12/2004 JHU/IACL Digital Embedding of Contour Topology White Points: Black Points: Contour topology is determined by signs of the level set function at pixel locations Topology of the implicit contour is the same as the topology of the digital object

55. 7/12/2004 JHU/IACL Connectivity Rule of Contour Topology of digital contour determined by connectivity rule Same digital object, different topologies

56. 7/12/2004 JHU/IACL Topology Preservation Principle Preserving contour topology is equivalent to maintaining the topology of the digital object The digital object can only change topology when the level set function changes sign at a grid point Which sign changes can be allowed, and which cannot? To prevent the digital object from changing topology, the level set function should only be allowed to change sign at simple points

57. 7/12/2004 JHU/IACL Simple Point Definition: a point is simple if adding or removing the point from a binary object will not change the object topology Determination: can be characterized locally by the configuration of its neighborhood (8- in 2D, 26- in 3D) [Bertrand & Malandain 1994] Simple Non- Simple

58. 7/12/2004 JHU/IACL x is a Simple Point x xx

59. 7/12/2004 JHU/IACL x is Not a Simple Point X X

60. 7/12/2004 JHU/IACL Topology Preserving Geometric Deformable Model (TGDM) Evolve level set function according to GDM If level set function is going to change sign, check whether the point is a simple point –If simple, permit the sign-change –If not simple, prohibit the sign-change (replace the grid value by epsilon with same sign) –(Roughly, this step adds 7% computation time.) Extract the final contour using a connectivity consistent isocontour algorithm

61. 7/12/2004 JHU/IACL SGDMTGDM A 2D Demonstration

62. 7/12/2004 JHU/IACL PDM ResultTGDM Result No Self-intersections

63. 7/12/2004 JHU/IACL A 3D TGDM Demonstration Original Object SGDM Init #1 #1 #2 SGDM Init #2 TDGM Init #1 TDGM Init #2

64. 7/12/2004 JHU/IACL TGDM for Inner Surface Initial WM Isosurface Final GM/WM Interface

65. 7/12/2004 JHU/IACL TGDM for Inner Surface Evolution Equation Mean Curvature: and are weighting factors Region Force:

66. 7/12/2004 JHU/IACL TGDM for Central Surface Initialize with GM/WM surface Final Central Surface

67. 7/12/2004 JHU/IACL TGDM for Central Surface Gradient Vector Flow [Xu & Prince TIP98]

68. 7/12/2004 JHU/IACL TGDM for Central Surface Mean Curvature: Gradient Vector Flow Force:, and are weighting factors Region Force:  Evolution Equation

69. 7/12/2004 JHU/IACL Nesting Constraint Nested surfaces: –Central is outside GM/WM –Pial is outside central If level set function wants to go negative to positive –allow if inner level set function is positive –otherwise set to small positive epsilon

70. 7/12/2004 JHU/IACL TGDM for Outer Surface Final Pial SurfaceStart from Central Surface

71. 7/12/2004 JHU/IACL TGDM for Outer Surface Evolution Equation Region Force: Mean Curvature: Gradient Vector Flow Force:, and are weighting factors

72. 7/12/2004 JHU/IACL Coronal Results Visual Inspection Sagittal Slice views of three surfaces overlaid on cross-sections of the original image Axial

73. 7/12/2004 JHU/IACL Repeatability Analysis 3 subjects, each scanned twice Surface pairs rigidly registered Average errors: –signed distance –absolute distance

74. 7/12/2004 JHU/IACL Repeatability Results (mm)

75. 7/12/2004 JHU/IACL Landmark Validation Study

76. 7/12/2004 JHU/IACL Landmark Validation Analysis Raters: 12 Brains: 2 Landmarks: 10 per region Sulci: 33 / brain Geometry: 11 fundi, 11 gyri, 11 banks Surface: Inner & Pial Statistical software: “R” version 1.8.1 CRUISE surfaces are reference surfaces: yield “landmark offset” –signed and absolute Membership values –white matter –gray matter Statistical factors: –Brain –Geometry –Sulci

77. 7/12/2004 JHU/IACL Landmark Validation: Results MANOVA revealed significant factors: –geometry & sulci, but not brain Landmark offset –mean = - 0.35 mm –std = 0.65 mm –16% farther than 1 mm from reference ACE regions show smaller offsets Signed distance consistently negative outward bias of CRUISE –differs for geometry (largest for fundi) –differs for surface Note: we are optimizing parameters

78. 7/12/2004 JHU/IACL Nested Surface Segmentation Nearly fully automated –skull-stripping is semi-automated (10 minutes) –AC & PC need to be picked manually (5 minutes) –The rest is fully automated Less than 25 minutes for each brain –(Previous PDM version takes 2-3 hours) More than 200 brain datasets processed so far –average error is about 1/3 voxel –highly repeatable  scanner errors dominate Han et al, 2004

79. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

80. 7/12/2004 JHU/IACL Spherical and Partial Flattening [Tosun et al, 2003]

81. 7/12/2004 JHU/IACL Surface Inflation Coarsen shape More regular mesh structure Use relaxation operator: Check norm of mean curvature:

82. 7/12/2004 JHU/IACL Atlas Registration Simpler surface registered using modified ICP Atlas labels transfer easily Atlas Subject (a) (b) (c) (d)

83. 7/12/2004 JHU/IACL Spherical Mapping Single conformal map from atlas Inverse stereographic projection

84. 7/12/2004 JHU/IACL Automatic Labelling Brains mapped to sphere Segmented sulci compared to labelled atlas Simple voting scheme leads to >90% accuracy

85. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

86. 7/12/2004 JHU/IACL Sulcal Segmentation Goals: Automatically segment sulci carry out cortical parcellation Applications: Localizing activation sites in functional images Brain registration Understanding morphological changes in normal aging and disease Principle: Based on depth from “outer” surface

87. 7/12/2004 JHU/IACL Sulcal Regions Defined as buried cortical regions that surround sulcal spaces

88. 7/12/2004 JHU/IACL Classifying Gyral and Sulcal Regions Generate a shrink-wrap surface Sulcal regions distinguished from gyral regions based on distance to shrink-wrap surface

89. 7/12/2004 JHU/IACL Sulcal/Gyral Classification sulcal regions (red) and gyral regions (blue) Euclidean distance to outer surface sulci > 2 mm from outer surface

90. 7/12/2004 JHU/IACL Watershed Segmentation Classification does not separate sulci Further segmentation is required Watershed by immersion is intuitive idea:

91. 7/12/2004 JHU/IACL Geodesic Distance Computation use Fast Marching (Kimmel and Sethian, ’98) initial contour at time zero is gyral/sulcal boundary Propagation at unit speed in normal direction on mesh geodesic distance is arrival time of evolving contour

92. 7/12/2004 JHU/IACL Watershed Computation Each local minimum produces a catchment basin (CB). Critique: true sulci are separated single sulci are over-segmented.

93. 7/12/2004 JHU/IACL Merging Algorithm Addresses over-segmentation problem Small ridges in sulcal regions result in formation of separate CBs Criterion for merging CBs: 1) height of ridge 2) size of CB Provides different “levels” of merging

94. 7/12/2004 JHU/IACL Sulcal Segmentation Results Height threshold = 1 cm Size threshold = 3 cm 2 Rettmann et al. MMBIA 2000

95. 7/12/2004 JHU/IACL Sulcal Segmentation Results

96. 7/12/2004 JHU/IACL Cross-Sections

97. 7/12/2004 JHU/IACL

98. 7/12/2004 JHU/IACL Outline Introduction Fuzzy Classification Nested Surface Segmentation Spherical Mapping and Partial Inflation Sulcal Segmentation Applications

99. 7/12/2004 JHU/IACL Repeat Scan Validation Superior frontal sulcus scan 1scan 2scan 3

100. 7/12/2004 JHU/IACL Shape Analysis Left Right Cingulate Subject 1Subject 2

101. 7/12/2004 JHU/IACL Geometric Features mean curvature geodesic depth

102. 7/12/2004 JHU/IACL Cortical Thickness [Yezzi et al, 2003]

103. 7/12/2004 JHU/IACL Baltimore Longitudinal Study of Aging PI: Susan Resnick (NIA) 1994-2003 Ages 55-85, 158 participants >1000 separate scans, 1 per year per subject volumetric SPGR brain scans 0.9375x0.9375x1.5mm voxel size

104. 7/12/2004 JHU/IACL Thickness Map from CRUISE Typical Thickness Map

105. 7/12/2004 JHU/IACL Cross-sectional Study of Cortical Thickness Preliminary study on 35 subjects

106. 7/12/2004 JHU/IACL The END