1. OPAL Workflow: Model Generation Tricia Pang February 10, 2009

2. OPAL Workflow, 10 Feb 20092 Overview Motivation OPAL Approach 5-Stage OPAL Workflow Challenges Future directions

3. OPAL Workflow, 10 Feb 20093 Motivation ArtiSynth [1]: 3D Biomechanical Modeling Toolkit Ideally: Model derived from single subject source High resolution model

4. OPAL Workflow, 10 Feb 20094 Motivation Obstructed sleep apnea (OSA) disorder Caused by collapse of soft tissue walls in airway Ideally: Ability to run patient- specific simulations to help diagnosis Quick and accurate method of generating model Credit: Wikipedia

5. OPAL Workflow, 10 Feb 20095 OPAL Project Dynamic Modeling of the Oral, Pharyngeal and Laryngeal (OPAL) Complex for Biomedical Engineering Patient-specific modeling and model simulation for study of OSA Tools for clinician use in segmenting image and importing to ArtiSynth Come up with protocol, tools/techniques and modifications needed for end-to-end process

6. OPAL Workflow, 10 Feb 20096 OPAL Project 3D Medical DataBiomechanical Model

7. OPAL Workflow, 10 Feb 20097 Workflow Stages 1. Imaging 2. Image processing & reconstruction 3. Reference model generation 4. Patient-specific model fitting 5. Biomechanical model

8. OPAL Workflow, 10 Feb 20098 Workflow Stages 1. Imaging 2. Image processing & reconstruction 3. Reference model generation 4. Patient-specific model fitting 5. Biomechanical model

9. OPAL Workflow, 10 Feb 20099 Stage 1: Imaging Structures Tongue Soft palate Hard palate Epiglottis Pharyngeal wall Airway Jaw Teeth

10. OPAL Workflow, 10 Feb 200910 Data Source MRI Credit: Klearway, Inc. Dental Appliance w/ Markers Cone CT of Dental Cast Other: laser scans, planar/full CT scans, tagged MRI, ultrasound, fluoroscopy, cadaver data…

11. OPAL Workflow, 10 Feb 200911 MRI & Protocol Normal subject vs. OSA patients Control vs. treatment (appliance)

12. OPAL Workflow, 10 Feb 200912 Workflow Stages 1. Imaging 2. Image processing & reconstruction 3. Reference model generation 4. Patient-specific model fitting 5. Biomechanical model

13. OPAL Workflow, 10 Feb 200913 Stage 2: Image processing & Reconstruction N3 correction [2] (Non-parametric non-uniform intensity normalization) Cropping Cubic interpolation Image registration & reconstruction (Bruno’s work) Combining 3 data sets → high-quality data set

14. OPAL Workflow, 10 Feb 200914 Workflow Stages 1. Imaging 2. Image processing & reconstruction 3. Reference model generation 4. Patient-specific model fitting 5. Biomechanical model

15. OPAL Workflow, 10 Feb 200915 Stage 3: Reference Model Generation Goal: High quality model Focus on bottom-up semi-automatic segmentation approaches eg. Livewire [3]

16. OPAL Workflow, 10 Feb 200916 3D Livewire Seed points (forming contours) drawn in 2 orthogonal slice directions, and seed points automatically generated in third slice direction

17. OPAL Workflow, 10 Feb 200917 Livewire Model Refinement Morphological operations Contour smoothening (active contours [4]) 3D surface reconstruction (non-parallel curve networks [5]) (Claudine & Tanaya)

18. OPAL Workflow, 10 Feb 200918 Workflow Stages 1. Imaging 2. Image processing & reconstruction 3. Reference model generation 4. Patient-specific model fitting 5. Biomechanical model

19. OPAL Workflow, 10 Feb 200919 Stage 4: Patient-Specific Model Generation Goal: Accurate model, generated with minimal user interaction Focus on top-down or automated approaches Morphological warping operations Deformable model crawlers

20. OPAL Workflow, 10 Feb 200920 Thin-Plate Spline Warping Thin-plate spline (TPS) deformation [6]: interpolating surfaces over a set of landmarks based on linear and affine-free local deformation Reference Model Warp Result Warp field

21. OPAL Workflow, 10 Feb 200921 TPS Warping, Phase 1 Patient MRI Reference Model List of corresponding points User selects a point on both patient MRI and reference model Hard to pinpoint landmarks on 3D model

22. OPAL Workflow, 10 Feb 200922 TPS Warping, Phase 2 Reference MRI (has a pre-built 3D model) Patient MRI Predefined landmarks shown on reference MRI, user selects equivalent point on patient MRI Can be improved by automated point-matching

23. OPAL Workflow, 10 Feb 200923 Chan-Vese Active Contours Highly automated method Combine 2D segmentation of axial slices in Matlab User-indicated start point Iterate sequentially using previous segmentation as starting contour for Chan- Vese active contours [7] Livewire 3D (~2 hours) Livewire + post processing Automated AC on axial (2 minutes)

24. OPAL Workflow, 10 Feb 200924 Deformable Organism Crawler Automatically segment airway by growing a tubular organism, guided by image data and a priori anatomical knowledge Developed in I-DO toolkit [8] Advantages: Analysis and labeling capabilities Ability to incorporate shape-based prior knowledge Modular hierarchical development framework

25. OPAL Workflow, 10 Feb 200925 Workflow Stages 1. Imaging 2. Image processing & reconstruction 3. Reference model generation 4. Patient-specific model fitting 5. Biomechanical model

26. OPAL Workflow, 10 Feb 200926 Stage 5: Biomechanical Model Import surface mesh into ArtiSynth Work in progress Challenges: Determining “rest” position from inverse modeling Defining interior nodes and muscle end points

27. OPAL Workflow, 10 Feb 200927 Challenges in Segmentation Medical image data quality Bottom-up methods: Need for general procedure and abstraction from anatomy being segmented Top-down methods: Need good atlas model Validation with gold standard segmentation

28. OPAL Workflow, 10 Feb 200928 Future Directions in Segmentation Deformable organism crawler Automated morphing of reference model into patient model Additions to Livewire Oblique slices Sub-pixel resolution Convert to graphics implementation Add smoothness by regularization (eg. by spline, a priori model, …)

29. OPAL Workflow, 10 Feb 200929 Thank you! Questions?

30. OPAL Workflow, 10 Feb 200930 References [1] Fels, S., Vogt, F., van den Doel, K., Lloyd, J., Stavness, I., and Vatikiotis-Bateson, E. Developing Physically-Based, Dynamic Vocal Tract Models using ArtiSynth. Proc. Int. Seminar Speech Production (2006), 419-426. [2] Sled, G., Zijdenbos, A. P., and Evans, A. C. Non-parametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans. in Medical Imaging 17, 1 (1998), 87-97. [3] Poon, M., Hamarneh, G., and Abugharbieh, R. Effcient interactive 3d livewire segmentation of complex objects with arbitrary topology. Comput. Med Imaging and Graphics (2009), in press. [4] Hamarneh, G., Chodorowski, A., and Gustavsson, T. Active Contour Models: Application to Oral Lesion Detection in Color Images. IEEE International Conference on Systems, Man, and Cybernetics 4 (2000), 2458 -2463. [5] Liu, L., Bajaj, C., Deasy, J. O., Low, D. A., and Ju, T. Surface reconstruction from non-parallel curve networks. Eurographics 27, 2 (2008), 155-163. [6] Bookstein, F. L. Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 11, 6 (1989), 567-585. [7] Chan, T., and Vese, L. Active contours without edges. IEEE Transactions on Image Processing 10, 2 (2001), 266-277. [8] McIntosh, C. and Hamarneh, G. I-DO: A “Deformable Organisms” framework for ITK. Medical Image Analysis Lab, SFU. Release 0.50.