NA-MIC National Alliance for Medical Image Computing 3D Slicer & NA-MIC: Overview and Applications Steve Pieper, PhD

National Alliance for Medical Image Computing 2 Acknowledgments F. Jolesz, R. Kikinis, C. Tempany, P. Black, S. Wells, CF. Westin, M. Halle, N. Hata, T. Kapur, A.Tannenbaum, M. Shenton, E. Grimson, P.Golland, W.Schroeder, J. Miller, N. Aucoin, K. Hayes, S. Barre, W. Plesniak, R. Gollub, S. Pujol and many more….

National Alliance for Medical Image Computing 3 Overall Goal for NA-MIC Why Medical Research? –To Help Patients!

National Alliance for Medical Image Computing 4 How to Make this Happen? Enable World-Class Biomedical Research Develop a Comprehensive Platform Create a Community of Developers and Users Ensure Maximum Reusability of Software

National Alliance for Medical Image Computing Outline Survey of Application Targets Introduction to NA-MIC Overview of 3D Slicer Features 5

National Alliance for Medical Image Computing 6 Autism Development Correlate Localized Brain Development with Observed Behavioral Characteristics of Autism Structural and Diffusion Imaging at 2yo and 4yo Cortical Thickness and Subcortical Volume Metrics Blue: Growth Red: Atrophy Green: No Change Images: Hazlett et al

National Alliance for Medical Image Computing 7 Schizophrenia Structural and Diffusion to Discover Disruptions Correlated with Symptoms First Generation DBP –Regional Analysis of FA Based on Manual Tract Identification Second Generation DBP –Stochastic Tractography, Improved Metrics, Registration Images Ngo et al.

National Alliance for Medical Image Computing 8 Prostate Disorders Segmentation tool for Brachytherapy Planning and Biopsy Navigation for Biopsy Future FUS and Robot Applications Haker SJ, Mulkern RV, Roebuck JR, Barnes AS, Dimaio S, Hata N, Tempany CM.: Magnetic resonance-guided prostate interventions. Top Magn Reson Imaging Oct;16(5):

National Alliance for Medical Image Computing 9 Lupus Lesions Automatic Analysis of White Matter Abnormalities in Neuropsychiatric SLE (Lupus) About 1.5 Million Americans with Lupus, Underlying Pathologic Processes Unknown – Possibly Vascular Hypointense on T1Hyperintense T2Hyperintense on FLAIR Images: Bockholt et al

National Alliance for Medical Image Computing Patient-Specific Finite Element Model Development Iowa: Kiran H. Shivanna, Vincent A. Magnotta, Nicole M. Grosland, NA-MIC: Steve Pieper, Curt Lisle Automate the generation of high quality hexahedral meshes Inclusion of soft tissues such as cartilage Automated Segmentation Validation Published / Accepted –Devries NA, Gassman EE, Kallemeyn NA, Shivanna KH, Magnotta VA, Grosland NM. Validation of phalanx bone three- dimensional surface segmentation from computed tomography images using laser scanning. Skeletal Radiol Jan;37(1): Epub 2007 Oct 25. –Gassman EE, Powell SM, Kallemeyn NA, DeVries NA, Shivanna KH, Magnotta VA, Ramme AJ, Adams BD, Grosland NM, Automated Bony Region Identification Using Artificial Neural Networks: Reliability and Validation Measurements. Skeletal Radiology (accepted / online). Grant funding NIH –R21 (EB001501) –R01 (EB005973)

National Alliance for Medical Image Computing Alcohol and Stress Primate (Rhesus and Vervet) Models to Study the Effect of Chronic Alcohol Self- Administration on Brain Structure and Function Compare Mother-Reared vs. Nursery-Reared Apply Automated Segmentation Tools to Assess Cortical and Subcortical Regions (EMSegmenter, Pohl et al) 11 Chlorocebus aethiops (vervet) * Rhesus template and atlas graciously provided by Martin Styner and collaborators Images: Wyatt et al

National Alliance for Medical Image Computing 12 Is the active visualization of medical images to aid in decision making during a procedure. Allows physician to –See Beyond the Surface –DefineTargets –Control the Interventions Enables new procedures, decreases invasiveness, optimizes resection Image Guided Therapy (IGT) Dimaio SP, Archip N, Hata N, Talos IF, Warfield SK, Majumdar A, Mcdannold N, Hynynen K, Morrison PR, Wells WM 3rd, Kacher DF, Ellis RE, Golby AJ, Black PM, Jolesz FA, Kikinis R.: Image-guided neurosurgery at Brigham and Women's Hospital.IEEE Eng Med Biol Mag Sep- Oct;25(5):67-73

National Alliance for Medical Image Computing 13 Alignment of all pre-operative datasets to the intra-operative images achieved during the neurosurgery. Provided by Archip, Warfield Mapping Archip N, Clatz O, Whalen S, Kacher D, Fedorov A, Kot A, Chrisochoides N, Jolesz F, Golby A, Black PM, Warfield SK. Non-rigid alignment of pre- operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image- guided neurosurgery. Neuroimage Apr 1;35(2):609-24

National Alliance for Medical Image Computing Liver Lesion Treatment Needle Placement for CT-Guided Radio Frequency Ablation (RFA) –Unresectable Hepatocellular Carcinoma (HCC) and Liver Metastases Open Source Environment –Slicer3 and IGSTK Automated Liver Segmentation Path Planning Tools for Multiple Overlapping Lesions Validation on Swine Model 14 Images: Cleary et al

National Alliance for Medical Image Computing 15 NA-MIC: A Network of Peers Leadership: –BWH: Ron Kikinis, (Overall PI) Core 1 Algorithms –Utah: Ross Whitaker (Core 1 PI) –MIT: Eric Grimson –UNC: Guido Gerig –MGH: Bruce Fischl, Dave Kennedy –GaTech: Allen Tannenbaum Core 2 Engineering –Kitware: Will Schroeder (Core 2 PI) –GE: Jim Miller –Isomics: Steve Pieper –UCSD: Mark Ellisman –UCLA: Art Toga Core 3 DBP –BWH: Martha Shenton –Dartmouth: Andy Saykin –UCI: Steve Potkin –UofT: Jim Kennedy DBP 2007 –UNC: H. Cody –BWH: M. Kubicki –Mind Institute: J. Bockolt –Queens University: G. Fichtinger Core 4 Service –Kitware: Will Schroeder Core 5 Training –MGH: Randy Gollub Core 6 Dissemination –Isomics: Steve Pieper, Tina Kapur Core 7 Management –BWH: S. Manandhar Provided by Pieper, Kikinis

National Alliance for Medical Image Computing 16 NA-MIC Governance NIH Goals for Software Sharing: …software should be freely available … …permit the commercialization of enhanced or customized versions … …include the ability of researchers outside the center and its collaborating projects to modify the source code and to share modifications …

National Alliance for Medical Image Computing 17 BSD Style License Open Source No license fees No restriction on use –Clinical and commercial uses do not require our permission No guarantees –User is responsible for making sure that the software works, we promise nothing. Same for compliance with all regulations. –E.g. if you want to use our software for clinical trials, you have to apply for the proper authorizations at your institution NO requirement to give your code back to open source (NOT viral). –You can contribute code back to us, if you want to do that and it is our decision if we will accept it. Acknowledge our contribution See for more information

National Alliance for Medical Image Computing 18 FOSS – A Public Highway… Open-source is like a Public Road System –Provides Infrastructure for a Variety of Uses –Driveways can Lead to Anything: a Public Park a Private Facility FOSS= Free Open Source Software Provided by Pieper, Kikinis

National Alliance for Medical Image Computing 19 Dissemination and Training National and International Events –MIT, MGH, UNC, EPFL, NIH, UNM, UCSD… All Materials on Wiki Project Weeks –Full Week Each Summer –½ Week at Winter AHM Workshops –MICCAI 2005, 2006, 2007, –OHBM, RSNA, Munich, NCI…

National Alliance for Medical Image Computing 20 Project Week Next Week (June 23-27) at MIT Stata Center > 100 Participants 32 Institutions 6 Countries 5 Companies Project Teams Sit and Work Together Face to Face Communication

National Alliance for Medical Image Computing 21 What is 3D Slicer? Reference NA-MIC Kit Implementation A platform for image analysis and visualization Current Releases 2.7 and 3.2 –2.7 most features and documentation –3.0 focus of current activity A freely-downloadable program –Source code and executables available for Windows, Linux, and Mac OS X –All Code Reusable Slicer is a research platform: –NOT an FDA approved medical device –NOT finished – some parts will work better than others

National Alliance for Medical Image Computing 22 Slicer Features I/O: –Image: DICOM, NIfTI, Analyze, Meta, NRRD, MGZ... –Surface: vtk, vtp, stl, freesurfer, fiber bundle Coordinate Systems: All Data is Patient-Referenced Visualization: Volume Rendering, Surfaces, Slices Planes, Clipping, Volume Overlays... Filtering: Denoising, Nonlinear/Aniostropic Smoothing, Format Conversion Registration: Multimodal (e.g. CT/MR), Rigid, Affine, BSpline. Apply Transforms to other volumes.

National Alliance for Medical Image Computing 23 More Slicer Features Segmentation: Manual Editor, Semi-Automated (Region Growing), Statistical Classifiers, Atlas Based Diffusion Imaging: DICOM Import, Tensor Tools, Tractography Quantification: Volume Measurements, Points, Lines Real-time: Networked Trackers and Volume I/O And more... –Meshing, Fiducials, Batch Processing, Remote Data I/O, Extensible in C++/Tcl/Python, Slicer Daemon...

National Alliance for Medical Image Computing 24 Integrated Scene XML-Based MRML File Stores Scene Description –Volumes (Images, Label Maps) –Models –Hierarchical Affine Transforms –Scene Data (Cameras, Colors, Fiducials, etc). Manipulated in World Coordinates based on Patient RAS (Right-Anterior-Superior) Provided by N. Archip et al

National Alliance for Medical Image Computing Integrated Visualization 25

National Alliance for Medical Image Computing Volume Rendering 26

National Alliance for Medical Image Computing 3D Widgets 27

National Alliance for Medical Image Computing Light Box Viewing 28

National Alliance for Medical Image Computing Scene Snapshot 29

National Alliance for Medical Image Computing Label Map Editor 30

National Alliance for Medical Image Computing Rich I/O Support 31

National Alliance for Medical Image Computing Diffusion Volumes 32

National Alliance for Medical Image Computing Remote Data I/O 33

National Alliance for Medical Image Computing OpenIGTLink 34

National Alliance for Medical Image Computing Registration 35

National Alliance for Medical Image Computing Segmentation 36