NA-MIC National Alliance for Medical Image Computing 3D Slicer Modules and Extensions Steve Pieper, Ph.D.

National Alliance for Medical Image Computing How Modules Should Work Slicer 3.4 is a Stable Base for Many Medical Image Computing Apps –Core File I/O, Visualization, Interface… –Tutorials and Other Resources to Get New Users Up to Speed Developers of New Techniques can Draw on the NA-MIC Kit to Implement Compatible Modules –Consistent Framework for Deployment and User Interface –Several Styles of Development to Meet Varied Requirements

National Alliance for Medical Image Computing Provided by D. Gering Slicer3 data model Provided by R. Kikinis

National Alliance for Medical Image Computing Slicer3 “Observer MVC” Pattern MRML (Model) –For Scene Description and Application State –MRML Nodes are Persistent and Undoable –Scene and Nodes are Observable Logic Encapsulate VTK and ITK Pipelines (Controller) –Observe MRML to Configure Pipelines –Help Create/Manage Nodes –No UI Components (no Widgets, Actors, Mappers, Renderers or RenderWindows) GUI (View) –Observe and Edit MRML –Interact with User and Display Hardware Modules Should Follow Same Conventions Logic MRML Nodes GUI WidgetsRenderers Edit Observe Edit “Observe” means generic event mechanisms are used to pass information. “Edit” means code can directly call methods. Example: GUI can call methods in Logic classes, but Logic cannot call GUI methods. MRML cannot call Logic or GUI methods. There can be many observers for any event.

National Alliance for Medical Image Computing Variety of levels of integration Slicer Libs ModuleDescriptionParser GenerateCLP vtkITK MRML Non-slicer specific support libraries Slicer Base Application logic Widgets Common infrastructure for Slicer applications Built in modules Slice viewers Models Fiducials Transforms Full access to Slicer internals Loadable modules Query Atlas QDEC Volume rendering ChangeTracker EMSegment Full access to Slicer internals Scripted modules Editor Teem Two Tensor Tractography VMTK Limited access to Slicer internals Command line modules RegistrationRestricted access to Slicer internals Daemon OpenIGTLink Stochastic Tractography Access to MRML

National Alliance for Medical Image Computing Base Features vs. Modules FeaturesBaseModules VisualizationMRML: Models, Volumes, Volume Rendering, Fiducials, Continuum Meshes, Labeled Data Can create custom MRML Nodes and behavior FilteringNoneImplemented as Modules using ITK or other Libraries RegistrationTransform Display and Edit, Save/RestoreCalculate Transforms, Resample Data SegmentationLabel Maps, Parcellated SurfacesSegmentation Algorithms in ITK or other Libraries QuantificationLabel, Image, Volume Statistics; Numpy access to MRML Applications in Python or MATLAB Real-time IntegrationVTK Rendering, KWWidgets framework, Tracker Support (as Transforms) Direct Manipulation of the MRML Scene; 2D/3D Widgets; Device Interfaces, OpenIGTLink Module Diffusion ImagingDWI, DTI, Fiber BundlesTractography, Clustering, Atlases Applications“Bundles” of Modules in Distribution: Registration, Editor, some Filters, Customized Extensions, Domain- specific code, Optimized Interfaces…

National Alliance for Medical Image Computing Each module has … … an entry in the module menu … a panel of user interface controls

National Alliance for Medical Image Computing Module Terminology Built-In Modules –Few Core Features Linked to Slicer3.cxx –Non-Core Modules becoming Loadable Loadable Modules –Dynamically Discovered and Loaded –Can Access Application’s MRML Scene Instance and Call SlicerBaseGUI and Related APIs (vtkRendering, KWWidgets…) Scripted Modules –Like Loadable, but Written in Tcl or Python Command Line Modules –Can Run as Stand Alone Executables –Provide Standard Command Line Parsing, which Allows Slicer to Generate GUI Automatically –Can be Compiled as Shared Library for Dynamic Link to Slicer

National Alliance for Medical Image Computing Built in module vtkMRMLNode.hvtkSlicerModuleGUI.h vtkSlicerModuleLogic.h vtkMRMLYourModuleNode.h GetNodeTagName() Copy() ReadXMLAttributes() WriteXML() vtkSlicerYourModuleGUI.h BuildGUI() ProcessGUIEvents() ProcessLogicEvents() ProcessMRMLEvents() vtkSlicerYourModuleLogic.h ProcessMRMLEvents() ProcessLogicEvents() Slicer provided Module writer provided

National Alliance for Medical Image Computing Loadable module vtkMRMLNode.hvtkSlicerModuleGUI.h vtkSlicerModuleLogic.h vtkMRMLYourModuleNode.h GetNodeTagName() Copy() ReadXMLAttributes() WriteXML() vtkSlicerYourModuleGUI.h BuildGUI() ProcessGUIEvents() ProcessLogicEvents() ProcessMRMLEvents() vtkSlicerYourModuleLogic.h ProcessMRMLEvents() ProcessLogicEvents() YourModule.so GetLoadableModuleDescription() GetLoadableModuleLogic() GetLoadableModuleGUI() Slicer provided Module writer provided

National Alliance for Medical Image Computing Scripted Modules A Generic ScriptedModule Module Provides Mapping of C++ Method Entry Points to a Corresponding Naming Convention for Scripts –vtkMRMLScriptedModuleNode, vtkSlicerScriptedModuleLogic, vtkSlicerScriptedModuleGUI Scripts are Dynamically Discovered at Runtime A Generic ScriptedModuleNode Allows Save/Restore of Keyword / Value Pairs to Represent Module State Full API of Slicer, MRML, KWWidgets, VTK, available (except where not wrappable; these APIs have been designed for scriptability) Editor is extensive Tcl-based Module Several Python examples are included in current svn trunk –Python modules can also access Numpy routines for Numerical processing (Volumes and other DataArrays are directly mapped to Numpy arrays for easy manipulation)

National Alliance for Medical Image Computing Tcl scripted module vtkMRMLScriptedModule Node.h vtkSlicerScriptedModule GUI.h vtkSlicerScriptedModule Logic.h YourModuleNode.tcl YourModuleGUI.tcl proc YourModuleBuildGUI proc YourModuleProcessGUIEvents proc YourModuleProcessLogicEvents YourModuleLogic.tcl YourModule.tcl package provide YourModule x.y Slicer provided Module writer provided

National Alliance for Medical Image Computing Python scripted module vtkMRMLScriptedModule Node.h vtkSlicerScriptedModule GUI.h vtkSlicerScriptedModule Logic.h YourModuleGUI.py def BuildGUI() def ProcessGUIEvents() def ProcessLogicEvents() Slicer provided Module writer provided YourModuleNode.py Optional State Management YourModuleLogic.py Optional Helper Code

National Alliance for Medical Image Computing Command Line Modules Use the GenerateCLP command (provided with Slicer) to convert XML description of program arguments into a C++.h file Main program of module uses PARSE_ARGS macro provided by.h file to instance variables corresponding to parameters defined in XML file PARSE_ARGS also defines a --xml argument that returns original XML string on stdout Slicer parses the XML to create GUI for each module Are Covered Extensively in the HelloWorld Tutorial Note: Command Line Modules can be written in any language. C++ utilities are provided, but can be emulated / recreated as needed.

National Alliance for Medical Image Computing Command line module vtkMRMLCommandLine ModuleNode.h vtkCommandLine ModuleGUI.h vtkCommandLine ModuleLogic.h YourModule.xml YourModule.so XMLModuleDescription GetXMLModuleDescription() ModelEntryPoint() Slicer provided Module writer provided YourModule (exe) YourModule –xml YourModule.py XML def toXML() def Execute() or

National Alliance for Medical Image Computing All modules are created equal… … an entry in the module menu … a panel of user interface controls

National Alliance for Medical Image Computing Daemons slicerget.tcl name > image.nrrd cat image.nrrd | slicerput.tcl name YourTool (exernal process such as MATLAB) OpenIGTLink Protocol YourTool (exernal process such as Tracker or Scanner) SlicerDaemon OpenIGTLinkIF

National Alliance for Medical Image Computing Variety of levels of integration Slicer Libs ModuleDescriptionParser GenerateCLP vtkITK MRML Non-slicer specific support libraries Slicer Base Application logic Widgets Common infrastructure for Slicer applications Built in modules Slice viewers Models Fiducials Transforms Full access to Slicer internals Loadable modules Query Atlas QDEC Volume rendering ChangeTracker EMSegment Full access to Slicer internals Scripted modules Editor Teem Two Tensor Tractography VMTK Limited access to Slicer internals Command line modules RegistrationRestricted access to Slicer internals Daemon OpenIGTLink Stochastic Tractography Access to MRML

National Alliance for Medical Image Computing Extension Goals Keep the base package “lean and mean” Modules have individual identity –Per-module web site, svn, downloads, mailing lists, wiki… Allow users to assemble their own set of tools –Customized ‘Bundles’ by task or application Easy to download compatible extensions –Analogous to Firefox extensions –Integrate extension builds into developer/nightly/release process NITRC Supplement to NA-MIC helping to pay for needed infrastructure (Neuroimaging Informatics Tools and Resources Clearinghouse) –NITRC can host neuroimaging projects (gforge implementation) Other repositories can also be used –NAMICSandBox, Institution’s svn, Google Code, etc…

National Alliance for Medical Image Computing Extension Management Wizard* * Under development

National Alliance for Medical Image Computing NITRC Sample Code for CommandLine, Loadable, and Scripted Extension Modules

National Alliance for Medical Image Computing Extension Process Write your module following the examples on nitrc.org Create.s3ext description file – –Svn path, dependencies, homepage, category, status, description… Nightly builds create compatible sets of extension binaries that correspond to the slicer builds (windows, linux, mac) –Correct lib versions, compiler versions, etc Users can install from Extension Manager

National Alliance for Medical Image Computing CTK Issues Can some or all Slicer Module types be interoperable with other CTK Tools –Automatic Adaptors? What will be the CTK Preferred Module Implementation? –Web Services? daemons? OSGi?...