1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org 3D Slicer Modules and Extensions Steve Pieper, Ph.D.

2. National Alliance for Medical Image Computing http://na-mic.org How Modules Should Work Slicer 3.4 is a Stable Base for Many Medical Image Computing Apps –Core File I/O, Visualization, Interface… –Tutorials and Other Resources to Get New Users Up to Speed Developers of New Techniques can Draw on the NA-MIC Kit to Implement Compatible Modules –Consistent Framework for Deployment and User Interface –Several Styles of Development to Meet Varied Requirements

3. National Alliance for Medical Image Computing http://na-mic.org Provided by D. Gering Slicer3 data model Provided by R. Kikinis

4. National Alliance for Medical Image Computing http://na-mic.org Slicer3 “Observer MVC” Pattern MRML (Model) –For Scene Description and Application State –MRML Nodes are Persistent and Undoable –Scene and Nodes are Observable Logic Encapsulate VTK and ITK Pipelines (Controller) –Observe MRML to Configure Pipelines –Help Create/Manage Nodes –No UI Components (no Widgets, Actors, Mappers, Renderers or RenderWindows) GUI (View) –Observe and Edit MRML –Interact with User and Display Hardware Modules Should Follow Same Conventions Logic MRML Nodes GUI WidgetsRenderers Edit Observe Edit “Observe” means generic event mechanisms are used to pass information. “Edit” means code can directly call methods. Example: GUI can call methods in Logic classes, but Logic cannot call GUI methods. MRML cannot call Logic or GUI methods. There can be many observers for any event.

5. National Alliance for Medical Image Computing http://na-mic.org Variety of levels of integration Slicer Libs ModuleDescriptionParser GenerateCLP vtkITK MRML Non-slicer specific support libraries Slicer Base Application logic Widgets Common infrastructure for Slicer applications Built in modules Slice viewers Models Fiducials Transforms Full access to Slicer internals Loadable modules Query Atlas QDEC Volume rendering ChangeTracker EMSegment Full access to Slicer internals Scripted modules Editor Teem Two Tensor Tractography VMTK Limited access to Slicer internals Command line modules RegistrationRestricted access to Slicer internals Daemon OpenIGTLink Stochastic Tractography Access to MRML

6. National Alliance for Medical Image Computing http://na-mic.org Base Features vs. Modules FeaturesBaseModules VisualizationMRML: Models, Volumes, Volume Rendering, Fiducials, Continuum Meshes, Labeled Data Can create custom MRML Nodes and behavior FilteringNoneImplemented as Modules using ITK or other Libraries RegistrationTransform Display and Edit, Save/RestoreCalculate Transforms, Resample Data SegmentationLabel Maps, Parcellated SurfacesSegmentation Algorithms in ITK or other Libraries QuantificationLabel, Image, Volume Statistics; Numpy access to MRML Applications in Python or MATLAB Real-time IntegrationVTK Rendering, KWWidgets framework, Tracker Support (as Transforms) Direct Manipulation of the MRML Scene; 2D/3D Widgets; Device Interfaces, OpenIGTLink Module Diffusion ImagingDWI, DTI, Fiber BundlesTractography, Clustering, Atlases Applications“Bundles” of Modules in Distribution: Registration, Editor, some Filters, Customized Extensions, Domain- specific code, Optimized Interfaces…

7. National Alliance for Medical Image Computing http://na-mic.org Each module has … … an entry in the module menu … a panel of user interface controls

8. National Alliance for Medical Image Computing http://na-mic.org Module Terminology Built-In Modules –Few Core Features Linked to Slicer3.cxx –Non-Core Modules becoming Loadable Loadable Modules –Dynamically Discovered and Loaded –Can Access Application’s MRML Scene Instance and Call SlicerBaseGUI and Related APIs (vtkRendering, KWWidgets…) Scripted Modules –Like Loadable, but Written in Tcl or Python Command Line Modules –Can Run as Stand Alone Executables –Provide Standard Command Line Parsing, which Allows Slicer to Generate GUI Automatically –Can be Compiled as Shared Library for Dynamic Link to Slicer

9. National Alliance for Medical Image Computing http://na-mic.org Built in module vtkMRMLNode.hvtkSlicerModuleGUI.h vtkSlicerModuleLogic.h vtkMRMLYourModuleNode.h GetNodeTagName() Copy() ReadXMLAttributes() WriteXML() vtkSlicerYourModuleGUI.h BuildGUI() ProcessGUIEvents() ProcessLogicEvents() ProcessMRMLEvents() vtkSlicerYourModuleLogic.h ProcessMRMLEvents() ProcessLogicEvents() Slicer provided Module writer provided

10. National Alliance for Medical Image Computing http://na-mic.org Loadable module vtkMRMLNode.hvtkSlicerModuleGUI.h vtkSlicerModuleLogic.h vtkMRMLYourModuleNode.h GetNodeTagName() Copy() ReadXMLAttributes() WriteXML() vtkSlicerYourModuleGUI.h BuildGUI() ProcessGUIEvents() ProcessLogicEvents() ProcessMRMLEvents() vtkSlicerYourModuleLogic.h ProcessMRMLEvents() ProcessLogicEvents() YourModule.so GetLoadableModuleDescription() GetLoadableModuleLogic() GetLoadableModuleGUI() Slicer provided Module writer provided

11. National Alliance for Medical Image Computing http://na-mic.org Scripted Modules A Generic ScriptedModule Module Provides Mapping of C++ Method Entry Points to a Corresponding Naming Convention for Scripts –vtkMRMLScriptedModuleNode, vtkSlicerScriptedModuleLogic, vtkSlicerScriptedModuleGUI Scripts are Dynamically Discovered at Runtime A Generic ScriptedModuleNode Allows Save/Restore of Keyword / Value Pairs to Represent Module State Full API of Slicer, MRML, KWWidgets, VTK, available (except where not wrappable; these APIs have been designed for scriptability) Editor is extensive Tcl-based Module Several Python examples are included in current svn trunk –Python modules can also access Numpy routines for Numerical processing (Volumes and other DataArrays are directly mapped to Numpy arrays for easy manipulation)

12. National Alliance for Medical Image Computing http://na-mic.org Tcl scripted module vtkMRMLScriptedModule Node.h vtkSlicerScriptedModule GUI.h vtkSlicerScriptedModule Logic.h YourModuleNode.tcl YourModuleGUI.tcl proc YourModuleBuildGUI proc YourModuleProcessGUIEvents proc YourModuleProcessLogicEvents YourModuleLogic.tcl YourModule.tcl package provide YourModule x.y Slicer provided Module writer provided

13. National Alliance for Medical Image Computing http://na-mic.org Python scripted module vtkMRMLScriptedModule Node.h vtkSlicerScriptedModule GUI.h vtkSlicerScriptedModule Logic.h YourModuleGUI.py def BuildGUI() def ProcessGUIEvents() def ProcessLogicEvents() Slicer provided Module writer provided YourModuleNode.py Optional State Management YourModuleLogic.py Optional Helper Code

14. National Alliance for Medical Image Computing http://na-mic.org Command Line Modules Use the GenerateCLP command (provided with Slicer) to convert XML description of program arguments into a C++.h file Main program of module uses PARSE_ARGS macro provided by.h file to instance variables corresponding to parameters defined in XML file PARSE_ARGS also defines a --xml argument that returns original XML string on stdout Slicer parses the XML to create GUI for each module Are Covered Extensively in the HelloWorld Tutorial Note: Command Line Modules can be written in any language. C++ utilities are provided, but can be emulated / recreated as needed.

15. National Alliance for Medical Image Computing http://na-mic.org Command line module vtkMRMLCommandLine ModuleNode.h vtkCommandLine ModuleGUI.h vtkCommandLine ModuleLogic.h YourModule.xml YourModule.so XMLModuleDescription GetXMLModuleDescription() ModelEntryPoint() Slicer provided Module writer provided YourModule (exe) YourModule –xml YourModule.py XML def toXML() def Execute() or

16. National Alliance for Medical Image Computing http://na-mic.org All modules are created equal… … an entry in the module menu … a panel of user interface controls

17. National Alliance for Medical Image Computing http://na-mic.org Daemons slicerget.tcl name > image.nrrd cat image.nrrd | slicerput.tcl name YourTool (exernal process such as MATLAB) OpenIGTLink Protocol YourTool (exernal process such as Tracker or Scanner) SlicerDaemon OpenIGTLinkIF

18. National Alliance for Medical Image Computing http://na-mic.org Variety of levels of integration Slicer Libs ModuleDescriptionParser GenerateCLP vtkITK MRML Non-slicer specific support libraries Slicer Base Application logic Widgets Common infrastructure for Slicer applications Built in modules Slice viewers Models Fiducials Transforms Full access to Slicer internals Loadable modules Query Atlas QDEC Volume rendering ChangeTracker EMSegment Full access to Slicer internals Scripted modules Editor Teem Two Tensor Tractography VMTK Limited access to Slicer internals Command line modules RegistrationRestricted access to Slicer internals Daemon OpenIGTLink Stochastic Tractography Access to MRML

19. National Alliance for Medical Image Computing http://na-mic.org Extension Goals Keep the base package “lean and mean” Modules have individual identity –Per-module web site, svn, downloads, mailing lists, wiki… Allow users to assemble their own set of tools –Customized ‘Bundles’ by task or application Easy to download compatible extensions –Analogous to Firefox extensions –Integrate extension builds into developer/nightly/release process NITRC Supplement to NA-MIC helping to pay for needed infrastructure (Neuroimaging Informatics Tools and Resources Clearinghouse) –NITRC can host neuroimaging projects (gforge implementation) Other repositories can also be used –NAMICSandBox, Institution’s svn, Google Code, etc…

20. National Alliance for Medical Image Computing http://na-mic.org Extension Management Wizard* * Under development

21. National Alliance for Medical Image Computing http://na-mic.org NITRC http://www.nitrc.org/projects/slicer3examples/ Sample Code for CommandLine, Loadable, and Scripted Extension Modules

22. National Alliance for Medical Image Computing http://na-mic.org Extension Process Write your module following the examples on nitrc.org Create.s3ext description file –http://viewvc.slicer.org/viewcvs.cgi/trunk/Extensions/http://viewvc.slicer.org/viewcvs.cgi/trunk/Extensions/ –Svn path, dependencies, homepage, category, status, description… Nightly builds create compatible sets of extension binaries that correspond to the slicer builds (windows, linux, mac) –Correct lib versions, compiler versions, etc Users can install from Extension Manager

23. National Alliance for Medical Image Computing http://na-mic.org CTK Issues Can some or all Slicer Module types be interoperable with other CTK Tools –Automatic Adaptors? What will be the CTK Preferred Module Implementation? –Web Services? daemons? OSGi?...