1. Slicer3 Rendering Pipeline Study documents Raimundo Sierra

2. Nomenclature: Datavolume: x y z Slicer3 normal mode: pxpx pypy I1I1 I2I2 I3I3 Normal mode, Px, Py typically: 200x200  O(px* py) ≈ 40k pixels Full screen: 1000x1000 ≈ 1M pixels Factor 25 leads to perceivable slowdown, from 16ms/rendering to 200 ms pxpx pypy I1I1 Full screen mode:

3. Slicer 3 Rendering Pipeline Image Processing Data Volume Image Processing Rendering Display GUI, Programming GUI Navigation GPU CPU Painting? Image Processing vtkSlicerSliceLayerLogic.h vtkSlicerSliceLogic.h

4. A Image Processing Details vtkSlicerSliceLayerLogic.h Reslice Threshold AppendReslice Extract Alpha Cast Alpha logic Append Thres- hold Extract Lumi- nance Map to colors 2D 3D O(x*y*z)O(p x * p y ) RGB + standard VTK filters ─ 3D operations = SLOW!

5. A Image Processing Details Extended Reslice Reslice Cast Alpha logic Append Thres- hold Map to colors 2D 3D O(x*y*z)O(p x * p y ) RGB Map to window level colors New output: ImageData * BackgroundImageMask + faster ─ filter with additional source

6. A Image Processing Details Next: ExtractImage Reslice Cast Alpha logic Append Thres- hold Map to colors RGB Map to window level colors + only one iteration through data ─ flexibility? ─ still O(px* py)

7. Slicer 3 Rendering Pipeline Image Processing Data Volume Image Processing Rendering Display GUI, Programming GUI Navigation GPU CPU Painting? Image Processing vtkSlicerSliceLayerLogic.h vtkSlicerSliceLogic.h

8. Image Processing Details vtkSlicerSliceLogic.h Foreground Blend Background Labelmap Append

9. Slicer 3 Rendering Pipeline Image Processing Data Volume Image Processing Rendering Display GUI, Programming GUI Navigation GPU CPU Painting? Rendering

10. Slicer 3 Rendering Pipeline GPU Based rendering Image Processing Data Volume Image Processing Rendering Display GPU CPU 1.Submit complete datavolume to the GPU (glTexture3D) 2.Extract slice by specifying plane with associated texture coordinates (glVertex, glTextureCoord) 3.Render textured plane in different rendering targets (slice windows & 3D window

11. Current Rendering Pipeline vs. GPU Based rendering Pros Speed: Use of parallel rendering pipeline of GPU Cons Limited GPU memory in particular when volume consists of foreground, background, and labelmap datasets Implementation: –3D texture support in VTK –Large/significant changes in existing code