Institute of Simulation and Graphics, Otto-von-Guericke University Magdeburg Direct Volume Visualization Part I.

Slides:



Advertisements
Similar presentations
7.1 Vis_04 Data Visualization Lecture 7 3D Scalar Visualization Part 2 : Volume Rendering- Introduction.
Advertisements

ENV 2006 CS4.1 Envisioning Information: Case Study 4 Focus and Context for Volume Visualization.
Exploration of advanced lighting and shading techniques
Technische Universität München Fakultät für Informatik Computer Graphics SS 2014 Sampling Rüdiger Westermann Lehrstuhl für Computer Graphik und Visualisierung.
Technische Universität München Computer Graphics SS 2014 Graphics Effects Rüdiger Westermann Lehrstuhl für Computer Graphik und Visualisierung.
VIS Group, University of Stuttgart Tutorial T4: Programmable Graphics Hardware for Interactive Visualization Pre-Integrated Splatting (Stefan Roettger)
Graphics Pipeline.
Direct Volume Rendering. What is volume rendering? Accumulate information along 1 dimension line through volume.
A Coherent Grid Traversal Algorithm for Volume Rendering Ioannis Makris Supervisors: Philipp Slusallek*, Céline Loscos *Computer Graphics Lab, Universität.
3D Graphics Rendering and Terrain Modeling
REAL-TIME VOLUME GRAPHICS Christof Rezk Salama Computer Graphics and Multimedia Group, University of Siegen, Germany Eurographics 2006 Real-Time Volume.
Fast Volume Rendering Using a Shear-Warp Factorization of the Viewing Transformation Philippe Larcoute & Marc Levoy Stanford University Published in SIGGRAPH.
Other DVR Algorithms and A Comparison Jian Huang, CS 594, Spring 2002.
Interactive High-Quality Volume Rendering on Flexible Consumer Graphics Hardware Klaus Engel, Martin Kraus, Thomas Ertl Visualization and Interactive Systems.
Real-Time Rendering TEXTURING Lecture 02 Marina Gavrilova.
Part I: Basics of Computer Graphics
Tuesday February 19 th, 2002 Deep Shadow Maps Tom Lokovic & Eric Veach Pixar Animation Studios Presented by Tom Lechner.
Hank Childs, University of Oregon November 15 th, 2013 Volume Rendering, Part 2.
Volume Rendering Volume Modeling Volume Rendering Volume Modeling Volume Rendering 20 Apr
Direct Volume Rendering (DVR): Ray-casting
Splatting Jian Huang, CS 594, Spring 2002 This set of slides reference slides made by Ohio State University alumuni over the past several years.
Computing 3D Geometry Directly From Range Images Sarah F. Frisken and Ronald N. Perry Mitsubishi Electric Research Laboratories.
Sampling, Aliasing, & Mipmaps
Surface Reconstruction from 3D Volume Data. Problem Definition Construct polyhedral surfaces from regularly-sampled 3D digital volumes.
Splatting Josh Anon Advanced Graphics 1/29/02. Types of Rendering Algorithms Backward mapping Image plane mapped into data Ray casting Forward mapping.
Introduction to Volume Rendering Presented by Zvi Devir.
(conventional Cartesian reference system)
Direct Volume Rendering Joe Michael Kniss Scientific Computing and Imaging Institute University of Utah.
Final Gathering on GPU Toshiya Hachisuka University of Tokyo Introduction Producing global illumination image without any noise.
Memory Efficient Acceleration Structures and Techniques for CPU-based Volume Raycasting of Large Data S. Grimm, S. Bruckner, A. Kanitsar and E. Gröller.
Paper by Alexander Keller
Volume Rendering & Shear-Warp Factorization Joe Zadeh January 22, 2002 CS395 - Advanced Graphics.
Introduction to 3D Graphics John E. Laird. Basic Issues u Given a internal model of a 3D world, with textures and light sources how do you project it.
Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gröler Visualization II Instructor: Jessica Crouch.
Hidden Surface Removal
Ray Tracing Primer Ref: SIGGRAPH HyperGraphHyperGraph.
VIS Group, University of Stuttgart Tutorial T4: Programmable Graphics Hardware for Interactive Visualization Volume Graphics - Advanced Klaus Engel Volume.
Technology and Historical Overview. Introduction to 3d Computer Graphics  3D computer graphics is the science, study, and method of projecting a mathematical.
Stefan Roettger University of Stuttgart A Two-Step Approach for Interactive Pre-Integrated Volume Rendering of Unstructured Grids VolVis '02 A Two-Step.
19/18/ :34 Graphics II Volume Rendering Session 10.
Efficient Volume Visualization of Large Medical Datasets Stefan Bruckner Institute of Computer Graphics and Algorithms Vienna University of Technology.
Computer Graphics Texture Mapping
REAL-TIME VOLUME GRAPHICS Markus Hadwiger VRVis Research Center, Vienna Eurographics 2006 Real-Time Volume Graphics [04] GPU-Based Ray-Casting.
Lecture 3 : Direct Volume Rendering Bong-Soo Sohn School of Computer Science and Engineering Chung-Ang University Acknowledgement : Han-Wei Shen Lecture.
Scientific Visualization Module 6 Volumetric Algorithms (adapted by S.V. Moore – slides deleted, modified, and added) prof. dr. Alexandru (Alex) Telea.
ITUppsala universitet Advanced Computer Graphics Filip Malmberg
Direct Volume Rendering (DVR): Ray-casting Jian Huang This set of slides references slides used by Prof. Torsten Moeller (Simon Fraser), Prof. Han-Wei.
Advanced Computer Graphics Depth & Stencil Buffers / Rendering to Textures CO2409 Computer Graphics Week 19.
Visible-Surface Detection Jehee Lee Seoul National University.
1 Rendering Geometry with Relief Textures L.Baboud X.Décoret ARTIS-GRAVIR/IMAG-INRIA.
Unstructured Volume Rendering Jian Huang, CS 594, Spring 2002 This set of slides reference slides developed by Prof. Torsten Moeller, SFU, Canada.
CMSC 635 Volume Rendering. Volume data  3D Scalar Field: F(x,y,z) = ?  Implicit functions  Voxel grid  Scalar data  Density  Temperature  Wind.
Volume Rendering CMSC 491/635. Volume data  3D Scalar Field: F(x,y,z) = ?  Implicit functions  Voxel grid  Scalar data  Density  Temperature  Wind.
1 Real-time visualization of large detailed volumes on GPU Cyril Crassin, Fabrice Neyret, Sylvain Lefebvre INRIA Rhône-Alpes / Grenoble Universities Interactive.
Computer Graphics 2 Lecture 7: Texture Mapping Benjamin Mora 1 University of Wales Swansea Pr. Min Chen Dr. Benjamin Mora.
Computer Graphics II University of Illinois at Chicago Volume Rendering Presentation for Computer Graphics II Prof. Andy Johnson By Raj Vikram Singh.
Digital Image Processing Lecture 16: Segmentation: Detection of Discontinuities Prof. Charlene Tsai.
112/5/ :54 Graphics II Image Based Rendering Session 11.
Volume Visualization with Ray Casting
Single Pass Point Rendering and Transparent Shading Paper by Yanci Zhang and Renato Pajarola Presentation by Harmen de Weerd and Hedde Bosman.
Discontinuous Displacement Mapping for Volume Graphics, Volume Graphics 2006, July 30, Boston, MA Discontinuous Displacement Mapping for Volume Graphics.
CS559: Computer Graphics Final Review Li Zhang Spring 2010.
COMPUTER GRAPHICS CS 482 – FALL 2015 SEPTEMBER 29, 2015 RENDERING RASTERIZATION RAY CASTING PROGRAMMABLE SHADERS.
Digital Image Processing Lecture 16: Segmentation: Detection of Discontinuities May 2, 2005 Prof. Charlene Tsai.
High-Quality Pre-Integrated Volume Rendering Using Hardware Accelerated Pixel Shading Klaus Engel, Martin Kraus, Thomas Ertl Visualization and Interactive.
Visualization of Scanned Cave Data with Global Illumination
3D Graphics Rendering PPT By Ricardo Veguilla.
Volume Rendering Lecture 21.
Real-Time Volume Graphics [06] Local Volume Illumination
Volume Rendering Lecture 21.
Presentation transcript:

Institute of Simulation and Graphics, Otto-von-Guericke University Magdeburg Direct Volume Visualization Part I

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 2/69 Motivation Strongly calcified, distinctive aneurysm of the aorta Source:

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 3/69 Motivation Left and middle: Comminuted fracture of the shoulder Right: Detailed illustration of large vessels in the upper abdomen Source:

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 4/69 Motivation Left: Two-vessel supply of the kidney Middle: Severe cystic kidney Right: Illustration of tendons and muscles in the knee region Source:

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 5/69 Classification Direct Volume Visualization (part I)  Introduction  Image-based Procedures  Object-based Procedures  Hardware Texture Procedures  Projection Methods

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 6/69 Introduction Volume visualization is of high current interest  High number and performance of the software tools (Voreen, 3DSlicer, ImageVis3D, integration into diverse scene graphs)  Integration of volume visualization procedures in clinical work stations (SIEMENS, Philips, AGFA, Vital Images...)  Beyond medicine: Material testing in engineering, analysis of seismic data for oil and gas industry, … [Vital Images] [Siemens Medical]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 7/69 Medical Applications Anatomy  Training (e.g. Visible Human data sets), functional analyses (e.g. gas exchange) Radiology  Diagnosis of vessel diseases (stenosis, aneurysm), assessment of tumors (localization, vessel infiltration) Therapy planning  Irradiation planning (dose distribution, risk assessment)  Surgery planning (planning of pathways and resection areas)  Surgery simulation for training purposes

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 8/69 Requirements System requirements  Detailed visualization of original data (relevance for diagnostic and therapeutic purposes)  Good rendering of spatial relations (visual cues, such as shadows, highlights, depth attenuation)  High illustration speed  Processing of different data types (8 bit, 16 bit, intensity values, color values, transparency values)  Integration of surface and volume data (hybrid rendering)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 9/69 Definitions Voxel: elementary part of a volume data set; assumption: the intensity in the represented partial volume is constant. Cell: elementary part of a volume data set (corresponds to 1 voxel in case of radiological data)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 10/69 Process Basic procedure  Load in of data, partly conversions,...  Direct rendering »Volume sampling  Different resampling filters for interpolation between the voxels »Classification  Transfer function assigns color and opacity to volume values »Shading (lecture II) »Approximation of the volume rendering integral via iterative blending (composition in the image space)  Image or object space methods  Different projection procedures

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 11/69 Medical Data Sets Regular data in an orthogonal grid Resolution:  Anisotropic data sets (layer distance > distance of pixels in the layer)  Characteristic: CT or MRT images: 512x512 per layer, » layers, resolution: 12 Bit per layer,~20-50 Mvoxel  High-End: Multi-slice CT: 1024x1024 per layer, »up to 500 layers, ~ MVoxel Less common:  PET, SPECT, 3D ultra sound with lower resolution [Kabayim.com]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 12/69 Medical Data Sets Examples: Inner ear with HR-CT:  512x512x64, thickness: 1 mm, slice dist.: 0.5 mm resolution: 0.12 mm Intracranial vessels, CTA:  512x512x856, resolution: 1 mm, thickness: 1 mm Source: Rezk-Salama, 2002 Typical: many transparently or semi- transparently illustrated voxels Specification: by an appropriate TF

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 13/69 Transfer Functions Setting of TFs for gray values and transparency (often linear functions). Histogram depicted as context in a graphical editor. Source: Hastreiter, 1999 [Kubisch 2010A]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 14/69 Classification Interpolation and application of TF (classification)  Pre-Classification: Application of TF to all edge points in the filter range (result: RGBA quadruple); afterwards: (tri)linear interpolation of this quadruple  Post-Classification: Interpolation of the intensity values from the data (e.g. Hounsfield Units); afterwards: application of transfer function to the interpolated result (pre-integrated for quality enhancement) Pre-classification problems:  Perception of the interpolated colors is non-linear  Imprecise classification Pre-, post-, preintegrated classification [Engel06]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 15/69 Classification Interpolation and application of the transfer function Late application of TF is more precise! Source: Rezk-Salama, 2002

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 16/69 Classification Interpolation and application of the transfer function Source: Rezk-Salama, 2002

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 17/69 Volume Rendering Equation Volume rendering equation (without shadows and diffusion): I λ Amount of light of wave length λ r Direction of the viewed ray x Position on the image plane L Length of the light ray s Current point on the light ray C λ Light of wavelength λ that is reflected or emitted from s in direction to r αDensity of the particles that are reflected to the viewer [SiggraphCourse2008]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 18/69 Numerical Approximation Numerical approximation: C λ (s i )Local color at position s i α (s i )Opacity at position s i C λ (s i ) and α are the transfer functions Product describes the remaining visibility (after traversal of the light ray) [SiggraphCourse2008]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 19/69 Resampling Filter Each volume rendering technique (image-based, object- based, and texture-based) involves resampling at sampling points in the volume Different filter kernels for resampling (here in 1D)  Nearest neighbor  Tent (linearly interpolated)  Sinc (ideal signal reconstruction, but too costly) [Engel06]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 20/69 Trilinear Interpolation Trilinear interpolation (graphics hardware)  Stepwise 1D interpolation of extensions »Along an axis »Between the results of the first axis »… the second axis  Result: typically “rounded staircases” [Engel06]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 21/69 Color value and transparency of a pixels results from overlay of semi-transparent voxels (compositing). C i := C i-1 * (1-α i ) + α i C i (color) α I := α i-1 * (1-α i ) + α i (opacity) If 1-α i becomes very small, a termination is possible. Set: I (0) = C(0), α(0) =1 Compositing via α Blending

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 22/69 Sequence of the Composition Realization of overlapping Two variants:  Back-to-Front: Drawing starts with the rearmost layer. »Advantage: Each layer is visible during the course of image generation.  Back-to-Front: Drawing starts with the foremost layer. »Advantage: The overlapping can be terminated early, if one layer or parts of it are opaque.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 23/69 Overview of DVR Approaches DVR procedure for medical visualization:  Image-based techniques that (back)trace for each pixel one ray into the scene and that compose the color/gray value from the hit voxels (weighted by transparency)  Object-based techniques that scan the voxel space and compute the contribution of the voxels to the image.  Texture-based techniques that use hardware support to speed up the volume storage access

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 24/69 Light transfer in a variable density medium [Kratz06] [Engel06] First Hit-Raycasting [Tariq07] [Hadwiger05] Tracing of rays into the scene (Raycasting) Per scan point:  Rounding to the nearest voxel (nearest neighbor)  Resampling filter from the surrounding voxels Variations:  First-hit in case of iso-surfaces  Consideration of multiple reflexions (raytracing) Image-based Techniques

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 25/69 Image-based Techniques Basic Algorithm Raycasting: for y i = 1 to ImageHeight for x i = 1 to ImageWidth for z i = 1 to RayLength foreach x 0 in ResamplingFilter (x i, y i, z i ) foreach y 0 in ResamplingFilter (x i, y i, z i ) foreach z 0 in ResamplingFilter (x i, y i, z i ) add contribution of Voxel [x 0, y 0, z 0 ] to ImagePixel [x i, y i ] Resampling filter usually corresponds to trilinear interpolation (2x2x2 values). Problem: Volume is not traversed in the order in which it lies in the storage. Usually, voxels are required that are not in the cache or in the main storage.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 26/69 Image-based Techniques Problem: Coherent volume sampling for perspective projection (divergent rays) Possible solutions:  Ray splitting (split)  Ray integrates a broader area for distant layers Image plane

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 27/69 Image-based Techniques Acceleration Methods  Early ray termination (e.g. in case of 95% opacity) (Levoy [1990])  Adaptive ray sampling (sampling rate in strongly transparent areas, or increase in case of higher distance) (Danskin, Hanrahan [1992])  Hitpoint refinement (binary search of intersection point for iso- surfaces) (Sigg 2004) Binary search by dividing the step sizes [Sigg04] Adaptive ray sampling

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 28/69 Step Size Influence of the sampling rate to aliasing effects (stepwise: 2.0 voxel, 1.0 voxel, 0.1 voxel), (© Schroeder et al. [1998]) Recommendation: step size < 0.5 voxel (sampling theorem: sampling at least with the double frequency that exists in the discrete data).

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 29/69 Sampling Rate Influence of sampling rate to aliasing effects (Step size: 1.0 voxel, 0.2 voxel)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 30/69 Object-based Techniques Splatting procedures calculate the “footprint” of a voxel on the image (Westover [1990])  Choose suitable direction for volume traversal (vertical to the view direction)  Front-to-back: voxels are processed in layers. Characteristic: Use of a Gaussian filter for reconstruction (filter size depends on the size of image and volume). The filter center may differ from the pixel center. Perspective Projection: Reconstruction filter must be calculated for each voxel. Example of a filter for application in a splatting procedure (footprint evaluation)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 31/69 Object-based Techniques Left: Voxels are filtered by convolution and projected into the image plane. Right: Projection into the image plane takes place back-to- front to assure correct overlapping (axis-aligned splatting).

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 32/69 Object-based Techniques Image-aligned splatting. The splatting buffer is parallel to the image plane.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 33/69 Splatting Algorithm Basic Algorithm Splatting: for z 0 = 1 to VolumeDepth for y 0 = 1 to VolumeHeight for x 0 = 1 to VolumeWidth foreach x i in ResamplingFilter (x 0, y 0, z 0 ) foreach y i in ResamplingFilter (x 0, y 0, z 0 ) foreach z i in ResamplingFilter (x 0, y 0, z 0 ) add contribution of Voxel [x 0, y 0, z 0 ] to ImagePixel [x i, y i ] Compared to raycasting, the inner and outer layers are interchanged and the volume is traversed in a favorable sequence.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 34/69 Object-based Techniques Acceleration through use of coherency in the scene  Composition of hierarchical data structures (e.g. octtrees, pyramids) that represent the non-transparent voxels,  Advantage: The number of splats is decreased and thus also the storage access to the volume.  Problem: »Costly generation of hierarchical data structures during each change of the transfer function. »Parallelization on splats level (ensure correct compositing sequence!).

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 35/69 Object-based Techniques Efficient Visualization: Approximation of splats via Gouraud-shaded RGBA polygons (piecewise linear splat function approach) Different forms of splats and their approach through polygons Source: Laur and Hanrahan [1991]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 36/69 Object-based Techniques Different resolution and the resulting visualization through splatting. Size and transparency of the splats are adapted to the resolution. Source: Laur and Hanrahan [1991]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 37/69 Object-based Techniques Inner octtree knots contain the average of the contained voxels (successive downsampling). Instead of the average: Sorting of voxels and average generation between 4th and 5th voxel (from 8). Progressive refinement of an octtree according to an error criterion (sum of the deviation squares to the mean value). Source: Laur and Hanrahan [1991]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 38/69 Object-based Techniques Different image quality, example of an industrial CT Source: Laur and Hanrahan [1991]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 39/69 Direct Volume Visualization Acceleration methods (accord. to Noordmanns et al. [1997]), Speed-up and errors differ strongly from the data TechnologyAuthorsSpeed-upError (%) Adaptive refinementLevoy [1990]5-50Up to 30 Early ray terminationLevoy [1990] (2-3, accord. to Levoy) 0-1 Template-based viewing Yagel et al. [1992]5 (2-3, Levoy)10 Adaptive sampling Danskin and Hanrahan [1992] Adaptive subdivisionLevoy [1990] Shear-warp projectionLacroute, Levoy [1994] 2-3 (accord. to Lacroute factor 10) 5

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 40/69 Performance Analysis Run time depends on:  Storage and cache size (especially for algorithms that build upon comprehensive data structures)  Hard disk speed (in case of large data sets)  Image type (gray value or color rendering)  Type of projection  Viewing direction (performance “jumps” in case of changes of the main viewing direction are possible)  Algorithm-specific parameters (e.g. threshold for early ray termination, minimum opacity threshold, approximation error for the classification of hierarchical data structures)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 41/69 Performance Analysis (II)  Runtime dependent on the data and transfer function  Homogeneity (measurable via compression during RLE)  Amount of the transparent voxels  Average opacity of the non-transparent voxels  Amount of the visible voxels (voxels that are non- transparent and not occluded)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 42/69 Performance Analysis (II) Consequences: Performance analysis should:  Comprise several typical data sets  Consider different resolutions of the data sets  Consider different transfer functions  View renderings from different perspectives  Break down runtimes (e.g. preprocessing, e.g. RLE, traversal of data structures, application of filters, compositing)  Contain a detailed description and discussion of the used hardware Problem:  Leaps in hardware development -> many results become outdated very fast and limit the comparison.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 43/69 Texture-based Techniques Use of the texture storage of the graphics hardware  Highly efficient volume acces with trilinear interpolation  If volume > texture storage »Bricking  Dividing the volume into bricks  Overlapping of brick ends for correct interpolation at the edges »OcTree LOD  Dissection of the volume with OcTree  Similar to MipMapping, the different planes save the volume differently precise »Challenge: Dynamical reloading of the visible bricks/Octree Leafs [SiggraphCourse2008]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 44/69 Hardware Texture Storage What makes the texture storage so fast?  Data are not linear, but are stored in a special arrangement -> improved effectiveness of the cache  Goal: Data that are locally close to each other should also be close to each other in the storage »e. g. z-curve (Morton order)  Many things are “casted” in hardware: Address conversion, interpolation, own texture caches,...  Since 2000, 3D textures are available in consumer hardware (GeForce/Radeon) [WikiPedia]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 45/69 Texture-based Slicing Approach for use of 3D/2D textures [Engel06]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 46/69 Bricking Classification of volume into bricks: artefacts (black stripes) in case of naïve decomposition at the edges (© Peter Hastreiter, Uni Erlangen)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 47/69 Bricking Interpolation Classification into bricks, such that the data overlap with each other by one voxel in each dimension. Result: continuous values also at the boundaries. (© Peter Hastreiter, Uni Erlangen)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 48/69 Texture-based Techniques “Classical” slicing: Volume is sliced through equidistant planes  Textured polygons are generated for each cutting plane. They are drawn from back to front and semi-transparently overlapped. »If 3D texture support exists: planes that are parallel to the view plane (viewport-parallel). »Only 2D textures: Set of 2D textures that reflect the main axes. Shear-warp transformation of voxel data is required (axis-parallel). GPU Raycasting: State-of-the-Art method since the programmability of the graphics hardware is sufficient  Just as the classical image-based algorithm (ray tracing in fragment/pixel shader, highly parallel implementation)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 49/69 Texture-based Slicing Source: LaMar et al. (1999) Viewport-parallel slicing enhances the quality considerably, but is not always possible (e.g. OpenGL ES 2.0 and WebGL without 3D textures, iPhone...) In connection with hierarchical data structures and perspective illustration, different cutting directions for parts of the data are reasonable. Left: vp-parallel, right: axis-parallel

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 50/69 GPU Raycasting Creation of textures that encode the start and end positions of the rays  usually by means of rasterization of a simple hull geometry Integration along the ray  Simple programming model, very high performance  Transfer function and composition are part of the traversing for loop [Kubisch/Glaßer]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 51/69 Octree Raycasting Octree manages MipMap LOD steps of the volume  Depending on angle and distance, already the high MipMap steps are adequate. They require fewer storage, since they cover a larger area.  Enables the interactive rendering of volumes up to size 8192³ (512 GB data with 512 MB graphics memory) [Crassin2009]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 52/69 Texture-based Procedures Interpolation of slices Problem: In case of strongly anisotropic data the amount of slices is insufficient for a good quality.  Interpolation of additional slices increases the volume memory consumption.  Possible solution with programmable graphics hardware: high-quality reconstruction kernels, e.g. tricubic. [HadwigerDiss] [Kubisch]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 53/69 Projection Methods Average Projection Per ray: average of all hit voxels Simulation of X-ray projections Maximum (minimum) Intensity Projection (M(m)IP) Per ray: brightest (darkest) hit voxel Illusration of vessels, noisy data Closest Vessel Projection (Zuiderveld [1995]) Per ray: first local minimum above a threshold value Illustration of Vessels

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg Projection Methods CVP schemeMIP scheme

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 55/69 Projection Methods Projection methods require no transfer function. Transfer functions can be used with projection methods (e.g. adjustment of a transfer function that depicts bones fully transparent, and consideration of transparency during application of MIP). Combination of MIP and DVR: Weighted overlapping of the voxels resulting from MIP and DVR (continuum between MIP and DVR).

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 56/69 Projection Methods MIP (data: MR angiography)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 57/69 Projection Methods Average projection, MIP, compositing (© Schroeder et al. [1998])

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 58/69 Projection Methods Comparison of MIP and DVR of cerebral vessels (aneurysm diagnosis) (data: MR Angiography, Prof. Terwey, Bremen)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 59/69 Projection Methods Advanced MIP concepts:  (1) Remove structures that disturb the MIP interpretation. »Example: Removal of bones (interactively through region growing and placing a seed point)  (2) Apply the MIP to a certain partial volume. »Example: MIP visualization in a segmented organ for targeted assessment of this organ

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 60/69 Projection Methods MIP visualization of a segmented lung. Tumors at the lung boundary become visible. Source: Dicken et al., BVM 2003, Data: Prof. Günther (RWTH Aachen)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 61/69 Projection Methods Comparison of MIP and CVP (data: MR angiography) Difficulty when applying CVP: selection of a suitable threshold value

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 62/69 Projection Methods MIP and CVP of brain vessels (© Karel Zuiderveld) Often, movies with MIP and CVP rotations in a central perspective are used for assessment of spatial relations.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 63/69 Projection Methods Distance maps: Illustration of the projection of a segmented surface. The brightness encodes the distance to the viewer. Option: Gradient filtering for accentuation of surface- near bumps. Combined views: For diagnosis, a synchronization with the view of (familiar) tomographic images is essential.

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 64/69 Projection Methods Distance map of a segmented lung. Right: Gradient-filtered distance image (Roberts-Cross Operator) and tomographic image display. Source: Dicken et al., BVM 2003, Data: Prof. Günther (RWTH Aachen)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 65/69 Projection Methods Maximum Intensity Difference Accumulation (MIDA)  Hybrid approach between Mip and standard alpha compositing  Alpha value is changed, such that large opacity jumps are additionally highlighted (although they would otherwise be occluded)  Control value enables smooth transition from classical DVR compositing to Mida to MIP [Kubisch/Glaßer]

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg Projection Methods

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg Projection Methods DVR, MIDA, MIP Courtesy of Stefan Bruckner (University of Bergen)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 68/69 Summary Important parameters in terms of quality and calculation effort:  Image resolution in the plane (e.g. amount of rays)  Sampling rate in z-direction (e.g. ray step size)  Interpolation (nearest neighbor, (tri)linear, (tri)cubic interpolation)  GPU or CPU methods  Termination of ray tracing in case of a certain cumulative intensity  Compromise: »For interactions: reduced resolution (interactive resolution) »For display (printout): high resolution (static resolution)

Bernhard Preim, Institut für Simulation und Graphik, Otto-von-Guericke Universität Magdeburg 69/69 Summary Direct volume visualization of regular data sets Image-based procedures (raycasting) Object-based procedures (splatting) Use of graphics hardware  Very high performance  Limited size of graphics memory Projection methods (MIP, CVP, average projection, MIDA) Criteria: Efficiency and quality of the image generation