1. Cheuk Yiu Ip Amitabh Varshney Joseph JaJa
Hierarchical Exploration of Volumes Using Multilevel Segmentation of the Intensity-Gradient Histograms
Cheuk Yiu Ip Amitabh Varshney Joseph JaJa

2. Volume Exploration Challenge
Raw Volume Data Cube
Meaningful Visualization
[Voreen CG&A 09]
Similar to other papers in this session
Given a volumetric data cube on the left. We want to generate a meaningful visualization on the right.

3. Transfer Function Evolution
How do we find the “right” transfer function?
RGBA
Traditionally, we use a transfer function to control the appearance of a visualization, the transfer function may look pretty complex.
Finding the right transfer function has always been a challenge in scientific visualization
Intensity

4. Histogram Helps Transfer Function Design
To assist transfer function design, one of the first volume rendering papers shows a histogram may be helpful.
[Drebin et al. SIGGRAPH 88]

5. Histogram Helps Transfer Function Design
The peaks of the histogram are composed by the distributions of different constituents
[Drebin et al. SIGGRAPH 88]

6. Histogram Helps Transfer Function Design
By assigning different colors and opacities to these components. we can visualize different materials
[Drebin et al. SIGGRAPH 88]

7. Histogram Helps Transfer Function Design
For example, by following the peak in the histogram we can obtain this visualization of the cadaver head dataset

8. Histograms May Not Always Help
However, the separation of the distributions may not be always clear

9. Volume Exploration Exhaustively explore the dataset
In this case, we have to exhaustively explore the dataset by modifying the transfer function

10. Volume Exploration Seek for salient features
We seek for the salient features one by one

11. Volume Exploration
Seek for salient features

12. Volume Exploration
Seek for salient features

13. Volume Exploration Feature locations can be arbitrary
The meaningful feature locations can be pretty arbitrary

14. 2D Transfer Functions
Gradient ( f’(x) ) captures boundaries Histogram shapes ⇒ Volume segments
To assist the search for meaningful and salient features, the transfer function has been extended to 2D by incorporating gradient.
We plot a histogram in 2D with the intensity on the x axis, gradient on the y axis, the darkness of the shade represents the frequency
The gradient helps capturing material boundaries.
The amazing insight here is shapes on the 2D intensity-gradient histogram correspond to meaningful volumetric segments.
This has been implemented in popular visualization packages such as voreen Imagevis3d and VisIt
[Kindlmann & Durkin VolVis98,
Kniss et al. TVCG 02]
Gradient
Intensity

15. Advances on New Attributes
Higher Derivatives: f”(x) [Kindlmann & Durkin VolVis98] Specific features: Size [Correa and Ma TVCG 08] LH-transform [Sereda et al. TVCG 06], Domain specific semantic attributes [Salama et al. TVCG 06] Select good views: Visibility [Correa and Ma TVCG 10], Information divergence [Ruiz et al. TVCG 11]
Many subsequent work has introduced new attributes such as higher derivatives, size, LH Transform, and even domain specific semantic attributes
Visibility and information divergence have been used to select good views.
For this work we will go back to intensity and gradient

16. Challenges of the 2D Transfer Function
Search for separated meaningful features
1 Region 1 Minute
Let’s take a look at the use of a 2D transfer function
Users draw shapes on the 2D histogram to extract meaningful components
This video shows how a user works with the histogram in ImageVis3D
The arc structures can clearly be seen on the intensity gradient histogram
The user is trying to use a polygon widget to highlight the feature that corresponds to the top arc.
As we can see, manipulating the widget to separate the crown from the root takes quite some effort.
The challenge here is the histogram does not show where do the good features separate.
Extracting this crown takes almost 1 min

17. Histogram and Volume Features
Let’s take a look at the vismale dataset

18. Histogram and Volume Features
Skin

19. Histogram and Volume Features
Flesh

20. Histogram and Volume Features
Skull

21. Histogram and Volume Features
Sinus

22. Histogram and Volume Features
Teeth
We can see that meaningful components do correspond to histogram segments.
In this talk, my goal is to show you how to discover these regions in a systematic manner.

23. Approximate Histogram Transfer Functions
Existing approaches directly or indirectly fit the histogram
User Specified [Kniss et al. TVCG 02,
Fogal et al. 2010]
[Wang et al. TVCG 2011]
Existing approaches directly or indirectly fit the 2D histogram to assist transfer function design
These primitives may not cover the whole histogram completely to guarantee exhaustive exploration

24. Reduce Search to Classification
Recursive histogram classification Tight coverage with a few segments Exhaustive exploration
In this work, we attempt to reduce this search to classification by recursively segmenting the histogram.
We introduce a visual-inspired approach that tightly fits the histogram with a few segments.
We will show how to interactively and exhaustively explore the whole dataset

25. Overview Segment the histogram statistics
Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume,
we build an exhaustive multilevel hierarchy
and use it to provide interactive exploration.
Our approach provides a few advantages
Visual segmentation matches user intuition,
we guarantee tight and complete coverage,
Our approach augments the intensity-gradient feature space without requiring the users to learn any new features
Segment the histogram statistics
Build an exhaustive multilevel hierarchy
User interactive exploration

26. Overview
Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume,
we build an exhaustive multilevel hierarchy
and use it to provide interactive exploration.
Our approach provides a few advantages
Visual segmentation matches user intuition,
we guarantee tight and complete coverage,
Our approach augments the intensity-gradient feature space without requiring the users to learn any new features
Visual segmentation matches user intuition Complete coverage Users stay in a familiar feature space

27. Intensity-gradient Histogram
The intensity gradient histogram is very effective because
Human users can extract volume segments by recognizing the arcs and blobs
Given these histograms, can we extract these shapes automatically?
We mimic this user behavior by applying computer vision techniques to segment the histogram as an image in a top-down fashion
Intensity
Users implicitly recognize shapes from the histogram Segment this histogram as an image

28. Normalized-cut Image Segmentation
Normalized-cut (ncut) image segmentation [Shi & Malik PAMI 98] Min ncut produces balanced segments Eigenanalysis approximates the min ncut for k segments B
Instead of directly segmenting the volume data.
We segment the histogram by using normalized-cut to find shapes
This example shows the normalized cut provides intuitive segments on natural images.
Normalize cut models the image as a graph, with the similarly of pixels as edge weights
The weight of a cut in the graph is the total weights along the separating edges.
shi and malik normalize this cut by the weights of the associated segments in the graph.
The minimum normalize cut favors balanced segments
We find the top k eigenvectors to approximate this minimum cut for k different segments
[Wang et al. PATTERN RECOGN LETT 06]

29. Normalized-cut on Intensity-gradient Histogram
We apply normalized-cut on bit histograms k=2 separates the tooth from the volume box k=10 shows segments of the tooth crown and root k=20 shows different material boundaries
We apply normalized cut to segment 256 sq 8 bit histogram images.
Eigenanalysis on these images only takes a few seconds
When k = 2 the tooth is separated from the volume box
k=10 shows segments of the tooth crown and root
k = 20 shows different material boundaries. These segments tightly cover the whole histogram.
So how many segments should we choose to compose our visualization?

30. Which k should we pick ?
Iteratively picking k is tedious Increasing k may not subdivide region of user interest
Try k = 2, 3, 4, …
Iteratively test and pick k is time consuming and yet increasing k may not subdivide regions of interest.
For example when we increase k from 4 to 5 , the box is subdivided. We believe users would prefer subdividing the tooth.

31. Replace k with User-driven Exploration
Multilevel Segmentation Hierarchy: Apply normalized-cut recursively
we need to allow users to select the appropriate segments instead of choosing a specific k,
In order to accomplish this, we build a hierarchy by recursively applying the normalized cut.

32. Multilevel Segmentation Hierarchy
Selectively inspect segments of choice
This hierarchy leads the users to traverse the dataset, selectively subdivide and inspect segments of their choice

33. Multilevel Segmentation Hierarchy
Any cut guarantees complete coverage
View-dependent LoD hierarchies
[Xia & Varshney Vis 96,
Hoppe SIGGRAPH 97,
Luebke & Erikson SIGGRAPH 97]
Any cut along this hierarchy guarantees complete coverage.
This is similar to view dependent level of details hierarchy

34. Multilevel Segmentation Hierarchy
Having this flexibility, the next question is which segment should we subdivide?

35. Information Guided Traversal
Segment entropy High entropy ⇒ Complex segment
We evaluate information content of the segments to provide some guidance
We use red to show which segment has a higher entropy
for example the entropy of the tooth is higher than the volume box, suggesting the tooth is more complex, we subdivide the tooth.
Entropy
6.8
2.8

36. What if the Entropies are Similar…
The segment entropies can be similar Which segment should we divide next? Use Information Gain
Entropy
But, It is possible that the segment entropies are similar. For example there is no difference for the 2 segments in the right arc.
Which segment should we divide next ?
We use info gain to answer this question

37. Information-Gain Guided Traversal
Information Gain = Entropy reduction after a subdivision High Information Gain ⇒ Structural separation
we alternatively evaluate the information gain of a subdivision
The information gain is defined as the entropy reduction of a subdivision
Subdivisions with high information gain suggest separations of structures.
We use blue to represent the information gain, for example splitting the lower segment with high information gain separates the surfaces of the tooth crown and dentine
0.01
Information
Gain
0.11

38. Interactive Exploration
Explore the segmentation hierarchy Selective expansion Interactive visualizations Exhaustively explore the tooth in 1 minute
Here we show how a user can interactively explore the segmentation hierarchy, the user selectively click on the histogram to expand the segments and interactively compose visualizations
Notice that the tooth crown in the upper arc can be identified in only 3 expansions.
Users iteratively expand the regions of interest and hide uninteresting regions.
A visualization of the volume dataset is composed along this hierarchy traversal
In addition to identifying different meaningful segments, users can also segment and hide the undesired noise
In contrast with extracting a single region. We can now exhaustively explore the whole tooth in 1 min.

39. Examples Engine block Visible Human Male Head
Let’s take a look at some results
Internal components
Engine block and surface
Skull, flesh, skin, highlight sinus and teeth

40. Examples Tomato Hurricane Isabel Tomato Skin, Chamber, Center Column
Isabel pressure, with no distinct material boundaries
Eye, Eye Wall, Rain band, and further segment the eye

41. Conclusions
Computational segmentation mimics user interactions Intuitive volumetric classification Exhaustive multilevel hierarchy Information guided traversal Interactive exploration
In conclusion, we have shown the idea of using computational visual segmentation to effectively mimic user interaction.
It classifies intuitive volumetric regions.
We have built an exhaustive multilevel hierarchy from these segments
and We have provided information content to guide the hierarchy traversal
Users can interactively explore and visualize with these ingredients.

42. Future Work
Improve the information content measures Automatic color assignment for segments Segment histograms with different attributes Time varying datasets

43. Acknowledgements Thank you!
National Science Foundation: CCF , CMMI , CNS
NVIDIA CUDA Center of Excellence Program
Derek Juba, Sujal Bista, Yang Yang, M. Adil Yalcin, and the reviewers for improving this paper and presentation
The SciVis Best Paper Award committee
Thank you!

44. Questions ?
Source code for building the hierarchy: Papers and videos: Cheuk Yiu Ip GVIL Research Highlights