Presentation is loading. Please wait.

Presentation is loading. Please wait.

Cheuk Yiu Ip Amitabh Varshney Joseph JaJa

Similar presentations

Presentation on theme: "Cheuk Yiu Ip Amitabh Varshney Joseph JaJa"— Presentation transcript:

1 Cheuk Yiu Ip Amitabh Varshney Joseph JaJa
Hierarchical Exploration of Volumes Using Multilevel Segmentation of the Intensity-Gradient Histograms Cheuk Yiu Ip Amitabh Varshney Joseph JaJa

2 Volume Exploration Challenge
Raw Volume Data Cube Meaningful Visualization [Voreen CG&A 09] Similar to other papers in this session Given a volumetric data cube on the left. We want to generate a meaningful visualization on the right.

3 Transfer Function Evolution
How do we find the “right” transfer function? RGBA Traditionally, we use a transfer function to control the appearance of a visualization, the transfer function may look pretty complex. Finding the right transfer function has always been a challenge in scientific visualization Intensity

4 Histogram Helps Transfer Function Design
To assist transfer function design, one of the first volume rendering papers shows a histogram may be helpful. [Drebin et al. SIGGRAPH 88]

5 Histogram Helps Transfer Function Design
The peaks of the histogram are composed by the distributions of different constituents [Drebin et al. SIGGRAPH 88]

6 Histogram Helps Transfer Function Design
By assigning different colors and opacities to these components. we can visualize different materials [Drebin et al. SIGGRAPH 88]

7 Histogram Helps Transfer Function Design
For example, by following the peak in the histogram we can obtain this visualization of the cadaver head dataset

8 Histograms May Not Always Help
However, the separation of the distributions may not be always clear

9 Volume Exploration Exhaustively explore the dataset
In this case, we have to exhaustively explore the dataset by modifying the transfer function

10 Volume Exploration Seek for salient features
We seek for the salient features one by one

11 Volume Exploration Seek for salient features

12 Volume Exploration Seek for salient features

13 Volume Exploration Feature locations can be arbitrary
The meaningful feature locations can be pretty arbitrary

14 2D Transfer Functions Gradient ( f’(x) ) captures boundaries Histogram shapes ⇒ Volume segments To assist the search for meaningful and salient features, the transfer function has been extended to 2D by incorporating gradient. We plot a histogram in 2D with the intensity on the x axis, gradient on the y axis, the darkness of the shade represents the frequency The gradient helps capturing material boundaries. The amazing insight here is shapes on the 2D intensity-gradient histogram correspond to meaningful volumetric segments. This has been implemented in popular visualization packages such as voreen Imagevis3d and VisIt [Kindlmann & Durkin VolVis98, Kniss et al. TVCG 02] Gradient Intensity

15 Advances on New Attributes
Higher Derivatives: f”(x) [Kindlmann & Durkin VolVis98] Specific features: Size [Correa and Ma TVCG 08] LH-transform [Sereda et al. TVCG 06], Domain specific semantic attributes [Salama et al. TVCG 06] Select good views: Visibility [Correa and Ma TVCG 10], Information divergence [Ruiz et al. TVCG 11] Many subsequent work has introduced new attributes such as higher derivatives, size, LH Transform, and even domain specific semantic attributes Visibility and information divergence have been used to select good views. For this work we will go back to intensity and gradient

16 Challenges of the 2D Transfer Function
Search for separated meaningful features 1 Region 1 Minute Let’s take a look at the use of a 2D transfer function Users draw shapes on the 2D histogram to extract meaningful components This video shows how a user works with the histogram in ImageVis3D The arc structures can clearly be seen on the intensity gradient histogram The user is trying to use a polygon widget to highlight the feature that corresponds to the top arc. As we can see, manipulating the widget to separate the crown from the root takes quite some effort. The challenge here is the histogram does not show where do the good features separate. Extracting this crown takes almost 1 min

17 Histogram and Volume Features
Let’s take a look at the vismale dataset

18 Histogram and Volume Features

19 Histogram and Volume Features

20 Histogram and Volume Features

21 Histogram and Volume Features

22 Histogram and Volume Features
Teeth We can see that meaningful components do correspond to histogram segments. In this talk, my goal is to show you how to discover these regions in a systematic manner.

23 Approximate Histogram Transfer Functions
Existing approaches directly or indirectly fit the histogram User Specified [Kniss et al. TVCG 02, Fogal et al. 2010] [Wang et al. TVCG 2011] Existing approaches directly or indirectly fit the 2D histogram to assist transfer function design These primitives may not cover the whole histogram completely to guarantee exhaustive exploration

24 Reduce Search to Classification
Recursive histogram classification Tight coverage with a few segments Exhaustive exploration In this work, we attempt to reduce this search to classification by recursively segmenting the histogram. We introduce a visual-inspired approach that tightly fits the histogram with a few segments. We will show how to interactively and exhaustively explore the whole dataset

25 Overview Segment the histogram statistics
Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume, we build an exhaustive multilevel hierarchy and use it to provide interactive exploration. Our approach provides a few advantages Visual segmentation matches user intuition, we guarantee tight and complete coverage, Our approach augments the intensity-gradient feature space without requiring the users to learn any new features Segment the histogram statistics Build an exhaustive multilevel hierarchy User interactive exploration

26 Overview Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume, we build an exhaustive multilevel hierarchy and use it to provide interactive exploration. Our approach provides a few advantages Visual segmentation matches user intuition, we guarantee tight and complete coverage, Our approach augments the intensity-gradient feature space without requiring the users to learn any new features Visual segmentation matches user intuition Complete coverage Users stay in a familiar feature space

27 Intensity-gradient Histogram
The intensity gradient histogram is very effective because Human users can extract volume segments by recognizing the arcs and blobs Given these histograms, can we extract these shapes automatically? We mimic this user behavior by applying computer vision techniques to segment the histogram as an image in a top-down fashion Intensity Users implicitly recognize shapes from the histogram Segment this histogram as an image

28 Normalized-cut Image Segmentation
Normalized-cut (ncut) image segmentation [Shi & Malik PAMI 98] Min ncut produces balanced segments Eigenanalysis approximates the min ncut for k segments B Instead of directly segmenting the volume data. We segment the histogram by using normalized-cut to find shapes This example shows the normalized cut provides intuitive segments on natural images. Normalize cut models the image as a graph, with the similarly of pixels as edge weights The weight of a cut in the graph is the total weights along the separating edges. shi and malik normalize this cut by the weights of the associated segments in the graph. The minimum normalize cut favors balanced segments We find the top k eigenvectors to approximate this minimum cut for k different segments [Wang et al. PATTERN RECOGN LETT 06]

29 Normalized-cut on Intensity-gradient Histogram
We apply normalized-cut on bit histograms k=2 separates the tooth from the volume box k=10 shows segments of the tooth crown and root k=20 shows different material boundaries We apply normalized cut to segment 256 sq 8 bit histogram images. Eigenanalysis on these images only takes a few seconds When k = 2 the tooth is separated from the volume box k=10 shows segments of the tooth crown and root k = 20 shows different material boundaries. These segments tightly cover the whole histogram. So how many segments should we choose to compose our visualization?

30 Which k should we pick ? Iteratively picking k is tedious Increasing k may not subdivide region of user interest Try k = 2, 3, 4, … Iteratively test and pick k is time consuming and yet increasing k may not subdivide regions of interest. For example when we increase k from 4 to 5 , the box is subdivided. We believe users would prefer subdividing the tooth.

31 Replace k with User-driven Exploration
Multilevel Segmentation Hierarchy: Apply normalized-cut recursively we need to allow users to select the appropriate segments instead of choosing a specific k, In order to accomplish this, we build a hierarchy by recursively applying the normalized cut.

32 Multilevel Segmentation Hierarchy
Selectively inspect segments of choice This hierarchy leads the users to traverse the dataset, selectively subdivide and inspect segments of their choice

33 Multilevel Segmentation Hierarchy
Any cut guarantees complete coverage View-dependent LoD hierarchies [Xia & Varshney Vis 96, Hoppe SIGGRAPH 97, Luebke & Erikson SIGGRAPH 97] Any cut along this hierarchy guarantees complete coverage. This is similar to view dependent level of details hierarchy

34 Multilevel Segmentation Hierarchy
Having this flexibility, the next question is which segment should we subdivide?

35 Information Guided Traversal
Segment entropy High entropy ⇒ Complex segment We evaluate information content of the segments to provide some guidance We use red to show which segment has a higher entropy for example the entropy of the tooth is higher than the volume box, suggesting the tooth is more complex, we subdivide the tooth. Entropy 6.8 2.8

36 What if the Entropies are Similar…
The segment entropies can be similar Which segment should we divide next? Use Information Gain Entropy But, It is possible that the segment entropies are similar. For example there is no difference for the 2 segments in the right arc. Which segment should we divide next ? We use info gain to answer this question

37 Information-Gain Guided Traversal
Information Gain = Entropy reduction after a subdivision High Information Gain ⇒ Structural separation we alternatively evaluate the information gain of a subdivision The information gain is defined as the entropy reduction of a subdivision Subdivisions with high information gain suggest separations of structures. We use blue to represent the information gain, for example splitting the lower segment with high information gain separates the surfaces of the tooth crown and dentine 0.01 Information Gain 0.11

38 Interactive Exploration
Explore the segmentation hierarchy Selective expansion Interactive visualizations Exhaustively explore the tooth in 1 minute Here we show how a user can interactively explore the segmentation hierarchy, the user selectively click on the histogram to expand the segments and interactively compose visualizations Notice that the tooth crown in the upper arc can be identified in only 3 expansions. Users iteratively expand the regions of interest and hide uninteresting regions. A visualization of the volume dataset is composed along this hierarchy traversal In addition to identifying different meaningful segments, users can also segment and hide the undesired noise In contrast with extracting a single region. We can now exhaustively explore the whole tooth in 1 min.

39 Examples Engine block Visible Human Male Head
Let’s take a look at some results Internal components Engine block and surface Skull, flesh, skin, highlight sinus and teeth

40 Examples Tomato Hurricane Isabel Tomato Skin, Chamber, Center Column
Isabel pressure, with no distinct material boundaries Eye, Eye Wall, Rain band, and further segment the eye

41 Conclusions Computational segmentation mimics user interactions Intuitive volumetric classification Exhaustive multilevel hierarchy Information guided traversal Interactive exploration In conclusion, we have shown the idea of using computational visual segmentation to effectively mimic user interaction. It classifies intuitive volumetric regions. We have built an exhaustive multilevel hierarchy from these segments and We have provided information content to guide the hierarchy traversal Users can interactively explore and visualize with these ingredients.

42 Future Work Improve the information content measures Automatic color assignment for segments Segment histograms with different attributes Time varying datasets

43 Acknowledgements Thank you!
National Science Foundation: CCF , CMMI , CNS NVIDIA CUDA Center of Excellence Program Derek Juba, Sujal Bista, Yang Yang, M. Adil Yalcin, and the reviewers for improving this paper and presentation The SciVis Best Paper Award committee Thank you!

44 Questions ? Source code for building the hierarchy: Papers and videos: Cheuk Yiu Ip GVIL Research Highlights


Download ppt "Cheuk Yiu Ip Amitabh Varshney Joseph JaJa"

Similar presentations

Ads by Google