Presentation on theme: "Cheuk Yiu Ip Amitabh Varshney Joseph JaJa"— Presentation transcript:
1 Cheuk Yiu Ip Amitabh Varshney Joseph JaJa Hierarchical Exploration of Volumes Using Multilevel Segmentation of the Intensity-Gradient HistogramsCheuk Yiu Ip Amitabh Varshney Joseph JaJa
2 Volume Exploration Challenge Raw Volume Data CubeMeaningful Visualization[Voreen CG&A 09]Similar to other papers in this sessionGiven a volumetric data cube on the left. We want to generate a meaningful visualization on the right.
3 Transfer Function Evolution How do we find the “right” transfer function?RGBATraditionally, we use a transfer function to control the appearance of a visualization, the transfer function may look pretty complex.Finding the right transfer function has always been a challenge in scientific visualizationIntensity
4 Histogram Helps Transfer Function Design To assist transfer function design, one of the first volume rendering papers shows a histogram may be helpful.[Drebin et al. SIGGRAPH 88]
5 Histogram Helps Transfer Function Design The peaks of the histogram are composed by the distributions of different constituents[Drebin et al. SIGGRAPH 88]
6 Histogram Helps Transfer Function Design By assigning different colors and opacities to these components. we can visualize different materials[Drebin et al. SIGGRAPH 88]
7 Histogram Helps Transfer Function Design For example, by following the peak in the histogram we can obtain this visualization of the cadaver head dataset
8 Histograms May Not Always Help However, the separation of the distributions may not be always clear
9 Volume Exploration Exhaustively explore the dataset In this case, we have to exhaustively explore the dataset by modifying the transfer function
10 Volume Exploration Seek for salient features We seek for the salient features one by one
13 Volume Exploration Feature locations can be arbitrary The meaningful feature locations can be pretty arbitrary
14 2D Transfer FunctionsGradient ( f’(x) ) captures boundaries Histogram shapes ⇒ Volume segmentsTo assist the search for meaningful and salient features, the transfer function has been extended to 2D by incorporating gradient.We plot a histogram in 2D with the intensity on the x axis, gradient on the y axis, the darkness of the shade represents the frequencyThe gradient helps capturing material boundaries.The amazing insight here is shapes on the 2D intensity-gradient histogram correspond to meaningful volumetric segments.This has been implemented in popular visualization packages such as voreen Imagevis3d and VisIt[Kindlmann & Durkin VolVis98,Kniss et al. TVCG 02]GradientIntensity
15 Advances on New Attributes Higher Derivatives: f”(x) [Kindlmann & Durkin VolVis98] Specific features: Size [Correa and Ma TVCG 08] LH-transform [Sereda et al. TVCG 06], Domain specific semantic attributes [Salama et al. TVCG 06] Select good views: Visibility [Correa and Ma TVCG 10], Information divergence [Ruiz et al. TVCG 11]Many subsequent work has introduced new attributes such as higher derivatives, size, LH Transform, and even domain specific semantic attributesVisibility and information divergence have been used to select good views.For this work we will go back to intensity and gradient
16 Challenges of the 2D Transfer Function Search for separated meaningful features1 Region 1 MinuteLet’s take a look at the use of a 2D transfer functionUsers draw shapes on the 2D histogram to extract meaningful componentsThis video shows how a user works with the histogram in ImageVis3DThe arc structures can clearly be seen on the intensity gradient histogramThe user is trying to use a polygon widget to highlight the feature that corresponds to the top arc.As we can see, manipulating the widget to separate the crown from the root takes quite some effort.The challenge here is the histogram does not show where do the good features separate.Extracting this crown takes almost 1 min
17 Histogram and Volume Features Let’s take a look at the vismale dataset
22 Histogram and Volume Features TeethWe can see that meaningful components do correspond to histogram segments.In this talk, my goal is to show you how to discover these regions in a systematic manner.
23 Approximate Histogram Transfer Functions Existing approaches directly or indirectly fit the histogramUser Specified [Kniss et al. TVCG 02,Fogal et al. 2010][Wang et al. TVCG 2011]Existing approaches directly or indirectly fit the 2D histogram to assist transfer function designThese primitives may not cover the whole histogram completely to guarantee exhaustive exploration
24 Reduce Search to Classification Recursive histogram classification Tight coverage with a few segments Exhaustive explorationIn this work, we attempt to reduce this search to classification by recursively segmenting the histogram.We introduce a visual-inspired approach that tightly fits the histogram with a few segments.We will show how to interactively and exhaustively explore the whole dataset
25 Overview Segment the histogram statistics Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume,we build an exhaustive multilevel hierarchyand use it to provide interactive exploration.Our approach provides a few advantagesVisual segmentation matches user intuition,we guarantee tight and complete coverage,Our approach augments the intensity-gradient feature space without requiring the users to learn any new featuresSegment the histogram statisticsBuild an exhaustive multilevel hierarchyUser interactive exploration
26 OverviewGiven a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume,we build an exhaustive multilevel hierarchyand use it to provide interactive exploration.Our approach provides a few advantagesVisual segmentation matches user intuition,we guarantee tight and complete coverage,Our approach augments the intensity-gradient feature space without requiring the users to learn any new featuresVisual segmentation matches user intuition Complete coverage Users stay in a familiar feature space
27 Intensity-gradient Histogram The intensity gradient histogram is very effective becauseHuman users can extract volume segments by recognizing the arcs and blobsGiven these histograms, can we extract these shapes automatically?We mimic this user behavior by applying computer vision techniques to segment the histogram as an image in a top-down fashionIntensityUsers implicitly recognize shapes from the histogram Segment this histogram as an image
28 Normalized-cut Image Segmentation Normalized-cut (ncut) image segmentation [Shi & Malik PAMI 98] Min ncut produces balanced segments Eigenanalysis approximates the min ncut for k segmentsBInstead of directly segmenting the volume data.We segment the histogram by using normalized-cut to find shapesThis example shows the normalized cut provides intuitive segments on natural images.Normalize cut models the image as a graph, with the similarly of pixels as edge weightsThe weight of a cut in the graph is the total weights along the separating edges.shi and malik normalize this cut by the weights of the associated segments in the graph.The minimum normalize cut favors balanced segmentsWe find the top k eigenvectors to approximate this minimum cut for k different segments[Wang et al. PATTERN RECOGN LETT 06]
29 Normalized-cut on Intensity-gradient Histogram We apply normalized-cut on bit histograms k=2 separates the tooth from the volume box k=10 shows segments of the tooth crown and root k=20 shows different material boundariesWe apply normalized cut to segment 256 sq 8 bit histogram images.Eigenanalysis on these images only takes a few secondsWhen k = 2 the tooth is separated from the volume boxk=10 shows segments of the tooth crown and rootk = 20 shows different material boundaries. These segments tightly cover the whole histogram.So how many segments should we choose to compose our visualization?
30 Which k should we pick ?Iteratively picking k is tedious Increasing k may not subdivide region of user interestTry k = 2, 3, 4, …Iteratively test and pick k is time consuming and yet increasing k may not subdivide regions of interest.For example when we increase k from 4 to 5 , the box is subdivided. We believe users would prefer subdividing the tooth.
31 Replace k with User-driven Exploration Multilevel Segmentation Hierarchy: Apply normalized-cut recursivelywe need to allow users to select the appropriate segments instead of choosing a specific k,In order to accomplish this, we build a hierarchy by recursively applying the normalized cut.
32 Multilevel Segmentation Hierarchy Selectively inspect segments of choiceThis hierarchy leads the users to traverse the dataset, selectively subdivide and inspect segments of their choice
33 Multilevel Segmentation Hierarchy Any cut guarantees complete coverageView-dependent LoD hierarchies[Xia & Varshney Vis 96,Hoppe SIGGRAPH 97,Luebke & Erikson SIGGRAPH 97]Any cut along this hierarchy guarantees complete coverage.This is similar to view dependent level of details hierarchy
34 Multilevel Segmentation Hierarchy Having this flexibility, the next question is which segment should we subdivide?
35 Information Guided Traversal Segment entropy High entropy ⇒ Complex segmentWe evaluate information content of the segments to provide some guidanceWe use red to show which segment has a higher entropyfor example the entropy of the tooth is higher than the volume box, suggesting the tooth is more complex, we subdivide the tooth.Entropy6.82.8
36 What if the Entropies are Similar… The segment entropies can be similar Which segment should we divide next? Use Information GainEntropyBut, It is possible that the segment entropies are similar. For example there is no difference for the 2 segments in the right arc.Which segment should we divide next ?We use info gain to answer this question
37 Information-Gain Guided Traversal Information Gain = Entropy reduction after a subdivision High Information Gain ⇒ Structural separationwe alternatively evaluate the information gain of a subdivisionThe information gain is defined as the entropy reduction of a subdivisionSubdivisions with high information gain suggest separations of structures.We use blue to represent the information gain, for example splitting the lower segment with high information gain separates the surfaces of the tooth crown and dentine0.01InformationGain0.11
38 Interactive Exploration Explore the segmentation hierarchy Selective expansion Interactive visualizations Exhaustively explore the tooth in 1 minuteHere we show how a user can interactively explore the segmentation hierarchy, the user selectively click on the histogram to expand the segments and interactively compose visualizationsNotice that the tooth crown in the upper arc can be identified in only 3 expansions.Users iteratively expand the regions of interest and hide uninteresting regions.A visualization of the volume dataset is composed along this hierarchy traversalIn addition to identifying different meaningful segments, users can also segment and hide the undesired noiseIn contrast with extracting a single region. We can now exhaustively explore the whole tooth in 1 min.
39 Examples Engine block Visible Human Male Head Let’s take a look at some resultsInternal componentsEngine block and surfaceSkull, flesh, skin, highlight sinus and teeth
40 Examples Tomato Hurricane Isabel Tomato Skin, Chamber, Center Column Isabel pressure, with no distinct material boundariesEye, Eye Wall, Rain band, and further segment the eye
41 ConclusionsComputational segmentation mimics user interactions Intuitive volumetric classification Exhaustive multilevel hierarchy Information guided traversal Interactive explorationIn conclusion, we have shown the idea of using computational visual segmentation to effectively mimic user interaction.It classifies intuitive volumetric regions.We have built an exhaustive multilevel hierarchy from these segmentsand We have provided information content to guide the hierarchy traversalUsers can interactively explore and visualize with these ingredients.
42 Future WorkImprove the information content measures Automatic color assignment for segments Segment histograms with different attributes Time varying datasets
43 Acknowledgements Thank you! National Science Foundation: CCF , CMMI , CNSNVIDIA CUDA Center of Excellence ProgramDerek Juba, Sujal Bista, Yang Yang, M. Adil Yalcin, and the reviewers for improving this paper and presentationThe SciVis Best Paper Award committeeThank you!
44 Questions ?Source code for building the hierarchy: Papers and videos: Cheuk Yiu Ip GVIL Research Highlights