SPONSORED BY Data-driven Segmentation and Labeling of Freehand Sketches Zhe Huang, Hongbo Fu, Rynson W.H. Lau City University of Hong Kong

SA2014.SIGGRAPH.ORG SPONSORED BY Objective Derive part-based segmentation and labeling from a sketch.

SA2014.SIGGRAPH.ORG SPONSORED BY Motivation Assembly-based modeling using sketch input [Funkhouser et al. 2004]

SA2014.SIGGRAPH.ORG SPONSORED BY Related Works [Sun et al. 2012] Free Hand-Drawn Sketch Segmentation [Gennari et al. 2005] Combining geometry and domain knowledge to interpret hand-drawn diagrams [Noris et al. 2012] Smart Scribbles for Sketch Segmentation

SA2014.SIGGRAPH.ORG SPONSORED BY Handling Freehand Sketch is Hard Arbitrary stroke ordering A single stroke may cover multiple components. A single component may have multiple strokes.

SA2014.SIGGRAPH.ORG SPONSORED BY Handling Freehand Sketch is Hard Where is the chair back?

SA2014.SIGGRAPH.ORG SPONSORED BY Our Method

SA2014.SIGGRAPH.ORG SPONSORED BY Our Intuition Find the best combination of components to explain the sketch Leverage the relationship between parts to narrow down the search.

SA2014.SIGGRAPH.ORG SPONSORED BY Overview of Our Method Inferred Structure Sketch Labeling Input Sketch (Chair) Local Interpretation (Matching) Global Interpretation (Combinatorial Search) Shape Repository (Chair)

SA2014.SIGGRAPH.ORG SPONSORED BY Preprocessing

SA2014.SIGGRAPH.ORG SPONSORED BY Specifying 3D View Sketch Manually specified 3D view by aligning a 3D model User Interaction

SA2014.SIGGRAPH.ORG SPONSORED BY Over Segmentation Sketch (Color by strokes) Breaking at junction or corner Stroke Segments (Color by segments) In later stages: Segmentation = grouping stroke segments Labeling = labeling each stroke segment

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Candidates

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Convolution Score Map

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Local Maximums

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Clustered

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Cluster Centers

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() ICP Refinement before after Cluster Centers

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() ICP Refinement before after Final Candidates Cluster Centers

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Sketch Element Outline conv() Match

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Sketch Element Outline Match All Candidates All chair backs

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Sketch Element Outline Match Filtered Candidates All chair backs

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation

SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Possible Connections

SA2014.SIGGRAPH.ORG SPONSORED BY Global Interpretation

SA2014.SIGGRAPH.ORG SPONSORED BY Formulation Problem: Find an assignment to X and Y such that the cost E(X,Y) is minimized. Note: In the paper, the cost function is E(X,Y,U), where U is stroke segment assignment variable  i-th candidate is selected  i-th and j-th candidates are connected Decision Variables:

SA2014.SIGGRAPH.ORG SPONSORED BY Cost Function Fitness Connection is better than because

SA2014.SIGGRAPH.ORG SPONSORED BY Cost Function Overlap Coverage Others… (see the paper for detail) is better thanbecause is better than because Overlap No coverage

SA2014.SIGGRAPH.ORG SPONSORED BY Results and Evaluation

SA2014.SIGGRAPH.ORG SPONSORED BY Results Ours Direct Retrieval [Xu et al. 2011] [Eitz et al. 2012] [Shen et al. 2012]

SA2014.SIGGRAPH.ORG SPONSORED BY Quantitative Evaluation 10 classes, 300 sketches in total

SA2014.SIGGRAPH.ORG SPONSORED BY Quantitative Evaluation 10 classes, 300 sketches in total

SA2014.SIGGRAPH.ORG SPONSORED BY Application 3D reconstruction if the labeling is perfect.

SA2014.SIGGRAPH.ORG SPONSORED BY Limitation Incorrect local interpretation. Hollowness ambiguity Missing small elements.

SA2014.SIGGRAPH.ORG SPONSORED BY Thank You