1. Roee Litman, Alexander Bronstein, Michael Bronstein
Diffusion-geometric maximally stable component detection in deformable shapes
Diffusion-geometric maximally stable component detection in deformable shapes
Roee Litman, Alexander Bronstein, Michael Bronstein

2. MSER In a nutshell… Shape Diffusion Geometry MSER The Feature Approach
for Images
Deformable Shape Analysis
MSER
Maximally Stable Extremal Region
Shape
MSER
Diffusion Geometry

3. Feature Approach in Images
Feature based methods are the infrastructure laid in the base of many computer vision algorithms:
Content-based image retrieval
Video tracking
Panorama alignment
3D reconstruction form stereo

4. Problem formulation Find a semi-local feature detector
High repeatability
Invariance to isometric deformation
Robustness to noise, sampling, etc.
Add discriminative descriptor

5. The “what”
Results

6. Visual Example

7. Visual Example

8. (Taken from the TOSCA dataset)
More Results
(Taken from the TOSCA dataset)
- part human, part horse
- partial matching
- how to match?
Horse regions +
Human regions

9. 1st, 2nd, 4th, 10th, and 15th matches
Region Matching
-now let's test it on real data
Query
1st, 2nd, 4th, 10th, and 15th matches

10. 3D Human Scans
Taken from the SCAPE dataset

11. Scanned Region Matching
Query
1st, 2nd, 4th, 10th, and 15th matches

12. Volume & surface isometry
Volume vs. Surface
Original
Volume & surface isometry
Boundary isometry

13. Volumetric Shapes
Usually shapes are modeled as 2D boundary of a 3D shape.
Volumetric shape model better captures "natural" behavior of non-rigid deformations. (Raviv et-al)
Diffusion geometry terms can easily be applied to volumes
2D Meshes can be voxelized

14. Volumetric Regions
Taken from the SCAPE dataset

15. (The “how”)
Methodology

16. Original MSER (Matas et-al)
Show some epipolar lines :)

17. MSER Popular image blob detector Near-linear complexity:
High repeatability [Mikolajczyk et al. 05]
Robust to affine transformation and illumination changes

18. MSER – In a nutshell Threshold image at consecutive gray-levels
Search regions whose area stay nearly the same through a wide range of thresholds
Efficient detection of maximally stable regions requires construction of a component tree

19. MSER – In a nutshell
SKIP?

20. Algorithm overview

21. Algorithm overview Represent as weighted graph Component tree
Stable component detection
Represent as weighted graph
Component tree
Stable component detection

22. Algorithm overview Represent as weighted graph Component tree
Stable component detection
Represent as weighted graph

23. Image as weighted graph
An undirected graph can be created from an image, where:
Vertices are pixels
Edges by adjacency rule, e.g. 4-neiborhood

24. Weighting the graph In images Gray-scale as vertex-weight
Color as edge-weight [Forssen]
In Shapes
Curvature (not deformation invariant)
Diffusion Geometry

25. Weighting Option For every point on the shape:
Calculate the prob. of a random walk to return to the same point.
Similar to Gaussian curvature
Intrinsic, i.e. – deformation invariant

26. Weight example
Color-mapped
Level-set animation

27. Diffusion Geometry Analysis of diffusion (random walk) processes
Governed by the heat equation
Solution is heat distribution at point at time

28. Heat-Kernel Given Solve using: i.e. - find the “heat-kernel”
Initial condition
Boundary condition, if these’s a boundary
Solve using:
i.e. - find the “heat-kernel”

29. Probabilistic Interpretation
The probability density for a transition
by random walk of length ,
from to

30. Spectral Interpretation
How to calculate ?
Heat kernel can be calculated directly from eigen-decomposition of the Laplacain
By spectral decomposition theorem:

31. Laplace-Beltrami Eigenfunctions

32. Deformation Invariance

33. Computational aspects
Shapes are discretized as triangular meshes
Can be expressed as undirected graph
Heat kernel & eigenfunctions are vectors
Discrete Laplace-Beltrami operator
Several weight schemes for
is usually discrete area elements

34. Computational aspects
In matrix notation
Solve eigendecomposition problem

35. Scale Space
The time parameter of the heat kernel spans different scales of transition length
is not invariant to shape’s scale
Commute-time kernel - scale invariant
Probability of a transition by random walk of any length

36. Auto-diffusivity Special case - The chance of returning to after time
Related to Gaussian curvature by
Now we can attach scalar value to shapes!

37. Weight example
Color-mapped
Level-set animation

38. Algorithm overview Represent as weighted graph Component tree
Stable component detection
Represent as weighted graph
Component tree
Stable component detection

39. The Component Tree
Tree construction is a pre-process of stable region detection
Contains level-set hierarchy, i.e. nesting relations.
Constructed based on a weighted graph (vertex- or edge-weight)
Tree’s nodes are level-sets (of the graph’s cross-sections)

40. Tree Example

41. Preliminaries - 1/3
We focus on the undirected graph with the vertex set and edge set , denoted
Vertices and are adjacent if
The ordered sequence is a path if every consecutive pair is adjacent
and are linked
A graph is connected if every vertex pair is linked

42. Preliminaries - 2/3 A graph is called a sub-graph of
Given , we take only vertices belonging to an edge
A graph is a (connected) component of if it is a maximal connected sub-graph

43. Preliminaries - 3/3 Vertex-weighted graph - Edge-weighted graph -
A cross-section is a sub-graph of a weighted graph, with all weights
Level-set is a (connected) component of a cross-section
Altitude of a level-set is the maximal weight it contains (can be smaller than )
vertex weight are scalar only
edge weight are more general

44. “Graphic” Example A graph Edge-weighted 7 Cross-Section
Two 5 level-sets (with altitude 4)
Every level-set has
Size (area)
Altitude (maximal weight) 4 7 8 9 8 1 4 1

45. Tree Construction Iterate over vertices by order of weight
Create a new component from vertex
If vertex is adjacent to existing component(s)
add exiting component’s vertices to new one
Store component’s area & weight

46. Tree Construction 1 4 7 8 9 4 7 8 9 8 1 4 1

47. Algorithm overview Represent as weighted graph Component tree
Stable component detection
Represent as weighted graph
Component tree
Stable component detection

48. Detection Process For every leaf component in the tree:
“Climb” the tree to its root, creating the sequence:
Calculate component stability
Local maxima of the sequence are “Maximally stable components”

49. The Detils
Performance

50. Benchmarking The Method
Method was tested on SHREC 2010 data-set:
3 basic shapes (human, dog & horse)
9 transformations, applied in 5 different strengths
138 shapes in total
Scale
Original
Deformation
Holes
Noise

51. Results

52. Quantitative Results Vertex-wise correspondences were given
Regions were projected onto another shape, and overlap ratio was measured
Overlap ratio between a region and its projected counterpart is
Repeatability is the percent of regions with overlap above a threshold

53. Repeatability
65% at 0.75

54. Conclusion Stable region detector for deformable shapes
Generic detection framework:
Vertex- and edge-weighted graph representation
Works on surface and/or volume data
Partial matching & retrieval potential
Tested quantitatively (on SHREC10)

55. Thank You
Any Questions?