1. Algorithms for 3D Isometric Shape Correspondence
Follow in slide show (F5) mode for the explanatory animations.
Yusuf Sahillioğlu
Computer Eng. Dept., Koç University, Istanbul, Turkey (PhD)
Computer Eng. Dept., METU, Ankara, Turkey (Asst. Prof.)

2. Problem Definition & Apps
Goal: Find a mapping between two shapes.
Shape interpolation, animation.
Attribute transfer.
Shape registration.
Images taken from (top to bottom):
M. Kilian, N. Mitra, and H. Pottman. Geometric modeling in shape space. Proc. SIGGRAPH, vol. 26, pp. 1–8, 2007.
R. W. Sumner and J. Popovic. Deformation transfer for triangle meshes. Proc. SIGGRAPH, vol. 23, no. 3, 2004.
W. Chang and M. Zwicker. Automatic registration for articulated shapes. Computer Graphics Forum, vol. 27, no. 5, pp. 1459–1468, 2008.
Y. Pekelny and C. Gotsman. Articulated object reconstruction and markerless motion capture from depth video. CGF, vol. 27, no. 2, pp. 399–408, 2008.
M. Hilaga, Y. Shinagawa, T. Kohmura, and T. Kunii. Topology matching for fully automatic similarity estimation of 3d shapes. Proc. SIGGRAPH, pp. 203–212, 2001.
forgotten
Registration/alignment: detect correspondences + deform (iterate until converge, i.e. source converges to the shape of target)
How to merge/stitch aligned data points: use marching cubes to extract isosurface, e.g. Hoppe’s thesis (fit tangent planes to k-nearest neighbors; that defines marching-grid-point to plane distance; so, we have a signed distance field attached to cubic grid; extract isosurface from there)
Time-varying: to extract surface of 3rd frame, set it as target, and all others as source one by one; once all frames are non-rigidly aligned to 3rd frame’s pose, we have plenty of unorganized 3d data representing 3rd frame (hopefully w/o missing parts ‘cos all other frames compensated); use isosurface extraction as described above.
Time-varying reconstruction.
Shape matching.
Statistical shape analysis.

3. Scope Correspondence algorithms for (nearly) isometric *
complete shapes.
partial shapes (part or most general).
CVPR: Y. Sahillioglu and Y. Yemez. 3D Shape Correspondence by Isometry-Driven Greedy Optimization, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp , 2010.
PAMI: Y. Sahillioglu and Y. Yemez. Minimum-Distortion Isometric Shape Correspondence Using EM Algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), to appear, 2012.
SGP: Y. Sahillioglu and Y. Yemez. Coarse-to-Fine Combinatorial Matching For Dense Isometric Shape Correspondence, Computer Graphics Forum (SGP’11), Vol. 30, No. 5, pp , 2011.
CGF: Y. Sahillioglu and Y. Yemez. Coarse-to-Fine Isometric Shape Correspondence by Tracking Symmetric Flips. Computer Graphics Forum Vol. 32, No. 1, pp , 2013.
PG: Y. Sahillioglu and Y. Yemez. Scale Normalization for Isometric Shape Matching. Computer Graphics Forum (Proc. PG), Vol. 31, No. 7, pp , 2012.
CGF: Y. Sahillioglu and Y. Yemez. Partial 3D Correspondence from Shape Extremities, Computer Graphics Forum, Vol. 33, No. 6, pp , 2014.
CVPR
PAMI
Coarse correspondences: PG
CGF
SGP
CGF
Dense correspondences.

4. Scope Correspondence algorithms for (nearly) isometric *
complete shapes collections. //not in thesis (done during post-doc).
PG: Y. Sahillioglu and Y. Yemez. Multiple Shape Correspondence by Dynamic Programming. Computer Graphics Forum (Proc. PG), Vol. 33, No. 7, pp , 2014.
Coarse correspondences: PG
Dense correspondences.

5. Scope Correspondence algorithms for (nearly) isometric *
Idea for multiple
shape correspondence:

6. Problems Complete shape correspondence at coarse resolution
joint sampling symmetric flips
dense resolution
timing
Partial shape correspondence
scale normalization outliers

7. All Algorithms in a Nutshell
Please see the Computational Complexity section of the PG’14 paper for the analysis of our Multiple Shape Correspondence algorithm.
V: # vertices in the original mesh, N: # samples at coarse resolution, M=5.

8. Contributions Sampling algorithms.
COES, coarse-to-fine, and two extremity sampling methods.
Isometric distortion without embedding.
Distortion minimization by well-established paradigms.
Graph matching, greedy optimization, EM algo, combinatorial optimization.
Map tracking to handle the symmetric flip problem.
Dense correspondence w/ the lowest time complexity.
Correspondences that are partial and dense at the same time.
Partial correspondence in the most general setting.
No restriction on topology and triangulation type.
Euclidean
embedding
Spherical embedding & GMDS images taken from:
C. Gotsman, X. Gu, and A. Sheffer. Fundamentals of spherical parameterization for 3D meshes. ACM Trans. Graph., Vol: 22/3, pp , 2003.
&
A. M. Bronstein, M. M. Bronstein, R. Kimmel. Calculus of non-rigid surfaces for geometry and texture manipulation. IEEE Trans.
Visualization and Computer Graphics, Vol 13/5, pp , 2007.

9. Global Similarity: Isometry
All of our methods are purely isometric.
Similar shapes have similar metric structures.
Metric: geodesic distance (in use) vs. diffusion-based distances.
Diffusion metric is less accurate than the geodesic metric but generally considered as more robust to topological noise as it considers all paths between two points unlike the geodesic that is based only on the shortest path.

10. Local Similarity: Descriptors
More consistent joint-sampling which helps matching.
Gaussian curvatures and average geodesic distances in use.
So, local shape descriptors help explicitly on joint-sampling and implicitly on matching, i.e., they never appear in the distortion functions to be minimized but appear on the selection of samples fed into these functions.

11. Scale Normalization
Scale normalization to prepare geodesic distances for upcoming isometric distortion computations.
Complete shapes (scale by max geodesic)
Partial shapes (max geodesic based normalization fails)
Partial shapes (scale by trusted matches)
Partial shapes (scale by Euclidean embedding, e.g., Möbius)

12. Isometric Distortion O(N2) for a map of size N.
Given , measure its isometric distortion:
in the most general setting.
: normalized geodesic distance b/w two vertices.
O(N2) for a map of size N.

13. Isometric Distortion Illustrationg g g g g g g
in action: g
average for

14. Scale-invariant Isometric Distortion
Given , measure its scale-inv. isometric distortion:
This measure based on raw geodesics provides few trusted matches to be used in scale normalization.
O(N3) for a map of size N.
Once scale normalization is done, more matches can be obtained. With more matches, problems in the initial few trusted matches, such as symmetric flips, can also be healed.
: unnormalized/raw geodesic distance b/w two vertices.

15. Scale-inv. Isometric Distortion Illustration
in action:
average for

16. Minimizing Isometric Distortion
Optimization by
Greedy (CVPR’10).
EM framework (PAMI).
Combinatorial in C2F fashion (SGP’11, CGF’13).
Rank-and-vote-and-conquer (CGF’14).
Combinatorial (PG’12).

17. Greedy Optimization Initialization by spectral embedding & alignment.
Refinement by greedy optimization.
Yusuf Sahillioğlu and Yücel Yemez, 3D Shape Correspondence by Isometry-Driven Greedy Optimization,
Proc. Computer Vision and Pattern Recognition (CVPR), pp , 2010.

18. EM Framework
Refine initial spectral correspondence even further in EM framework.
Minimization of the isometric distortion = Maximization of the log-likelihood function encoded in matrix Q: probability of matching source si to target tj.
Yusuf Sahillioğlu and Yücel Yemez, Minimum-Distortion Isometric Shape Correspondence Using EM Algorithm, PAMI, to appear, 2012.

19. EM Framework (Results)
Initial spectral correspondence refined (one-to-one and many-to-one maps).

20. EM Framework (Results)

21. EM Framework (Comparisons)
GMDS
Our method
Our method
Spectral
GMDS: A. M. Bronstein, M. M. Bronstein, R. Kimmel. Efficient computation of isometry-invariant distances between surfaces. SIAM J. Scientific Computing, 28, 5, (2006).
Spectral: V. Jain and H. Zhang. Robust 3d shape correspondence in the spectral domain. IEEE Int. Conf. on Shape Modeling and Applications (SMI), pp. 118–129, 2006.
Our method
Spectral
Spectral : worse worsts, missing salient points.
GMDS : clustered matches, missing salient pnts.

22. EM Framework (Limitations)
Mismatches due to lack of samples.
No efficient extension to dense correspondence due to cubic EM framework.
No caution for symmetric flips.
No support for partially isometric shapes.
Sufficient # of samples.
Limitation 1 handled by adjusting sampling distance parameter or in coarse-to-fine (C2F) fashion as proposed in SGP w/o any user interaction.
Limitation 2 handled by SGP which is less accurate than this in achieving sparse correspondences.
Limitation 3 handled by CGF extension of SGP.
Limitation 4 handled partially by PG and fully by SIGGRAPH Asia.

23. C2F Combinatorial Optimization
Optimal mapping maps nearby vertices in source to nearby vertices in target.
Recursively subdivide matched patches into smaller patches (C2F sampling) to be matched (combinatorial search).
That is combinatorial matching in a coarse-to-fine fashion.
Yusuf Sahillioğlu and Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape
Correspondence, Computer Graphics Forum (SGP), Vol. 30, No. 5, pp , 2011.

24. C2F Combinatorial Optimization
C2F sampling.
greens inherited from level k−1
blues are all vertices ( )
blacks + greens =
patches being defined ( )
Yusuf Sahillioğlu and Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape
Correspondence, Computer Graphics Forum (SGP), Vol. 30, No. 5, pp , 2011.

25. C2F Combinatorial Optimization
Combinatorial matching.
greens inherited from level k−1
blacks + greens =

26. C2F Combinatorial Optimization
Merging patch-to-patch correspondences into one correspondence over the whole surface.
Trim matches with diso > 2Diso, i.e., outliers.
Multi-graph  single graph. Also, diso values made available.
1st pass over source samples to keep only one match per sample, the one
with the min diso.
2nd pass over target samples to assign one match per isolated sample, the
one with the min diso.

27. C2F Combinatorial Optimization
Inclusion assertion for algorithm correctness.

28. C2F Combinatorial Optimization
Saliency sorting.
C2F sampling.
Restricted to the patch to be sampled, Dijkstra’s shortest paths takes:
K = log_M^N because M^K = N

29. C2F Combinatorial Optimization
Patch-based combinatorial matching.
because each pair is matched in time.
Merging.
Mk: size of the mapping at level k.
E: evenly-spaced subset of E (= 100) matches as .
3-step merging takes time.
diso computations

30. C2F Combinatorial Optimization
Overall
K = log_M^N because M^K = N

31. C2F Combinatorial Optimization (Results)
Details captured, smooth flow.
Many-to-one.
6K vs. 16K
red line: the worst match
w.r.t. isometric distortion.
Two meshes at different resolutions.

32. C2F Combinatorial Optimization (Results)
red line: the worst match w.r.t. isometric distortion.

33. C2F Combinatorial Optimization (Comparisons)
Nonrigid world dataset
GMDS O(N2logN)
Spectral O(N2logN)
Our method O(NlogN)
GMDS: A. M. Bronstein, M. M. Bronstein, R. Kimmel. Efficient computation of isometry-invariant distances between surfaces. SIAM J. Scientific Computing, 28, 5, (2006).
Spectral: V. Jain and H. Zhang. Robust 3d shape correspondence in the spectral domain. IEEE Int. Conf. on Shape Modeling and Applications (SMI), pp. 118–129, 2006.
Our method O(NlogN)

34. C2F Combinatorial Optimization (Limitation)
Symmetric flip problem.
Purely isometry-based methods naturally fail at symmetric inputs.
Due to multiple local minima of non-convex distortion function, our method initialized w/ coarse sampling may fail to find the true optimum.
Solution is based on map tracking.

35. C2F Combinatorial Optim. w/ Tracking
Track potential maps decided at level 0 until level 4 and maintain the best.
Yusuf Sahillioğlu and Yücel Yemez, Fast Dense Correspondence for Isometric Shapes,
Computer Graphics Forum (CGF), in revision cycle.

36. C2F Combinatorial Optim. w/ Tracking
Maps to be tracked are before the first jump in plot of initial distortions.

37. C2F Combinatorial Optim. w/ Tracking
In addition to addressing the symmetric problem inherent to all multiresolution isometric shape matchers, this extension is tested with
Five benchmarks (TOSCA, Watertight, SHREC’11, SCAPE, Nonrigid World), and
two state-of-the-arts (Blended Intrinsic Maps, GMDS).
Tracking is embedded in our C2F algorithm (SGP) as well as in GMDS.
Roughly speaking, 50% improvement on symmetric flips (see paper).
Final dense maps are better than or on a par with competitors regarding isometric distortions (see paper).
GMDS Our method
BIM Our method

38. Complete Correspondence Done
Complete shape correspondence at coarse (CVPR, PAMI) and dense (SGP) resolutions with special care on symmetric flip (CGF) for the latter is done.
Time to match partially similar shape pairs.
Algorithms naturally apply to complete matching.

39. Combinatorial Optim. for Part Matching
The most extreme M source vertices are matched w/ |T| target extremities in the guidance of an isometric distortion measure.
computational complexity where we set M=5 in the tests.
Yusuf Sahillioğlu and Yücel Yemez, Scale Normalization for Isometric Shape Matching,
Computer Graphics Forum (PG), submitted.

40. Combinatorial Optim. for Part Matching
Two isometric distortion measures in action.
Scale-invariant isometric distortion
Isometric distortion w/ normalized geodesics.
Winner

41. Combinatorial Optim. for Part Matching
Use initial coarse correspondence to bring the meshes to the same scale.
Scale the target mesh by
Dense sampling.
Same radius
100 here

42. Combinatorial Optim. for Part Matching
Dense matching.
Minimum-weight perfect matching on cost matrix C.
ci,j = cost of matching si to tj //generating is traversed by (si, tj).
Symmetric flip caring: repeat above (scaling, sampling, matching) with K-1 more generating initial coarse correspondences that follow in sorted distortions list.

43. Combinatorial Opt. for Part Matching (Results)
Not only for part matching
but also for complete matching
and for pairs w/ incompatible max geodesics.
Comparison w/ Möbius Voting (MV).
MV: bad extremity matching, triangulation.

44. Combinatorial Opt. for Part Matching (Limitation)
Presence of uncommon parts may fail this framework which forces to match M=5 most extremes as a whole.
Embedding into a more sophisticated framework should help as it handles arbitrary scaling of the similar parts.
Solution is our rank-and-vote-and-combine (RAVAC) algorithm.

45. RAVAC Optimization
Multiple common parts at arbitrary scales as well as uncommon parts.
Find sparse correspondence b/w shape extremes (green spheres) which will then be extended to a denser one.
Handles shape pairs w/ small similarity overlap (red regions), the smallest indeed to the best of our knowledge.

46. RAVAC Optimization Ranking
Explore the space of all possible partial maps b/w shape extremities to rank them w.r.t. the isometric distortion they yield.
Qualify matches w/ relatively low distortions, i.e., top-ranked.
Taking the minimum guarantees that if (si; tj) is a good match and traverses a list of matches from the same part it resides in, then its goodness is appreciated by using the lowest distortion.
Averaging over map sets of different cardinalities is also encouraged because small size maps, e.g., k = 2 or 3, are likely to fall in the same part as (si; tj) but may exhibit symmetric flip problems, whereas maps with big cardinalities, e.g., k = 4 or 5, are unlikely to be confused by flips but have the risk of including irrelevant samples from a distinct part.
: set of
all maps of size k, not including (si, tj).

47. RAVAC Optimization Voting
Qualified top-ranked matches analyzed at a denser reso to obtain confidences.
For each triplet of samples from source & target (potentially compatible greens)
Generate a safe map where all pairs are qualified.
Bring meshes to the same scale via
Decide the regions of interests.

48. RAVAC Optimization
Spread and match evenly-spaced dense samples on regions.
Add confidence votes to the generating matches that accumulate in
via where
Yet another example w/ a different generating pair of sample triplets.

49. RAVAC Optimization Combining
Iterate bipartite graph matching based on vote matrix by removing the least confident match at the end of each iteration.
Complete correspondence and part matching are handled naturally.
The harder case with uncommon parts.
Locally similar, globally not.

50. RAVAC Optimization Extension to dense map
For each map of size 3 chosen from optimal coarse correspondence, densely sample and match the regions as before (overlap trick in sampling).
: set of matches for dense source sample .
Geodesic centroid of is then which gives the
dense match where is a target vertex closest to .
In comparison w/ Möbius Voting (1st and 3rd pairs from the left).

51. RAVAC Optimization (Comparisons & Limitations)
More Möbius Voting comparisons.
Limitations
Each part to be matched must be represented by at least 3 samples, which is generally the case anyway.
Incorporate diffusion-based metrics for topological noise robustness.

52. Conclusions & Future Work
Four new sampling algorithms.
Isometric distortion functions and their optimizers in 3D Euclidean space.
The fastest computational complexity on dense correspondence.
Symmetric flip handling for all multiresolution isometric shape matchers.
Partial correspondence for shapes w/ significantly small similarity overlap.
Correspondences that are partial and dense at the same time.
Insensitivity to shape topology and peculiarities of the triangulation.
Investigate tradeoff b/w the accuracy of the geodesic metric in use and topological noise robustness of the diffusion-based metrics to be tested.
Incorporate more shapes into the process to establish or improve correspondences (Done during post-doc: [PG’14]).

53. People
Yusuf,
PhD student
Assoc. Prof. Yücel Yemez,
supervisor