1 Numerical geometry of non-rigid shapes Partial similarity Partial similarity © Alexander & Michael Bronstein, © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Advanced topics in vision Processing and Analysis of Geometric Shapes EE Technion, Spring 2010

2 Numerical geometry of non-rigid shapes Partial similarity Greek mythology

3 Numerical geometry of non-rigid shapes Partial similarity I am a centaur.Am I human?Am I equine? Yes, I’m partially human. Yes, I’m partially equine. Partial similarity Partial similarity is a non-metric relation

4 Numerical geometry of non-rigid shapes Partial similarity Human vision example ?

5 Numerical geometry of non-rigid shapes Partial similarity Visual agnosia Oliver Sacks The man who mistook his wife for a hat

6 Numerical geometry of non-rigid shapes Partial similarity Recognition by parts Divide the shapes into parts Compare each part separately using a full similarity criterion Merge the part similarities

7 Numerical geometry of non-rigid shapes Partial similarity Partial similarity X and Y are partially similar = X and Y have significantsimilar parts Illustration: Herluf Bidstrup

8 Numerical geometry of non-rigid shapes Partial similarity Ingredients Similar Dissimilar Part = subset of the shape Similarity = full similarity applied to parts

9 Numerical geometry of non-rigid shapes Partial similarity Significance Significance = measure of the part Area: size of the part SignificantInsignificant

10 Numerical geometry of non-rigid shapes Partial similarity Significance Problem: how to select significant parts? Significance of a part is a semantic definition Different shapes may have different definition of significance

11 Numerical geometry of non-rigid shapes Partial similarity Significance Do all parts have the same importance? It really depends on the data High Gaussian curvature High variation of curvature “features”

12 Numerical geometry of non-rigid shapes Partial similarity Statistical significance A. Bronstein, M. Bronstein, Y. Carmon, R. Kimmel, CVA 2009 syzygy in astronomy means alignment of three bodies of the solar system along a straight or nearly straight line. a planet is in syzygy with the earth and sun when it is in opposition or conjunction. the moon is in syzygy with the earth and sun when it is new or full. inis theor Query qDatabase D Frequent = important Frequent = non- discriminative syzygy Rare = discriminative Significance of a term t Term frequencyInverse document frequency

13 Numerical geometry of non-rigid shapes Partial similarity Numerical example A. Bronstein, M. Bronstein, Y. Carmon, R. Kimmel, CVA 2009

14 Numerical geometry of non-rigid shapes Partial similarity Multicriterion optimization Simultaneously minimize dissimilarity and insignificance over all the possible pairs of parts This type of problems is called multicriterion optimization Vector objective function How to solve a multicriterion optimization problem?

15 Numerical geometry of non-rigid shapes Partial similarity Pareto optimality Traditional optimizationMulticriterion optimization A solution is said to be a global optimum of an optimization problem if there is no other such that A solution is said to be a Pareto optimum of a multicriterion optimization problem if there is no other such that Optimum is a solution such that there is no other better solution In multicriterion case, better = all the criteria are better

16 Numerical geometry of non-rigid shapes Partial similarity Pareto frontier Vilfredo Pareto ( ) INSIGNIFICANCE DISSIMILARITY UTOPIA Attainable set Pareto frontier

17 Numerical geometry of non-rigid shapes Partial similarity Set-valued partial similarity The entire Pareto frontier is a set-valued distance The dissimilarity at coincides with full similarity

18 Numerical geometry of non-rigid shapes Partial similarity Scalar- vs. set-valued similarity INSIGNIFICANCE DISSIMILARITY Use intrinsic similarity as

19 Numerical geometry of non-rigid shapes Partial similarity Is a crocodile green or ? What is better: (1,1) or (0.5,0.5)? What is better: (1,0.5) or (0.5,1)? Order relations

20 Numerical geometry of non-rigid shapes Partial similarity Order relations There is no total order relation between vectors As a result, two Pareto frontiers can be non-commeasurable INSIGNIFICANCE DISSIMILARITY BLUE < RED GREEN ? RED

21 Numerical geometry of non-rigid shapes Partial similarity Scalar-valued partial similarity In order to compare set-valued partial similarities, they should be scalarized Fix a value of insignificance INSIGNIFICANCE DISSIMILARITY Can be written as using Lagrange multiplier Define scalar-valued partial similarity

22 Numerical geometry of non-rigid shapes Partial similarity Scalar-valued partial similarity INSIGNIFICANCE DISSIMILARITY Fix a value of dissimilarity Distance from utopia point Area under the curve

23 Numerical geometry of non-rigid shapes Partial similarity Characteristic functions Problem: optimization over all possible parts A part is a subset of the shape Can be represented using a characteristic function Still a problem: optimization over binary-valued variables (combinatorial optimization)

24 Numerical geometry of non-rigid shapes Partial similarity Fuzzy sets Relax the values of the characteristic function to the continuous range The value of is the membership of the point in the subset is called a fuzzy set

25 Numerical geometry of non-rigid shapes Partial similarity Fuzzy partial similarity Compute partial similarity using fuzzy parts Fuzzy insignificance Fuzzy dissimilarity where is the weighted distortion

26 Numerical geometry of non-rigid shapes Partial similarity Numerical optimization GMDS problem: A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

27 Numerical geometry of non-rigid shapes Partial similarity Numerical optimization Freeze parts and solve for correspondence Freeze correspondence and solve for parts 1 2 A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

28 Numerical geometry of non-rigid shapes Partial similarity Example of intrinsic partial similarity Partial similarity Full similarity Scalar partial similarity Human Human- like Horse-like A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

29 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity How to make ICP compare partially similar shapes? Introduce weights into the shape-to-shape distance to reject points with bad correspondence Possible selection of weights: where is a threshold on normals is a threshold on distance and are the normals to shapes and

30 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Without rejectionWith rejection

31 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Parts obtained by matching of man and centaur using ICP with rejection for different thresholds Problem: there is no explicit influence of the rejection thresholds and the size of the resulting parts Pareto framework allows to control the size of the selected parts!

32 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Correspondence stage (fixed and ) Closest point correspondence Weighted rigid alignment Correspondence

33 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Part selection stage (fixed and )

34 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Controllable part size by changing the value of

35 Numerical geometry of non-rigid shapes Partial similarity Not only size matters What is better?... Many small parts……or one large part? A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

36 Numerical geometry of non-rigid shapes Partial similarity Regularity Irregular = long boundaryRegular = short boundary Shape factor (circularity) A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

37 Numerical geometry of non-rigid shapes Partial similarity Regularity Irregular = four connected components Regular = one connected component Equal shape factor! Topological regularity A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

38 Numerical geometry of non-rigid shapes Partial similarity Mumford-Shah functional Salvation comes from image segmentation [Mumford&Shah]: given image, find the segmented region replace by a membership function The two problems are equivalent

39 Numerical geometry of non-rigid shapes Partial similarity Mumford-Shah functional In our problem, we need only the integral along the boundary which becomes in the fuzzy setting

40 Numerical geometry of non-rigid shapes Partial similarity Numerical example INSIGNIFICANCE DISSIMILARITY IRREGULARITY A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

41 Numerical geometry of non-rigid shapes Partial similarity ICP example Significance Regularity