1. Linear Solution to Scale and Rotation Invariant Object Matching Professor: 王聖智 教授 Student : 周 節

2. Outline  Introduction  Method  Result  Conclusion  Reference

3. Problem Template image Target image 3

4. Problem Template image and mesh Target image Matching Target Mesh

5. Challenges Template Image Target Images

6. Challenges Template Image Target Images

7. :wRelated Works  Hough Transform and RANSAC  Graph matching  Dynamic Programming  Max flow – min cut [Ishikawa 2000, Roy 98]  Greedy schemes (ICM [Besag 86], Relaxation Labeling [Rosenfeld 76])  Back tracking with heuristics [Grimson 1988]  Graph Cuts [Boykov & Zabih 2001]  Belief Propagation [Pearl 88, Weiss 2001]  Convex approximation [Berg 2005, Jiang 2007, etc.]

8. Method 1.A linear method to optimize the matching from template to target 2.The linear approximation is simplified so that its size is largely decoupled from the number of target candidate features points 3.Successive refinement for accurate matching

9. Method 1.A linear method to optimize the matching from template to target 2.The linear approximation is simplified so that its size is largely decoupled from the number of target candidate features points 3.Successive refinement for accurate matching

10. The Optimization Problem

11. Matching costPairwise consistency

12. In Compact Matrix Form Matching costPairwise consistency

13. In Compact Matrix Form We will turn the terms in circles to linear approximations Matching costPairwise consistency

14. The L1 Norm Linearization  It is well known that L1 norm is “linear” min |x|min (y + z) Subject to x = y - z y, z >= 0

15. The L1 Norm Linearization  It is well known that L1 norm is “linear”  By using two auxiliary matrices Y and Z, | EMR – sEXT |1’ n e ( Y + Z ) 1 2 subject to Y – Z = EMR – sEXT Y, Z >= 0 Consistency term min |x|min (y + z) Subject to x = y - z y, z >= 0

16. Linearize the Scale Term For any given s Cut into Ns part 0 <S1<...< S ns

17. Linearize the Scale Term For any given s Cut into Ns part 0 <S1<...< S ns For any given X X1, X2…X ns

18. Linearize the Scale Term For any given s Cut into Ns part 0 <S1<...< S ns For any given X X1, X2…X ns

19. Linearize the Scale Term For any given s Cut into Ns part 0 <S1<...< S ns For any given X X1, X2…X ns

20. Linearize the Scale Term All sites select the same scale

21. Linearize Rotation Matrix In graphics:

22. The Linear Optimization T* = XT Target point estimation:

23. Method 1.A linear method to optimize the matching from template to target 2.The linear approximation is simplified so that its size is largely decoupled from the number of target candidate features points 3.Successive refinement for accurate matching

24. Convex optimization problem 24 Reference: http://www.stanford.edu/class/ee364a/lectures/problems.pdf

25. The Lower Convex Hull Property  Cost surfaces can be replaced by their lower convex hulls without changing the LP solution A cost “surface” for one point on the template The lower convex hull

26. Removing Unnecessary Variables  Only the variables that correspond to the lower convex hull vertices need to be involved in the optimization # of original target candidates: 2500 # of effective target candidates: 19

27. Complexity of the LP 28 effective target candidates1034 effective target candidates 10,000 target candidates1 million target candidates The size of the LP is largely decoupled from the number of target candidates

28. Method 1.A linear method to optimize the matching from template to target 2.The linear approximation is simplified so that its size is largely decoupled from the number of target candidate features points 3.Successive refinement for accurate matching

29. Successive Refinement

30. Result

31. Matching Objects in Real Images

32. Result Videos

33. Statistics bookmagazinebearbutterflybeefish #frames856601 771101131 #model151409235124206130 #target214317241683140510297316 time1.6s11s2.2s1s2s0.9s accuracy99%97%88%95%79%95%

34. Results Comparison

35. Accuracy and Efficiency

36. Experiments on Ground Truth Data Error distributions for fish dataset The proposed method Other methods

37. Experiments on Ground Truth Data Error distributions for random dot dataset The proposed method Other methods

38. Conclusion  A linear method to solve scale and rotation invariant matching problems accurately and efficiently  The proposed method is flexible and can be used to match images using different features  It is useful for many applications including object detection, tracking and activity recognition

39. Reference  Hao Jiang and Stella X. Yu, “Linear Solution to Scale and Rotation Invariant Object Matching”, IEEE Conference on Computer Vision and Pattern Recognition 2009 (CVPR'09)  http://www.stanford.edu/class/ee364a/lectures.html