1. Professor: S. J. Wang Student : Y. S. Wang
Object Recognition by Discriminative Combinations of Line Segments, Ellipses and Appearance Features
Professor: S. J. Wang
Student : Y. S. Wang

2. Outline Background System Overview Shape-Token
Code-Book of Shape-Token
Code-Word Combination
Hybrid Detector
Experimental Result
Conclusion
這是今天的outline，首先我會介紹為何會有這篇論文，這篇論文的目標，會遇到的困難，本篇論文所提出的方法。
本篇論文所reference到的方法，最後在介紹一些實驗結果，以及conclusion。

3. Background Contour Based Detection Method Problem of Contour Fragment:
Storage requirement is large for training.
Slow matching speed.
Not scale invariant.
Solution provided is Shape-Token.

4. System Overview

5. Shape Token What is Shape-Tokens? Constructing Shape-Tokens
Describing Shape-Tokens
Matching Shape-Tokens

6. What is Shape-Tokens?
Use the combination of line and ellipse to represent the contour fragments.
Line for line.
Ellipse for curve.
Example:
Why shape-tokens?
Several parameters are enough for us to describe the contour fragment.

7. Constructing Shape-Tokens
Extract Shape Primitives of line segments and ellipses by [16] [17].
Pairing reference primitive to its neighboring primitive.
Different type combination: Take ellipse as reference.
Same type combination: Consider each as reference in turn.
Three types of Shape-Tokens:
Line-Line, Ellipse-Line, Ellipse-Ellipse.

8. Constructing Shape-Tokens
Line-Line
Combine neighboring line which has any point falling in trapezium searching area.
Ellipse-Line & Ellipse-Ellipse
Circular Search Area. Consider primitives has any point within searching area and weakly is connected to reference ellipse.
Ellipse的neighbor需要兩個條件，第一個必須有point存在於search area裡，第二個是weakly connected的特性
此特性是指今天先把整張map都用前面提到的梯形方法來做相連形成LEM(line edge map)，然後某一線段可在LEM上找到一條path與橢圓相連接，則稱這條線滿足weak connectivity。
Indep. with orientation to avoid missing neighbors when pose of an object changes.

9. Describing Shape-Tokens
𝜽 : Orientation of a Primitive.
𝒗 𝒙 𝒗 𝒚 : Unit vector from center of reference primitive to center of its neighbor.
𝒉 : Distance between centers of primitives.
𝒍 𝒘 : Length and Width for each primitives.

10. Matching Shape-Tokens
Dissimilarity Measure (Shape Distance)

11. Matching Shape-Tokens
More general for multiple scale matching
Normalize descriptor against object scale 𝑏 𝑠

12. Codebook of Shape-Tokens
Extracting Shape-Tokens inside bounding boxes of training images.
Producing Code-words
Clustering by Shape
Clustering by Relative Positions
Selecting representative code-words into codebook for specific target object.

13. K-Medoid Method Similar to the k-means method. Procedure:
Randomly select k of the n data points as medoids.
Associate each data point to the closest medoid.
For each medoid m For each non-medoid data point o Swap m and o and compute the total cost of the configuration.
Select the configuration with the lowest cost.
Repeat the steps above until there is no change in the medoid.

14. K-Medoid Method
First two steps

15. K-Medoid Method
Third to Fourth step

16. Clustering by Shape Method:
Use k-medoid method to cluster the shape- tokens for each type separately.
Repeat the step above until the dissimilarity value for each cluster is lower then a specific threshold.
Metric:
Dissimilarity Value: average shape distance between the medoid and its members.
Threshold: 20% of the maximum of D(.).

17. Clustering by relative positions
Target:
Partition the clusters obtained from previous step by 𝑥 to attain sub-clusters whose members have similar shape and position relative to the centroid of object.
𝑥 : vector direct from object centroid to the shape-token centroid.
Method: Mean-Shift.

18. Candidate Code-Word 𝜑 Medoid for each sub-cluster. Parameters:
Shape Distance Threshold 𝜏 : Mean shape distance of the cluster plus one standard deviation.
Relative Position Center 𝑐 : Mean of vectors 𝑥 of the sub-clusters members.
Radius 𝑟 : Euclidean distance between 𝑐 to 𝑥 of each sub-cluster member plus one standard deviation.

19. Candidate Code-Words
Example: the Weizmann horse dataset.

20. Selecting Candidates into Codebook
Intuition: Size of cluster.
Problem: Lots of selected candidates belong to background clutter.
What kind of candidates we prefer ?
Distinctive Shape.
Flexible enough to accommodate intra-class variations.
Precise Location for its members.

21. Selecting Candidates into Codebook
Instead of using cluster size directly, the author scores each candidate by a product “𝑡” consists of three values.
Intra-cluster shape similarity value
“𝑑−𝜏” where 𝑑 is the maximum of the range of shape distance for the type of candidate currently considered.
The number of unique training bounding boxes its members are extracted from.
Its value of 1 𝑟 .

22. Selecting Candidates into Codebook
One more problem left:
If use 𝑡 to choose the candidate directly, it may cause not ideal spatial distribution.
Solution: Radial Ranking Method

23. Selecting Candidates into Codebook
Example: the Weizmann horse dataset.

24. Code-Word Combination
Why code-word combination ?
One can use a single code-word that is matched in test image to predict object location. => Less discriminative and easy to matched in background.
Instead, a combination of several code-words can be more discriminative.

25. Code-Word Combination
Matching a code-word combination
Way to match code-word combination.
Finding all matched code-word combinations in training images
Exhaustive set of code-word combinations.
Learning discriminative xCC (x-codeword combination)

26. Matching a Code-Word Combination
Criteria:
Shape Constraint : Shape distance between each code-word and shape-token in image should be less then shape distance threshold 𝜏 .
Geometric Constraint: Centroid prediction by all code-words in the combination concur.

27. Matching a Code-Word Combination
Example:

28. Finding all matched code-word combinations in training images
Goal: Finding an exhaustive set of possible candidates of code-word combinations.
Method: (Similar to Sliding-Window Search)
For each candidate window at scale 𝑠 and location 𝑥 in image I, we try to find there is any match for each code-word or not. And the combination of each matched code-word will be a possible combination candidate.

29. Finding all matched code-word combinations in training images
Specify a variable 𝑅 𝑖 ( 𝑠 , 𝑥 ) to represent the matching condition of a specific code- word 𝛾 𝑖 .

30. Finding all matched code-word combinations in training images
If 𝑅 𝑖 ( 𝑠 , 𝑥 )< ∞ ,then we say that the code-word 𝛾 𝑖 is matched at scale 𝑠 and location 𝑥 .
Any combination of these matched code- word will produce a candidate combination.
Why not consider the geometric constraint?

31. Finding all matched code-word combinations in training images

32. Learning Discriminative xCC
We’d like to obtain a xCC which satisfies the following three constraint.
Shape Constraint : Highly related Code-Book Establishment
Geometric Constraint: Object Location Agreement.
Structural Constraint : Reasonable code-word combination for different poses of object.

33. Learning Discriminative xCC
Example:

34. Learning Discriminative xCC
Binary Tree to represent a xCC.
Each node is a decision statement:

35. Learning Discriminative xCC
AdaBoost Training Procedure to produce one xCC from each iteration.
The Binary Tree depth “k” can be obtained by 3-fold cross validation.

36. Learning Discriminative xCC
Example:

37. Learning Discriminative xCC
Example:

38. Learning Discriminative xCC
Example:

39. Hybrid Detector xMCC
Incorporating SIFT as appearance information to enhance the performance.
Procedure: (same as previous section)

40. Hybrid Detector xMCC
Example:

41. Hybrid Detector xMCC
Example:

42. Hybrid Detector xMCC
Example:

43. Experimental Result
Contour only result under viewpoint change. (train on side-view only)

44. Experimental Result
Contour only result for discriminating similar shape object classes.

45. Experimental Result
Compare with Shotton [6] on Weizmann Horse test set.
Shotton [6]: Use contour fragment, fixed number of code-words for each combination.

46. Experimental Result
Weizmann Horse Test Set.

47. Experimental Result
Graz-17 classes.

48. Experimental Result
Graz-17 dataset.

49. Experimental Result
Hybrid-Method result

50. Conclusion
This article provide a contour based method that exploits very simple and generic shape primitives of line segments and ellipses for image classification and object detection.
Novelty:
Shape-Token to reduce the time cost for matching and the need of memory storage.
No restriction on the number of shape-tokens for combinations.
Allow combination of different feature types.