1. Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology

2. Large baseline stereo Features – what for ? Stitching Object recognition [Dorko & Schmid’05] [Lowe ’04] [Brown & Lowe ’03] [Tuytelaars & Van Gool ’00]

3. Moving the viewpoint

4. 232 keypoints extracted Features stability Features stability is not perfect… 240 keypoints extracted

5. First stage – feature detector difference of gaussians [Crowley’84] Kadir & Brady [Kadir’02] Harris [Harris’88] Affine invariant Harris [Mikolajczyk’02]

6. Second stage – feature descriptor SIFTSteerable filters Differential invariants Shape context [Lowe ’04][Freeman’91 ] [Belongie’02 ][Schmid’97 ]

7. Evaluations – Mikolajczyk ’03-’05 Large viewpoint change Computation of ground truth positions via a homography

8. Evaluations – Mikolajczyk ’03-’05 [CVPR’03][PAMI’04] [submitted] SIFT-based descriptors rule ! All affine-invariant detectors are good, they should all be used together.

9. 2D vs. 3D Ranking of detectors/descriptors combinations are modified when switching from 2D to 3D objects

10. Dataset – 100 3D objects

11. Viewpoints 45° apart

15. Ground truth - Epipolar constraints

16. Testing setup Unrelated images used to load the database of features.

17. Distance ratio Correct matches are highly distinctive  lower ratio Incorrect correspondences are ‘random correspondences’  low distinctiveness and ratio close to 1 [Lowe’04]

18. Are we accepting wrong matches ? Manual user classification into correct and incorrect triplets Comparison with a simpler system: 2 views, only one epipolar constraint. Pietro said maybe don’t need this slide – I think it is important to justify our 3-cameras setup

19. Detectors / descriptors tested DetectorsDescriptors Harris Hessian Harris-affine Hessian-affine Difference-of-gaussians MSER Kadir-Brady SIFT steerable filters differential invariants shape context PCA-SIFT

20. Results – viewpoint change Mahalanobis distance No ‘background’ images

21. Results – lighting / scale changes Change in light – result averaged over 3 lighting conditions. Change in scale - 7.0mm to 14.6mm

22. Conclusions Automated ground truth for 3D objects/scenes Ranking changes from 2D to 3D Stability is much lower for 3D Detectors – affine-rectified detectors are indeed best Descriptors – SIFT and shape context performed best. Application: use ground truth in order to learn probability densities: ‘how does a correct match look like ?’