Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology.

Slides:

Advertisements

Similar presentations

Distinctive Image Features from Scale-Invariant Keypoints

Advertisements

Outline Feature Extraction and Matching (for Larger Motion)

Object Recognition using Invariant Local Features Applications l Mobile robots, driver assistance l Cell phone location or object recognition l Panoramas,

Instructor: Mircea Nicolescu Lecture 15 CS 485 / 685 Computer Vision.

Matching with Invariant Features

Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.

Computational Photography

Locating and Describing Interest Points

1 Interest Operators Find “interesting” pieces of the image –e.g. corners, salient regions –Focus attention of algorithms –Speed up computation Many possible.

Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.

Recognising Panoramas

Features-based Object Recognition Pierre Moreels California Institute of Technology Thesis defense, Sept. 24, 2007.

1 Interest Operator Lectures lecture topics –Interest points 1 (Linda) interest points, descriptors, Harris corners, correlation matching –Interest points.

Scale Invariant Feature Transform

Distinctive Image Feature from Scale-Invariant KeyPoints

Interest Points and Corners Computer Vision CS 143, Brown James Hays Slides from Rick Szeliski, Svetlana Lazebnik, Derek Hoiem and Grauman&Leibe 2008 AAAI.

Object Recognition Using Distinctive Image Feature From Scale-Invariant Key point D. Lowe, IJCV 2004 Presenting – Anat Kaspi.

Jianke Zhu From Haibin Ling’s ICCV talk Fast Marching Method and Deformation Invariant Features.

Object Class Recognition Using Discriminative Local Features Gyuri Dorko and Cordelia Schmid.

Scale Invariant Feature Transform (SIFT)

Automatic Matching of Multi-View Images

1 Invariant Local Feature for Image Matching Presented by Wyman Advised by Michael R. Lyu 11 th December, 2006.

1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.

The Terrapins Computer Vision Laboratory University of Maryland.

Distinctive Image Features from Scale-Invariant Keypoints David G. Lowe – IJCV 2004 Brien Flewelling CPSC 643 Presentation 1.

Matthew Brown University of British Columbia (prev.) Microsoft Research [ Collaborators: † Simon Winder, *Gang Hua, † Rick Szeliski † =MS Research, *=MS.

Overview Introduction to local features

The Beauty of Local Invariant Features

Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.

Image Stitching Tamara Berg CSE 590 Computational Photography Many slides from Alyosha Efros & Derek Hoiem.

Local Invariant Features

Recognition and Matching based on local invariant features Cordelia Schmid INRIA, Grenoble David Lowe Univ. of British Columbia.

Object Tracking/Recognition using Invariant Local Features Applications l Mobile robots, driver assistance l Cell phone location or object recognition.

Overview Harris interest points Comparing interest points (SSD, ZNCC, SIFT) Scale & affine invariant interest points Evaluation and comparison of different.

Local invariant features Cordelia Schmid INRIA, Grenoble.

Reporter: Fei-Fei Chen. Wide-baseline matching Object recognition Texture recognition Scene classification Robot wandering Motion tracking.

Building local part models for category-level recognition C. Schmid, INRIA Grenoble Joint work with G. Dorko, S. Lazebnik, J. Ponce.

Feature Detection and Descriptors

CSE 185 Introduction to Computer Vision Local Invariant Features.

Features-based Object Recognition P. Moreels, P. Perona California Institute of Technology.

Evaluation of interest points and descriptors. Introduction Quantitative evaluation of interest point detectors –points / regions at the same relative.

CVPR 2003 Tutorial Recognition and Matching Based on Local Invariant Features David Lowe Computer Science Department University of British Columbia.

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Deformation Invariant Image Matching Haibin Ling and David W. Jacobs.

A Sparse Texture Representation Using Affine-Invariant Regions Svetlana Lazebnik, Jean Ponce Svetlana Lazebnik, Jean Ponce Beckman Institute University.

Local invariant features Cordelia Schmid INRIA, Grenoble.

Harris Corner Detector & Scale Invariant Feature Transform (SIFT)

Vision and SLAM Ingeniería de Sistemas Integrados Departamento de Tecnología Electrónica Universidad de Málaga (Spain) Acción Integrada –’Visual-based.

Deconstruction: Discriminative learning of local image descriptors Samantha Horvath Learning Based Methods in Vision 2/14/2012.

Overview Introduction to local features Harris interest points + SSD, ZNCC, SIFT Scale & affine invariant interest point detectors Evaluation and comparison.

Matching features Computational Photography, Prof. Bill Freeman April 11, 2006 Image and shape descriptors: Harris corner detectors and SIFT features.

Project 3 questions? Interest Points and Instance Recognition Computer Vision CS 143, Brown James Hays 10/21/11 Many slides from Kristen Grauman and.

CS654: Digital Image Analysis

Presented by David Lee 3/20/2006

Real-Time Feature Matching using Adaptive and Spatially Distributed Classification Trees Aurélien BOFFY, Yanghai TSIN, Yakup GENC.

CSE 185 Introduction to Computer Vision Local Invariant Features.

Recognizing specific objects Matching with SIFT Original suggestion Lowe, 1999,2004.

Distinctive Image Features from Scale-Invariant Keypoints Presenter :JIA-HONG,DONG Advisor : Yen- Ting, Chen 1 David G. Lowe International Journal of Computer.

SIFT Scale-Invariant Feature Transform David Lowe

Lecture 13: Feature Descriptors and Matching

Presented by David Lee 3/20/2006

Distinctive Image Features from Scale-Invariant Keypoints

Project 1: hybrid images

Corners and interest points

Source: D. Lowe, L. Fei-Fei Recap: edge detection Source: D. Lowe, L. Fei-Fei.

Future directions Build true 3D models

CSE 455 – Guest Lectures 3 lectures Contact Interest points 1

Recognition and Matching based on local invariant features

Presentation transcript:

Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology

Large baseline stereo Features – what for ? Stitching Object recognition [Dorko & Schmid’05] [Lowe ’04] [Brown & Lowe ’03] [Tuytelaars & Van Gool ’00]

Moving the viewpoint

232 keypoints extracted Features stability Features stability is not perfect… 240 keypoints extracted

First stage – feature detector difference of gaussians [Crowley’84] Kadir & Brady [Kadir’02] Harris [Harris’88] Affine invariant Harris [Mikolajczyk’02]

Second stage – feature descriptor SIFTSteerable filters Differential invariants Shape context [Lowe ’04][Freeman’91 ] [Belongie’02 ][Schmid’97 ]

Evaluations – Mikolajczyk ’03-’05 Large viewpoint change Computation of ground truth positions via a homography

Evaluations – Mikolajczyk ’03-’05 [CVPR’03][PAMI’04] [submitted] SIFT-based descriptors rule ! All affine-invariant detectors are good, they should all be used together.

2D vs. 3D Ranking of detectors/descriptors combinations are modified when switching from 2D to 3D objects

Dataset – 100 3D objects

Viewpoints 45° apart

Ground truth - Epipolar constraints

Testing setup Unrelated images used to load the database of features.

Distance ratio Correct matches are highly distinctive  lower ratio Incorrect correspondences are ‘random correspondences’  low distinctiveness and ratio close to 1 [Lowe’04]

Are we accepting wrong matches ? Manual user classification into correct and incorrect triplets Comparison with a simpler system: 2 views, only one epipolar constraint. Pietro said maybe don’t need this slide – I think it is important to justify our 3-cameras setup

Detectors / descriptors tested DetectorsDescriptors Harris Hessian Harris-affine Hessian-affine Difference-of-gaussians MSER Kadir-Brady SIFT steerable filters differential invariants shape context PCA-SIFT

Results – viewpoint change Mahalanobis distance No ‘background’ images

Results – lighting / scale changes Change in light – result averaged over 3 lighting conditions. Change in scale - 7.0mm to 14.6mm

Conclusions Automated ground truth for 3D objects/scenes Ranking changes from 2D to 3D Stability is much lower for 3D Detectors – affine-rectified detectors are indeed best Descriptors – SIFT and shape context performed best. Application: use ground truth in order to learn probability densities: ‘how does a correct match look like ?’