Final Exam Review CS485/685 Computer Vision Prof. Bebis

Final Exam Final exam will be comprehensive. –Midterm Exam material –SIFT –Object recognition –Face recognition using eigenfaces –Camera parameters –Camera calibration –Stereo

SIFT feature computation Steps –Scale space extrema detection (how is it different from Harris-Laplace? different parameters) –Keypoint localization (need to know main ideas, no equations; two thresholds, which ones?) –Orientation assignment (how are the histograms built? multiple peaks?) –Keypoint descriptor (how are the histograms built? partial voting, main parameters, invariance to illumination changes)

SIFT features Properties –Scale and rotation invariant –Highly distinctive –Partially invariant to 3D viewpoint and illumination changes –Fast and efficient computation Main parameters? Matching –How do we match SIFT features? –How do we evaluate the performance of a feature matcher? Applications

SIFT variations PCA SIFT SURF GLOH Need to know key ideas and steps (no need to remember exact parameter values) Similarities/Differences with SIFT Strengths/Weakeness

Object Recognition Model-based vs category-specific recognition –Preprocessing & Recognition Challenges? –Photometric effects, scene clutter, changes in shape (e.g., non-rigid objects), viewpoint changes Requirements? –Invariance, robustness Performance Criteria? –Efficiency (time + memory), accuracy

Object Recognition (cont’d) Representation schemes – advantages/disadvantages –Object centered (3D/3D or 3D/2D matching) –Viewer centered (2D/2D matching) Matching schemes – advantages/disadvantages –Geometry-based –Appearance-based

Object Recognition (cont’d) Main steps in matching: –Hypothesis generation –Hypothesis verification Efficient hypothesis generation –Which scene features to choose? –How to organize and search the model database?

Object Recognition Methods Alignment Pose Clustering Geometric Hashing Main ideas and steps

Object Recognition using SIFT Main ideas and steps –Perform nearest neighbor search –Find clusters of features (pose clustering) –Perform verification Practical issues –Approximate nearest neighbors

Bag of Features Origins of bag of features method Computing Bag of Features –Feature extraction –Learn “visual vocabulary” (e.g., K-Means clustering) –Quantize features using “visual vocabulary”. –Represent images by frequencies of “visual words” (bugs of features)

Bag of Features (cont’d) Object categorization using bags of features. –Represent objects using Bag of Features –Classification (NN, kNN, SVM)

PCA Need to know steps and equations. What criterion does PCA minimize? How is the “best” low-dimensional space determined using PCA? What is the geometric interpretation of PCA? Practical issues (e.g., choosing K, computing error, standardization)

Using PCA for Face Recognition Represent faces using PCA – need to know steps and practical issues (e.g., AA T vs A T A) Face recognition using PCA (i.e., eigenfaces) –DIFS Face detection using PCA –DFFS Limitations

Camera Parameters Reference frames – what are they? –World –Camera –Image plane –Pixel plane Perspective projection –Should know how to derive equations –Matrix notation –Properties of perspective projection –Vanishing points, vanishing lines.

Camera Parameters Orthographic projection –How is related to perspective? –Study equations –Matrix notation –Properties Weak perspective projection –How is related to perspective? –Study equations –Matrix notation –Properties

Extrinsic camera parameters –What are they and what is their meaning? –Study equations Intrinsic camera parameters –What are they and what is their meaning? –Study equations Projection matrix –What does it represent? Camera Parameters (cont’d)

Camera Calibration What is the goal of camera calibration and how is it performed? Camera calibration using the projection matrix (study equations for step 1 only; you should remember how this method works in general) Direct parameter calibration (do not memorize equations but remember how they work); how is the orthogonally constraint of the rotation matrix enforced?

Stereo What is the goal of stereo vision? Triangulation principle. Familiarity with terminology (e.g., baseline, epipolar plane, epipolar lines, epipoles, disparity) Two main problems of stereo (i.e., correspondence + reconstruction) Recover depth from disparity – study proof.

Correspondence Problem What is the correspondence problem and why is it difficult? Main methods: intensity-based, feature-based –How do intensity-based methods work? –Main parameters of intensity-based methods. How can we choose them? –How do feature-based methods work? –Comparison between intensity-based and feature-based methods

Epipolar Geometry Stereo parameters: extrinsic + intrinsic What is the epipolar constraint, why is it important? How is epipolar geometry represented? –Essential matrix –Fundamental matrix

Essential Matrix What is the essential matrix? Properties of essential matrix Study equations Equation satisfied by corresponding points

Fundamental Matrix What is the fundamental matrix? Properties of fundamental matrix Study equations Equation satisfied by corresponding points

Eight-point algorithm What is it useful for? Study steps How is the rank(2) constraint enforced? Normalized eight-point algorithm Estimate epipoles and epipolar lines using the fundamental matrix?

Rectification What is the purpose of rectification? Why is it useful? Study steps

Stereo Reconstruction Three cases: –Known extrinsic and intrinsic parameters –Known intrinsic parameters –Unknown extrinsic and intrinsic parameters. What information could be recovered in each case? What are the main steps of the first two methods? (do not memorize equations)