1. A Study of Approaches for Object Recognition
Presented by Wyman Wong
12/9/2005

2. Outlines Introduction Model-Based Object Recognition
AAM
Inverse Composition AAM
View-Based Object Recognition
Recognition based on boundary fragments
Recognition based on SIFT
Proposed Research
Conclusion and Future Work

3. Introduction Object Recognition
A task of finding 3D objects from 2D images (or even video) and classifying them into one of the many known object types
Closely related to the success of many computer vision applications
robotics, surveillance, registration … etc.
A difficult problem that a general and comprehensive solution to this problem has not been made

4. Introduction Two main streams of approaches:
Model-Based Object Recognition
3D model of the object being recognized is available
Compare the 2D representation of the structure of an object with the 2D projection of the model
View-Based Object Recognition
2D representations of the same object viewed at different angles and distances when available
Extract features (as the representations of object) and compare them to the features in the feature database
The 3D model contains detailed information about the object, including the shape of its structure, the spatial
relationship between its parts and its appearance.

5. Introduction Pros and Cons of each main stream:
Model-Based Object Recognition
Model features can be predicted from just a few detected features based on the geometric constraints
Models sacrifice its generality
View-Based Object Recognition
Greater generality and more easily trainable from visual data
Matching is done by comparing the entire objects, some methods may be sensitive to clutter and occlusion

6. Model-Based Object Recognition
Commonly used in face recognition
General Steps:
Locate the object,
locate and label its structure,
adjust the model's parameters until the model generates an image similar enough to the real object.
Active Appearance Models (AAM) have been proved to be highly useful models for face recognition

7. Active Appearance Models
They model shape and appearance of objects separately
Shape: the vertex locations of a mesh
Appearance: the pixels’ values of a mesh
Both of the parameters above used PCA to generalize the face recognition to generic face
Fitting an AAM: non-linear optimization solution is applied which iteratively solve for incremental additive updates to the shape and appearance coefficients
SHORT description only

8. Inverse Compositional AAMs
The major difference of these models with AAMs is the fitting algorithm
AAM: additive incremental update shape and appearance parameters
ICAAM: inverse compositional update – The algorithm updates the entire warp by composing the current warp with the computed incremental warp

9. View-Based Object Recognition
Common approaches:
Correlation-based template matching (Li, W. et al. 95)
SEA, PDE, … etc
Not effective when the following happens:
illumination of environment changes
Posture and scale of object changes
Occlusion
Color Histogram (Swain, M.J. 90)
Construct histogram for an object and match it over image
It is robust to changing of viewpoint and occlusion
But it requires good isolation and segmentation of objects

10. View-Based Object Recognition
Common approaches:
Feature based
Extract features from the image that are salient and match only to those features when searching all location for matches
Feature types: groupings of edges, SIFT … etc
Feature’s property preferences:
View invariant
Detected frequently enough for reliable recognition
Distinctive
Image descriptor is created based on detected features to increase the matching performance
Image descriptor = Key / Index to database of features
Descriptor’s property preferences:
Invariant to scaling, rotation, illumination, affine transformation and noise
Feature, not necessary a corner

11. Nelson’s Approach Recognition based on 2D Boundary Fragments
Prepare 53 clean images for each object and build 3D recognition database:
Object
Camera

12. Nelson’s Approach
Test images used in Nelson’s experiment and their features

13. Nelson’s Approach
Nelson’s experiment has shown his approach has high accuracy
97.0% success rate for 24 objects database
under the following conditions:
Large number of images
Clean images
Very different objects
No occlusion and clutter

14. Lowe’s Approach
Recognition based on Scale Invariant Feature Transform (SIFT)
SIFT generates distinctive invariant features
SIFT based image descriptors are generally most resistant to common image deformations (Mikolajczyk 2005)
SIFT – four steps:
Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor computation

15. Scale-space extrema detection
DOG ~ LOG
Search over all sample points in all scales and find extrema that are local maxima or minima in laplacian space
Also talk about using extrema, instead of absolute threshold
Small keypoints  Solve occlusion problem
Large keypoints  Robust to noise and image blur

16. Keypoint localization
Reject keypoints with the following properties:
Low contrast (sensitive to noise)
Localized along edge (sliding effect)
Solution:
Filter points with value D below 0.03
Apply Hessian edge detector

17. Orientation assignment
Pre-compute the gradient magnitude and orientation
Use them to construct keypoint descriptor

18. Keypoint descriptor computation
Create orientation histogram over 4x4 sample regions around the keypoint locations
Each histogram contains 8 orientation bins
4x4x8 = 128 elements vectors (distinctively representing a feature)

19. Object Recognition based on SIFT
Nearest-neighbor algorithm
Matching: assign features to objects
There can be many wrong matches
Solution
Identify clusters of features
Generalized Hough transform
Determine pose of object and then discard outliers

20. Proposed Research
Personally, I think model-based approach does have better performance
Success of model-based approach requires:
All models of objects to be detected
Automatically construct models
Automatically select the best model
How do the system know which 3D model to be used on a specific image of object?
By view-based approach
Human looks at an image of object for a moment and then realize which model to be used on that object
Then use the specific model to refine the identification of the specific object

21. Hybrid of bottom-up and top-down
View-based approaches just presented are bottom-up approaches
Features: edges, extrema (Low Level)
Descriptors of features
Matching
Identification of object (High Level)
Can it be like that?
Features …
Matching (Lower Level)
Guessing of object (Higher Level)
Identification of object

22. Hierarchy of features Lowe’s system
All features have equal weight in voting of object during identification of object (subject to be verified by examining the opened source code)
Special features do not have enough voting power to shift the result to the correct one
Consider the following scenario:
Two objects have many similar features, a1 to a100 are similar to b1 to b100, and have just one very different feature, a* for object A and b* for object B
Many a1 to a100 may be poorly captured by imaging device and mismatched as b1 to b100 , even we can still recognize the feature a*, the system may still think the object is B
Object A
Object B

23. Extension of SIFT Color descriptors
Local texture measures incorporated into feature descriptors
Scale-invariant edge groupings
*Generic object class recognition

24. Conclusion and Future Work
Discussed the different approaches in object recognition
Discussed what is SIFT and how it works
Discussed the possible extensions to SIFT
Design hybrid approach
Design extensions

25. Q & A
Thank you very much!

26. Things to be understood
Find extrema over same scale space is good, why need to find over different scale?