1. Kapitel 14
Recognition
Scene understanding / visual object categorization
Pose clustering
Object recognition by local features
Image categorization
Bag-of-features models
Large-scale image search
TexPoint fonts used in EMF.
Read the TexPoint manual before you delete this box.: AA

2. Scene understanding (1)
Scene categorization
outdoor/indoor
city/forest/factory/etc.

3. Scene understanding (2)
Object detection
find pedestrians

4. Scene understanding (3)
sky
mountain
building
tree
building
banner
street lamp
market
people

5. Visual object categorization (1)
~10,000 to 30,000

6. Visual object categorization (2)

7. Visual object categorization (3)

8. Visual object categorization (4)
Recognition is all about modeling variability
camera position
illumination
shape variations
within-class variations

9. Visual object categorization (5)
Within-class variations (why are they chairs?)

10. Pose clustering (1) Working in transformation space - main ideas:
Generate many hypotheses of transformation image vs. model, each built by a tuple of image and model features
Correct transformation hypotheses appear many times
Main steps:
Quantize the space of possible transformations
For each tuple of image and model features, solve for the optimal transformation that aligns the matched features
Record a “vote” in the corresponding transformation space bin
Find "peak" in transformation space

11. Pose clustering (2)
Example: Rotation only. A pair of one scene segment and one model segment suffices to generate a transformation hypothesis.

12. Pose clustering (3)
C.F. Olson: Efficient pose clustering using a randomized algorithm. IJCV, 23(2): , 1997.
2-tuples of image and model corner points are used to generate hypotheses. (a) The corners found in an image. (b) The four best hypotheses found with the edges drawn in. The nose of the plane and the head of the person do not appear because they were not in the models.

13. Object recognition by local features (1)
D.G. Lowe: Distinctive image features from scale-invariant keypoints. IJCV, 60(2): , 2004
The SIFT features of training images are extracted and stored
For a query image
Extract SIFT features
Efficient nearest neighbor indexing
Pose clustering by Hough transform
For clusters with >2 keypoints (object hypotheses): determine the optimal affine transformation parameters by least squares method; geometry verification
Input Image
Stored Image

14. Object recognition by local features (2)
Cluster of 3 corresponding feature pairs

15. Object recognition by local features (3)

16. Object recognition by local features (4)

17. Image categorization (1)
Training Labels
Training Images
Training
Image Features
Classifier Training
Trained Classifier
Testing
Image Features
Trained Classifier
Prediction
Outdoor
Test Image

18. Image categorization (2)
Global histogram (distribution):
color, texture, motion, …
histogram matching distance

19. Image categorization (3)
Cars found by color histogram matching
See Chapter “Inhaltsbasierte Suche in Bilddatenbanken

20. Bag-of-features models (1)
Bag of ‘words’
Object

21. Bag-of-features models (2)
Origin 1: Text recognition by orderless document representation (frequencies of words from a dictionary), Salton & McGill (1983)
US Presidential Speeches Tag Cloud

22. Bag-of-features models (3)
Origin 2: Texture recognition
Texture is characterized by the repetition of basic elements or textons
For stochastic textures, it is the identity of the textons, not their spatial arrangement, that matters

23. Bag-of-features models (4)
histogram
Universal texton dictionary

24. Bag-of-features models (5)
We need to build a “visual” dictionary!
G. Cruska et al.: Visual categorization with bags of keypoints. Proc. of ECCV, 2004.

25. Bag-of-features models (6)
Main step 1: Extract features (e.g. SIFT) …

26. Bag-of-features models (7)
Main step 2: Learn visual vocabulary (e.g. using clustering)
clustering

27. Bag-of-features models (8)
visual vocabulary
(cluster centers)

28. Bag-of-features models (9)
Example: learned codebook …
Appearance codebook
Source: B. Leibe

29. Bag-of-features models (10)
Main step 3: Quantize features using “visual vocabulary”

30. Bag-of-features models (11)
Main step 4: Represent images by frequencies of “visual words” (i.e., bags of features)

31. Bag-of-features models (12)
Example: representation based
on learned codebook

32. Bag-of-features models (13)
Main step 5: Apply any classifier to the histogram feature vectors

33. Bag-of-features models (14)
Example:

34. Bag-of-features models (15)
Caltech6 dataset
Dictionary quality and size are very important parameters!

35. Bag-of-features models (16)
Caltech6 dataset
Action recognition
J.C. Niebles, H. Wang, L. Fei-Fei: Unsupervised learning of human action categories using spatial-tempotal words. IJCV, 79(3): , 2008.

36. Large-scale image search (1)
J. Philbin, et al.: Object retrieval with large vocabularies and fast spatial matching. Proc. of CVPR, 2007
Query
Results from 5k Flickr images

37. Large-scale image search (2)
Aachen Cathedral
[Quack, Leibe, Van Gool, CIVR’08]
Mobile tourist guide
self-localization
object/building recognition
photo/video augmentation

38. Large-scale image search (3)

39. Large-scale image search (4)
Moulin Rouge
Old Town Square (Prague)
Tour Montparnasse
Colosseum
Viktualienmarkt Maypole
Application: Image auto-annotation
Left: Wikipedia image Right: closest match from Flickr

40. Sources
K. Grauman, B. Leibe: Visual Object Recognition. Morgen & Claypool Publishers, 2011
R. Szeliski: Computer Vision: Algorithms and Applications. Springer, (Chapter 14 “Recognition”)
Course materials from others (G. Bebis, J. Hays, S. Lazebnik, …)