1. Large-Scale Object Recognition with Weak Supervision
Weiqiang Ren, Chong Wang, Yanhua Cheng,
Kaiqi Huang, Tieniu Tan

2. Task2 : Classification + Localization
Task 2b: Classification + localization with additional training data
— Ordered by classification error
Only classification labels are used
Full image as object location

3. Outline
Motivation
Method
Results

4. Motivation

5. Why Weakly Supervised Localization (WSL)?
Knowing where to look, recognizing objects will be easier !
However, in the classification-only task, no annotations of object location are available.
Weakly Supervised Localization

6. Current WSL Results on VOC07

7. 13.9: Weakly supervised object detector learning with model drift detection, ICCV 2011
15.0: Object-centric spatial pooling for image classification, ECCV 2012
22.4: Multi-fold mil training for weakly supervised object localization, CVPR 2014
22.7: On learning to localize objects with minimal supervision, ICML 2014
26.2: Discovering Visual Objects in Large-scale Image Datasets with Weak Supervision, submitted to TPAMI
26.4: Weakly supervised object detection with posterior regularization, BMVC 2014
31.6: Weakly supervised object localization with latent category learning, ECCV 2014
Sep 11, Poster Session 4A, #34

8. Our Work
VOC 2007
Results
Ours
31.6
DPM 5.0
33.7
VOC 2007
Results
Ours
26.2
DPM 5.0
33.7
Weakly Supervised Object Localization with Latent Category Learning
Discovering Visual Objects in Large-scale Image Datasets with Weak Supervision
ECCV 2014
Submitted to TPAMI
For the consideration of high efficiency in large-scale tasks, we use the second one.

9. Method

10. … Framework 2 3 4 1 Det Prediction Rescoring Cls Prediction
Conv Layers
Input Images 1
FC Layers

11. 1st : CNN Architecture
Chatfield et al. Return of the Devil in the Details: Delving Deep into Convolutional Nets

12. 2nd: MILinear SVM

13. MILinear : Region Proposal
Good region proposal algorithms
High recall
High overlap
Small number
Low computation cost
MCG pretrained on VOC 2012
Additional Data
Training: 128 windows/ image
Testing: 256 windows/image
Compared to Selective Search (~2000)

14. MILinear: Feature Representations
Low Level Features
SIFT, LBP, HOG
Shape context, Gabor, …
Mid-Level Features
Bag of Visual Words (BoVW)
Deep Hierarchical Features
Convolutional Networks
Deep Auto-Encoders
Deep Belief Nets

15. MILinear: Positive Window Mining
Clustering
KMeans
Topic Model
pLSA, LDA, gLDA
CRF
Multiple Instance Learning
DD, EMDD, APR
MI-NN,
MI-SVM, mi-SVM
MILBoost

16. MILinear: Objective Function and Optimization
Multiple instance Linear SVM
Optimization: trust region Newton
A kind of Quasi Newton method
Working in the primal
Faster convergence

17. MILinear: Optimization Efficiency

18. 3rd: Detection Rescoring
Rescoring with softmax
train
softmax
max … …
128 boxes …… ……
1000 dim
1000 dim
1000 classes
Softmax: consider all the categories simultaneously  at each minibatch of the optimization – Suppress the response of other appearance similar object categories

19. 4th: Classification Rescoring
Linear Combination … … …
1000 dim
1000 dim
1000 dim
One funny thing: We have tried some other strategies of score combination, but it seems not working !

20. Results

21. 1st: Classification without WSL
Method
Top 5 Error
Baseline with one CNN :
13.7
Average with four CNNs:
12.5

22. 2nd: MILinear on ImageNet 2014
Methods
Detection Error
Baseline (Full Image)
61.96
MILinear
40.96
Winner
25.3

23. 2nd: MILinear on VOC 2007

24. 2nd: MILinear on ILSVRC 2013 detection
mAP: 9.63%! vs 8.99% (DPM5.0)

25. 2nd: MILinear for Classification
Methods
Top 5 Error
Milinear
17.1

26. 3rd: WSL Rescoring (Softmax)
Method
Top 5 Error
Baseline with one CNN :
13.7
Average with four CNN :
12.5
MILinear
17.1
MILinear + Rescore
13.5
The Softmax based rescoring successfully suppresses the predictions of other appearance similar object categories !

27. 4th: Cls and WSL Combinataion
Method
Top 5 Error
Baseline with one CNN model:
13.7
Average with four CNN models:
12.5
MILinear
17.1
MILinear + Rescore
13.5
Cls (12.5) + MILinear (13.5)
11.5
WSL and Cls can be complementary to each other!

28. Russakovsky et al. ImageNet Large Scale Visual Object Challenge.

29. Conclusion WSL always helps classification
WSL has large potential: WSL data is cheap

30. Thank You!