1. Learning Deep Features for Discriminative Localization
Bolei Zhou， Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba
Computer Science and Artificial Intelligence Laboratory, MIT

2. Aim Method Remove fully-connected (FC) layers
Using global average pooling (GAP) instead of global max pooling
Class activation map (CAM)

3. pipeline

4. Class Activation Map 𝐹 𝑘 = 𝑥,𝑦 𝑓 𝑘 (𝑥,𝑦) 𝑆 𝑐 = 𝑘 𝑤 𝑘 𝑐 𝐹 𝑘
𝐹 𝑘 = 𝑥,𝑦 𝑓 𝑘 (𝑥,𝑦)
𝑓 𝑘 (𝑥,𝑦)
𝑆 𝑐 = 𝑘 𝑤 𝑘 𝑐 𝐹 𝑘
𝑃 𝑐 = exp⁡( 𝑆 𝑐 ) 𝑐 exp⁡( 𝑆 𝑐 )
unit k
𝑆 𝐴𝑢𝑠𝑡𝑟𝑎𝑙𝑖𝑎𝑛 𝑡𝑒𝑟𝑟𝑖𝑒𝑟
{𝑤 𝑘 𝐴𝑢𝑠𝑡𝑟𝑎𝑙𝑖𝑎𝑛 𝑡𝑒𝑟𝑟𝑖𝑒𝑟 } 𝑘=1,..,𝑛

5. Class Activation Map ⋯ 𝑆 𝑐 = 𝑘 𝑤 𝑘 𝑐 𝐹 𝑘 = 𝑘 𝑤 𝑘 𝑐 𝑥,𝑦 𝑓 𝑘 (𝑥,𝑦)
𝑆 𝑐 = 𝑘 𝑤 𝑘 𝑐 𝐹 𝑘
= 𝑘 𝑤 𝑘 𝑐 𝑥,𝑦 𝑓 𝑘 (𝑥,𝑦)
= 𝑘 𝑥,𝑦 𝑤 𝑘 𝑐 𝑓 𝑘 (𝑥,𝑦)
= 𝑥,𝑦 𝑘 𝑤 𝑘 𝑐 𝑓 𝑘 (𝑥,𝑦)
unit 1
unit n ⋯
𝑤 2
𝑀 𝑐 (𝑥,𝑦)= 𝑘 𝑤 𝑘 𝑐 𝑓 𝑘 (𝑥,𝑦)
𝑤 𝑛
𝑤 1
Class Activation Map for class c
Class Activation Map for class c

6. Experiment CNN types: AlexNet VGGnet GoogLeNet
Network in Network (NIN)
Dataset:
ILSVRC 2014

7. Classification

8. Localization

9. Deep Features for Generic Localization
CNN types:
AlexNet
GoogLeNet
GoogLeNet-GAP
Dataset:
SUN397
MIT Indoor67
Scene15
SUN Attribute
Caltech101
Caltech256
Stanford Action40
UIUC Event8

10. Deep Features for Generic Localization
CUB200 contains 11,788 images, with 5,994 images for training and 5,794 for test. (200 birds species)

11. Deep Features for Generic Localization

12. Deep Features for Generic Localization

13. Visualizing Class-Specific Units

14. 𝐹 𝑘