1. Towards Open Set Deep Networks
Abhijit Bendale and Terrance E. Boult, CVPR 2016

2. Meta-Recognition: The theory and Practice of Recognition Score Analysis

3. 1. Meta-Recognition

4. 2. Meta-Recognition for per instance
Success in a recognition system occurs when the match is the top score.
Instead of setting up thresholds for entire recognition system, it determines threshold for each instance by modeling scores of the instance.
Meta-Recognition problem is formalized as “Determining if the top K scores contain an outlier with respect to the current input instance’s nonmatch distribution”

5. 3. Extreme value Theory Definition 1(Fisher-Tippet Theorem)
Let ( 𝑠 1 , 𝑠 2 ,…) be a sequence of i.i.d. samples. Let 𝑀 𝑛 =max s 1 ,…, s n . If a sequence of pairs of real numbers 𝑎 𝑛 , 𝑏 𝑛 exists such that each 𝑎 𝑛 >0 and
lim 𝑛→∞ 𝑃 𝑀 𝑛 − 𝑏 𝑛 𝑎 𝑛 ≤𝑥 =𝐹 𝑥 ,
then if 𝐹 is a nondegenerate distribution function, it belongs to one of three extreme value distribution.

6. 4. Meta-Recognition via E.V.T

7. 5. Generalized Extreme Value Distribution
The three type of distribution can be unified into a generalized extreme value(GEV) distribution given by
𝐺𝐸𝑉 𝑡 = exp − 1+𝑘𝑥 −1/𝑘 , 𝑘≠0 exp − exp −𝑘 , 𝑘=0
where 𝑥= 𝑡−𝜏 𝜆 .
If match score is bounded, then the distribution of the maximum reduces to a Weibull(or Reversed Weibull) Distribution.

8. 6. Conclusion
Instead of setting up thresholds for entire recognition system, it determines threshold for each instance by modeling scores of the instance.
EVT can be generalized to the weaker assumption of exchangeable random variable.[6]
Extreme Value Theory generalizes to all recognition system producing distance or similarity scores over known images.

9. Towards Open Set Deep Networks

10. 1. Open Set Recognition
Definition 2 (The Open Set Recognition Problem)[4]
Open set recognition is to find a measurable recognition function 𝑓∈ℋ, where 𝑓 𝑥 >0 implies positive recognition, and 𝑓 is defined by minimizing the Open Set Risk:
arg min 𝑓∈ℋ ℛ 𝒪 𝑓 + 𝜆 𝑟 ℛ 𝜖 𝑓 ,
Where Open Space Risk ℛ 𝒪 𝑓 is defined by
ℛ 𝒪 𝑓 = 𝒪 𝑓 𝑥 𝑑𝑥 𝑆 𝑜 𝑓 𝑥 𝑑𝑥 ,
and Open Space 𝒪 is defined by[2]
𝒪= 𝒮 𝑜 − 𝑖∈𝑁 𝐵 𝑟 𝑥 𝑖

11. 2. Compact Abating Probability Model
Open space risk exists when a recognition model labels space far from any training data
“Abating” means that the spatial influence decreases with distance from certain point 𝑥 ∗
Abating Probabilistic model is that the probability of points associating abates as the spatial separation of any two points increase.

12. 2. Compact Abating Probability Model
Definition 2 (Open Space Risk of CAP model)
Let 𝑀 𝜏,𝑦 𝑥 be a probabilistic recognition function that uses a CAP model over a known training set for class y, where ∃ 𝑥 𝑖 ∈𝒦| 𝑀 𝜏,𝑦 𝑥 𝑖 >0.
If 𝑟>𝜏, then ℛ 𝒪 𝑀 𝜏,𝑦 =0, i.e., when the CAP distance threshold is smaller than the open space radius, the CAP model has zero open space risk.

13. 2. Compact Abating Probability Model
Corollary 1. (Thresholding CAP model probability manages Open Space Risk)
For any CAP model, considering only points with sufficiently high probability will reduce open space risk.

14. 3. Multi-class Meta-Recognition
Disadvantage of SoftMax
It is normalized to follow a logistic distribution.
It cannot reject without threshold
Activation Vector(AV)
Values from penultimate layer.
They are not aan independent per-class score estimate
They provide a distribution of what classes are related.

15. 3. Algorithm 1

16. 3. Algorithm 2

17. 5. Experiment Pre-trained AlexNet is used. The test set consists of
50K closed set images with the 1000 categories from ILSVRC 2012.
15K open set images with the 360 categories from ILSVRC 2010.
15K fooling images with 15 images each per ILSVRC 2012 categories.

18. 5. Experiment

19. 5. Experiment

20. 6. Limitation

21. Reference
[1] Abhijit Bendale and Terrance E. Boult, “Towards Open Set Deep Networks”, CVPR, 2016 [2] Abhijit Bendale and Terrance E. Boult, “Towards Open Set World Recognition”, CVPR, 2015 [3] Walter J. Scheirer, Lalit P. Jain, and Terrance E. Boult, “Probability Models for Open Set Recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 11, November, [4] Walter J. Scheirer, Andreson de Rezende Rocha, Archana Sapkota and Terrance E. Boult, “Toward Open Set Recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no.7, July, [5] Walter J. Scheirer, Anderson Rocha, Ross J. Micheals, and Terrance E. Boult, “Meta- Recognition: The Theory and Practice of Recongition Score Analysis.”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 8, August, [6] Simeon M. Berman, “Limiting Distribution of the Maximum Term in Sequences of Dependent Random Variables”, Annals of Math. Statistic, vol. 33, no. 3, pp , 1962.