Lior Wolf and Noga Levy The SVM-minus Similarity Score for Video Face Recognition Makarand Tapaswi CVPR Reading VGG 1

2 Same / Not Same ?

One liner “How similar is the face in one video sequence to the other, where the similarity is uncorrelated with pose-induced similarity” illumination, expression, image quality, pose classifier should – discriminate positive/negative AND – uncorrelate w.r.to additional feature set 3

Learning with Side Information Domain adaptation / Co-training Meta features Latent information: maximize /marginalize Learning Using Privileged Information (LUPI) – SVM+ gains additional discrimination power – SVM– eliminates a factor that is irrelevant 4

Basic Notation 5

One Shot Similarity (OSS) 6

Multiple One Shot Similarity (MSS) Complete bg. contains too much variation Classifier might distinguish something else rather than identity! Multiple bg. sets: identities, pose, etc. 7

Matched Background Similarity same person 8 B1 X1 B2 X2 B

MBGS different persons 9 B1 X1 B2 X2 B

MBGS 10

SVM-minus Classifier

SVM-minus classifier 12

SVM-minus Classifier (2) 13

SVM– loss function 14

Reduce to standard SVM 15

Projection Matrix 16

Cancel influence from pose +ve scoring poses need not be same person –ve scoring poses need not be different person SVM-minus Similarity same person Pan angle

One-Side SVM-minus Similarity 18

SVM-minus Similarity Use one-side SVM-minus for online tasks 19

YouTube Faces DB 20

Experimental info 21

MBGS Results from [36] Lior Wolf, Tal Hassner and Itay Maoz. Face Recognition in Unconstrained Videos with Matched Background Similarity. CVPR

This paper results Results where SVM– did most better than MBGS 23

Results MBGS > SVM– at Accuracy but, MBGS + SVM– wins Combination done by stacking – learning yet another SVM for the 2D scores 24

Is it really useful? Combined score “statistically significant” for [FP]LBP Use entire background set, AUC: 83.6% to 79.9% Online applications (one-side), AUC: 83.6% to 81.9% Correlations: – Within method higher, different scores – Across methods, highest for same feature (as expected) 25

Conclusion SVM– : unlearn using additional features MBGS : be choosy about the negative set 3D Pose : a good “privileged” information source They don’t talk about pose estimation accuracy Different types of privileged info that might work Metric learning (and relatives) not compared Thank You! 26

Some more results from other sources YouTube Faces DB Ref MethodAccuracy ± SEAUCEER [1] MBGS L2 mean, LBP76.4 ± [2]MBGS+SVM-78.9 ± [3]APEM-FUSION79.1 ± [4] STFRD+PMML79.5 ± References: [1] Lior Wolf, Tal Hassner and Itay Maoz. Face Recognition in Unconstrained Videos with Matched Background Similarity. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), [2] Lior Wolf and Noga Levy. The SVM-minus Similarity Score for Video Face Recognition. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), [3] Haoxiang Li, Gang Hua, Zhe Lin, Jonathan Brandt, Jianchao Yang. Probabilistic Elastic Matching for Pose Variant Face Verification. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), [4] Zhen Cui, Wen Li, Dong Xu, Shiguang Shan and Xilin Chen. Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013.