1. Overcoming Dataset Bias: An Unsupervised Domain Adaptation Approach Boqing Gong University of Southern California Joint work with Fei Sha and Kristen Grauman 1

2. Motivation Vision datasets Key to computer vision research Instrumental to benchmark different algorithms Recent trends Growing fast in scale: Gbytes to web-scale Providing better coverage: objects, scenes, activities, etc 2

3. Object recognition Many factors affect visual appearance Fundamentally, have to cover an exponentially large space Example 3 Limitation: datasets are biased Images from [Saenko et al.’10]. Amazon online catalogueOffice Monitor from two datasets/domains

4. Consequence: dataset-specific classifier 4 TRAINTEST Poor cross-dataset generalization Datasets have different underlying distributions Classifiers overfit to datasets’ idiosyncrasies Images from [Saenko et al.’10].

5. Mismatch is common Object recognition Video analysis Pedestrian detection 5 [Torralba & Efros’11, Perronnin et al.’10] [Duan et al.’09, 10] [Dollár et al.’09]

6. Our solution: unsupervised domain adaptation Setup Source domain (labeled dataset) Target domain (unlabeled dataset) Objective Train classification model to work well on the target 6 The two distributions are not the same! ?

7. Our algorithms Geodesic flow kernel (GFK): Learning domain-invariant representation Mitigating domain idiosyncrasy Landmarks Bridging two domains to have similar distributions Training discriminatively w.r.t. target without using labels 7 T S [Gong, Shi, Sha, Grauman, CVPR’12] [Gong, Sha, Grauman, ICML’13]

8. Correcting sample bias Ex. [Shimodaira’00, Huang et al.’06, Bickel et al.’07] Assumption: marginal distributions are the only difference Learning transductively Ex. [Bergamo & Torresani’10, Bruzzone & Marconcini’10] Assumption: shift classifiers via high-confident predictions Learning a shared representation Ex. [Daumé III’07, Pan et al.’09, Gopalan et al.’11] Assumption: a latent space exists s.t. Ps(X,Y)=Pt(X,Y) Related work 8

9. Our algorithms Geodesic flow kernel (GFK): learning domain-invariant representation mitigating domain idiosyncrasy Landmarks Bridging two domains to have similar distributions Training discriminatively w.r.t. target without using labels 9 T S [Gong, Shi, Sha, Grauman, CVPR’12] [Gong, Sha, Grauman, ICML’13]

10. Assume low-dimensional structure Ex. PCA, Partial Least Squares (source only) Modeling data with linear subspaces 10 Target Source

11. GFK: Domain-invariant features 11 TargetSource More similar to source.

12. GFK: Domain-invariant features 12 TargetSource More similar to target.

13. GFK: Domain-invariant features 13 TargetSource Blend the two.

14. We define the geodesic flow kernel (GFK): Advantages Analytically computable Encoded domain-invariant features Broadly applicable: can kernelize many classifiers GFK in closed form 14

15. Our algorithms Geodesic flow kernel (GFK): learning domain-invariant representation mitigating domain idiosyncrasy Landmarks Bridging two domains to have similar distributions Training discriminatively w.r.t. target without using labels 15 [Gong, Shi, Sha, Grauman, CVPR’12] [Gong, Sha, Grauman, ICML’13]

16. Landmarks Labeled source instances that are distributed as could be sampled from the target serve as conduit for discriminative loss in target domain 16 Landmarks Target Source

17. Key steps 17 Landmarks Target Source 1 Identify landmarks at multiple scales.

18. Key steps 18 2 Construct auxiliary domain adaptation tasks 3 MKL 4

19. Identifying landmarks Select landmarks such that 1)landmark distribution ≈ target distribution 2)#landmarks are balanced across different categories 19 Select landmarks Landmarks Target

20. Identifying landmarks 20 Landmarks Target Select landmarks 1)landmark distribution 2)#landmarks are bala Integer P  QP Distribution Category balance

21. Multi-scale analysis : GFK σ : multi-scale analysis  multiple sets of landmarks 21 Select landmarks such that 1)landmark distribution 2)#landmarks are bala QP

22. Exemplar images of landmarks 22 Target Source 6 0 -3

23. Use of multiple sets of landmarks 1. Auxiliary task At each scale, 23 Target Source Landmarks Source’ = source - landmarks Target’ = target + landmarks

24. Use of multiple sets of landmarks 1. Auxiliary task At each scale, Provably easier to solve 24 Target Source Landmarks Source’ = source - landmarks Target’ = target + landmarks

25. Use of multiple sets of landmarks 2. Discriminative learning, w.r.t. target Train classifiers using labeled landmarks Discriminative loss is measured towards target via landmarks 25 Target Source Landmarks

26. Experimental study Setup Cross-dataset generalization for object recognition among four domains Re-examine dataset’s values in light of domain adaptation Evaluation Caltech-256AmazonDSLRWebcam [Griffin et al.’07, Saenko et al.’10]

27. Classification accuracy: baseline 27 Accuracy (%) Source  Target

28. Classification accuracy: existing methods 28 Accuracy (%) Source  Target

29. Classification accuracy: GFK (ours) 29 Accuracy (%) Source  Target

30. Classification accuracy: landmark (ours) 30 Accuracy (%) Source  Target

31. Cross-dataset generalization [Torralba & Efros’11] 31 Accuracy (%) Caltech-101 generalizes the worst. Performance drop of ImageNet is big. Analyzing datasets in light of domain adaptation Performance drop!

32. Cross-dataset generalization [Torralba & Efros’11] 32 Accuracy (%) Performance drop becomes smaller! Caltech-101 generalizes the worst (w/ or w/o adaptation). There is nearly no performance drop of ImageNet. Analyzing datasets in light of domain adaptation

33. Summary Datasets are biased Overcome by domain adaptation (DA) Our solutions: new learning methods for DA Geodesic flow kernel (GFK) Learn a shared representation of domains Landmarks Discriminative loss is measured towards target via landmarks Data collection guided by DA Informative data: those cannot be classified well by adapting from existing datasets 33

34. 34 Target Source 6 0 -3

35. 35

36. Discussion Scalability GFK: eigen-decomposition, Landmarks: QP Overall, polynomial time 36 Target Source Landmarks

37. Which domain should be used as the source? 37 DSLR Caltech-256 Webcam Amazon

38. We introduce the Rank of Domains measure: Intuition –Geometrically, how subspaces disagree –Statistically, how distributions disagree 38 Automatically selecting the best

39. 39 Possible sources Our ROD measure Caltech-256 0.003 Amazon 0 DSLR 0.26 Webcam 0.05 Caltech-256 adapts the best to Amazon. Accuracy (%) Source  Target Automatically selecting the best