Presentation is loading. Please wait.

Presentation is loading. Please wait.

CVPR 2013 Diversity Tutorial Diverse M-Best Solutions in Markov Random Fields Dhruv Batra Virginia Tech Joint work with: Students: Payman Yadollahpour.

Similar presentations


Presentation on theme: "CVPR 2013 Diversity Tutorial Diverse M-Best Solutions in Markov Random Fields Dhruv Batra Virginia Tech Joint work with: Students: Payman Yadollahpour."— Presentation transcript:

1 CVPR 2013 Diversity Tutorial Diverse M-Best Solutions in Markov Random Fields Dhruv Batra Virginia Tech Joint work with: Students: Payman Yadollahpour (TTIC), Abner Guzman-Rivera (UIUC) Colleagues: Chris Dyer (CMU), Greg Shakhnarovich (TTIC), Pushmeet Kohli (MSRC), Kevin Gimpel (TTIC)

2 CVPR 2013 Diversity Tutorial (C) Dhruv Batra2

3 CVPR 2013 Diversity Tutorial Ambiguity Ambiguity Ambiguity (C) Dhruv Batra3 ? ? One instance / Two instances?

4 CVPR 2013 Diversity Tutorial Problems with MAP (C) Dhruv Batra4 Model-Class is Wrong! -- Approximation Error Not Enough Training Data! -- Estimation Error MAP is NP-Hard -- Optimization Error Inherent Ambiguity -- Bayes Error Make Multiple Predictions! Single Prediction = Uncertainty Mismanagement

5 CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra5 Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Sampling xxxxxxxxxxxxx

6 CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra6 Flerova et al., 2011 Fromer et al., 2009 Yanover et al., 2003 M-Best MAP Ideally: M-Best Modes Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Sampling

7 CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra7 Flerova et al., 2011 Fromer et al., 2009 Yanover et al., 2003 M-Best MAP Ideally: M-Best Modes Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Sampling Our work: Diverse M-Best in MRFs [ECCV 12] -Dont hope for diversity. Explicitly encode it. -Not guaranteed to be modes.

8 CVPR 2013 Diversity Tutorial (C) Dhruv Batra8 Example Result

9 CVPR 2013 Diversity Tutorial (C) Dhruv Batra9 Example Result Discriminative Re-ranking of Diverse Segmentation [Yadollahpour et al., CVPR13, Wednesday Poster]

10 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra10 kx1

11 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra11 kx1 10001000

12 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra12 kx1 01000100

13 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra13 kx1 00100010

14 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra14 kx1 00010001

15 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra15 kx1 00010001 k 2 x1

16 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra16 kx1 00010001 k 2 x1

17 CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra17 Graphcuts, BP, Expansion, etc

18 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best (C) Dhruv Batra18 MAP Diversity

19 CVPR 2013 Diversity Tutorial Diverse M-Best (C) Dhruv Batra19

20 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best (C) Dhruv Batra20 Q1: How do we solve DivMBest? Q2: What kind of diversity functions are allowed? Q3: How much diversity?

21 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best (C) Dhruv Batra21 Dualize Diversity-Augmented Score Primal

22 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best Lagrangian Relaxation (C) Dhruv Batra22 Diversity-Augmented Score Dual Concave (Non-smooth) Upper-Bound on Div2Best Score Subgradient Descent Div2Best score

23 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best Lagrangian Relaxation (C) Dhruv Batra23 Dualize Diversity-Augmented Energy Many ways to solve: 1.Subgradient Ascent. Optimal. Slow. 2. Binary Search. Optimal for M=2. Faster. 3. Grid-search on lambda. Sub-optimal. Fastest.

24 CVPR 2013 Diversity Tutorial Theorem Statement Theorem [Batra et al 12]: Lagrangian Dual corresponds to solving the Relaxed Primal: Based on result from [Geoffrion 74] (C) Dhruv Batra24 Dual Relaxed Primal

25 CVPR 2013 Diversity Tutorial Effect of Lagrangian Relaxation (C) Dhruv Batra25

26 CVPR 2013 Diversity Tutorial Effect of Lagrangian Relaxation (C) Dhruv Batra26

27 CVPR 2013 Diversity Tutorial Effect of Lagrangian Relaxation [Mezuman et al. UAI13] (C) Dhruv Batra27 Pairwise Potential Strength

28 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best (C) Dhruv Batra28 Q1: How do we solve DivMBest? Q2: What kind of diversity functions are allowed? Q3: How much diversity?

29 CVPR 2013 Diversity Tutorial Diversity [Special Case] 0-1 Diversity M-Best MAP –[Yanover NIPS03; Fromer NIPS09; Flerova Soft11] [Special Case] Max Diversity [Park & Ramanan ICCV11] Hamming Diversity Cardinality Diversity Any Diversity (C) Dhruv Batra29

30 CVPR 2013 Diversity Tutorial Hamming Diversity (C) Dhruv Batra30 01000100 0 1 0 0 01000100 1 0 0 0

31 CVPR 2013 Diversity Tutorial Hamming Diversity Diversity Augmented Inference: (C) Dhruv Batra31

32 CVPR 2013 Diversity Tutorial Hamming Diversity Diversity Augmented Inference: (C) Dhruv Batra32 Unchanged. Can still use graph-cuts! Simply edit node-terms. Reuse MAP machinery!

33 CVPR 2013 Diversity Tutorial Diverse 2 nd -Best (C) Dhruv Batra33 Q1: How do we solve DivMBest? Q2: What kind of diversity functions are allowed? Q3: How much diversity?

34 CVPR 2013 Diversity Tutorial How Much Diversity? Empirical Solution: Cross-Val for More Efficient: Cross-Val for (C) Dhruv Batra34

35 CVPR 2013 Diversity Tutorial Experiments 3 Applications –Interactive Segmentation: Hamming, Cardinality (in paper) –Pose Estimation: Hamming –Semantic Segmentation: Hamming Baselines: –M-Best MAP (No Diversity) –Confidence-Based Perturbation (No Optimization) (C) Dhruv Batra35

36 CVPR 2013 Diversity Tutorial Interactive Segmentation Setup –Model: Color/Texture + Potts Grid CRF –Inference: Graph-cuts –Dataset: 50 train/val/test images (C) Dhruv Batra36 Image + ScribblesDiverse 2 nd Best2 nd Best MAPMAP 1-2 Nodes Flipped 100-500 Nodes Flipped

37 CVPR 2013 Diversity Tutorial Pose Tracking Setup –Model: Mixture of Parts from [Park & Ramanan, ICCV 11] –Inference: Dynamic Programming –Dataset: 4 videos, 585 frames (C) Dhruv Batra37 Image Credit: [Yang & Ramanan, ICCV 11]

38 CVPR 2013 Diversity Tutorial (C) Dhruv Batra38

39 CVPR 2013 Diversity Tutorial Pose Tracking Chain CRF with M states at each time (C) Dhruv Batra39 M Best Solutions Image Credit: [Yang & Ramanan, ICCV 11]

40 CVPR 2013 Diversity Tutorial Pose Tracking (C) Dhruv Batra40 DivMBest + ViterbiMAP

41 CVPR 2013 Diversity Tutorial Pose Tracking (C) Dhruv Batra41 DivMBest (Re-ranked) [Park & Ramanan, ICCV 11] (Re-ranked) Confidence-based Perturbation (Re-ranked) 13% Gain Same Features Same Model 13% Gain Same Features Same Model #Solutions / Frame PCP Accuracy Better

42 CVPR 2013 Diversity Tutorial Machine Translation Input: Die Regierung will die Folter von Hexen unterbinden und gab eine Broschüre heraus MAP Translation: The government wants the torture of witch and gave out a booklet (C) Dhruv Batra42

43 CVPR 2013 Diversity Tutorial Machine Translation Input: Die Regierung will die Folter von Hexen unterbinden und gab eine Broschüre heraus 5-Best Translations: The government wants the torture of witch and gave out a booklet The government wants the torture of witch and gave out a brochure The government wants the torture of witch and gave out a leaflet The government wants the torture of witch and gave out a brochure (C) Dhruv Batra43

44 CVPR 2013 Diversity Tutorial Machine Translation Input: Die Regierung will die Folter von Hexen unterbinden und gab eine Broschüre heraus Diverse 5-Best Translations: The government wants the torture of witch and gave out a booklet The government wants to stop torture of witch and issued a leaflet issued The government wants to stop the torture of witches and gave out a brochure The government intends to the torture of witchcraft and were issued a leaflet The government is the torture of witches stamp out and gave a brochure (C) Dhruv Batra44

45 CVPR 2013 Diversity Tutorial Machine Translation Input: Die Regierung will die Folter von Hexen unterbinden und gab eine Broschüre heraus Diverse 5-Best Translations: The government wants the torture of witch and gave out a booklet The government wants to stop torture of witch and issued a leaflet issued The government wants to stop the torture of witches and gave out a brochure The government intends to the torture of witchcraft and were issued a leaflet The government is the torture of witches stamp out and gave a brochure Correct Translation: The government wants to limit the torture of witches, a brochure was released (C) Dhruv Batra45


Download ppt "CVPR 2013 Diversity Tutorial Diverse M-Best Solutions in Markov Random Fields Dhruv Batra Virginia Tech Joint work with: Students: Payman Yadollahpour."

Similar presentations


Ads by Google