1. A Markov Random Field Model for Term Dependencies Chetan Mishra CS 6501 Paper Presentation Ideas, graphs, charts, and results from paper of same name by Metzler and Croft 2005 (SIGIR)

2. Agenda 1.Motivation behind the work 2.Background – What is a Markov Random Field (MRF)? 3.Research Insight – How did the authors use MRF to model term dependencies? Results? 4.Future Work – If you thought this was interesting, how could you build on this? 5.Conclusion CS@UVaCS 6501: Text Mining2 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

3. Motivation Terms are not independently distributed – A model incorporating term dependencies should outperform a model that ignores them One problem: models incorporating term dependencies seemed no better or worse – Statistical models weren’t effectively modeling term dependencies – Why? CS@UVaCS 6501: Text Mining3 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

4. Motivation Two Problems (perspective of authors): – Problem 1: Most models have taken bag of word- like approaches (which have tremendous data requirements) – Solution 1: We need a new type of model – Problem 2: Term dependency modeling (even with a reasonable model) requires a significant corpus – Solution 2: Add to research testing collections large, web-scraped corpuses CS@UVaCS 6501: Text Mining4 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

5. Background What is a Markov random field (MRF) model? – Fancy name for a bidirectional graph-based model – Often used in machine learning to succinctly model joint distributions MRF models are used in the paper to tackle the problem of document retrieval with response to a query CS@UVaCS 6501: Text Mining5 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

6. Model Overview CS@UVaCS 6501: Text Mining6 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

7. Model Overview CS@UVaCS 6501: Text Mining7 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

8. Model Overview CS@UVaCS 6501: Text Mining8 By the joint probability law AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

9. The Markov Random Field Model CS@UVaCS 6501: Text Mining9 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

10. The Markov Random Field Model CS@UVaCS 6501: Text Mining10 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

11. The Markov Random Field Model The paper looks at the performance of three general types of dependencies: – Independence – Sequential dependence – Full dependence Visual Depiction: CS@UVaCS 6501: Text Mining11 Metzler and Croft ‘05 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

12. Potential Functions CS@UVaCS 6501: Text Mining12 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion All log scale!

13. Parameter Training CS@UVaCS 6501: Text Mining13 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

14. Parameter Training What optimization technique do we use? – Authors found a shape common to the metric surface via parameter sweep A hill-climbing search should work well CS@UVaCS 6501: Text Mining14 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

15. Results Did MRF’s help? – I’d say so. Significant gains across data sets CS@UVaCS 6501: Text Mining15 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion IndependentSequential DependenceFull Dependence

16. Future Work CS@UVaCS 6501: Text Mining16 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

17. 1.Motivation behind the work 2.Background – What is a Markov Random Field Model (MRF)? 3.Research Insight – How did the authors use MRF to model term dependencies? Results? 4.Future Work – If you thought this was interesting, how could you build on this? 5.Conclusion CS@UVaCS 6501: Text Mining17 AgendaMotivationBackgroundResearch InsightFuture WorkConclusion

18. Questions? CS@UVaCS 6501: Text Mining18