1. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring, 2006 Graphical Models Jeff A. Bilmes Lecture 18 Slides May 31 st, 2006

2. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 2 READING: –Google search on “bayesian networks” “approximate inference” No more homework this quarter, concentrate on final projects!! Final project presentations tomorrow in class!! Announcements

3. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 3 L1: Tues, 3/28: Overview, GMs, Intro BNs. L2: Thur, 3/30: semantics of BNs + UGMs L3: Tues, 4/4: elimination, probs, chordal I L4: Thur, 4/6: chrdal, sep, decomp, elim L5: Tue, 4/11: chdl/elim, mcs, triang, ci props. L6: Thur, 4/13: MST,CI axioms, Markov prps. L7: Tues, 4/18: Mobius, HC-thm, (F)=(G) L8: Thur, 4/20: phylogenetic trees, HMMs L9: Tue, 4/25: HMMs, inference on trees L10: Thur, 4/27: Inference on trees, start poly L11: Tues, 5/2: polytrees, start JT inference L12: Thur, 5/4: Inference in JTs Tues, 5/9: away Thur, 5/11: away L13: Tue, 5/16: JT, GDL, Shenoy-Schafer L14: Thur, 5/18: GDL, Search, Gaussians I L--: Mon, 5/22: laptop crash  L15: Tues, 5/23: search, Gaussians I L16: Thur, 5/25: Gaussians Mon, 5/29: Holiday L17: Tue, 5/30 L18: Wed, 5/31 L19: Thur, 6/1: final presentations L20: Tue, 6/6 Class Road Map

4. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 4 L1: Tues, 3/28: L2: Thur, 3/30: L3: Tues, 4/4: L4: Thur, 4/6: L5: Tue, 4/11: L6: Thur, 4/13: L7: Tues, 4/18: L8: Thur, 4/20: Team Lists, short abstracts I L9: Tue, 4/25: L10: Thur, 4/27: short abstracts II L11: Tues, 5/2: L12: Thur, 5/4: abstract II + progress L--: Tues, 5/9 L--: Thur, 5/11: 1 page progress report L13: Tue, 5/16: L14: Thur, 5/18: 1 page progress report L15: Tues, 5/23 L16: Thur, 5/25: 1 page progress report L17: Tue, 5/30: Today L18: Wed, 5/31: L19: Thur, 6/1: final presentations L20: Tue, 6/6 4-page papers due (like a conference paper), Only.pdf versions accepted. Final Project Milestone Due Dates Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only). Final reports must be turned in electronically in PDF (no other formats accepted). No need to repeat what was on previous progress reports/abstracts, I have those available to refer to. Progress reports must report who did what so far!!

5. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 5 Other forms of inference. Structure learning in graphical models Summary of Last Time

6. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 6 When the inference gets hard … Outline of Today’s Lecture

7. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 7 Books and Sources for Today Various sources on approximate inference (see references in presentation below). Papers by Yedidia, 2000,2002,2004

8. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 8 When exact inference is too expensive 1.Two general approaches: either an exact solution to an approximate problem, or an approximate solution to an exact problem. 2.Exact solution to approximate problem 1.Structure learning: find a low tree-width (or “cheap” in some way) graphical model that is still “high-quality” in some way, and then perform exact inference on the approximate model. 2.This can be easy or hard depending on the tree-width and on the measure of “high-quality”, and on the learning paradigm. 3.Approximate solution to an exact problem 1.Approximate inference, tries to approximate in some way what must be computed: Loopy Belief propagation, Variational/Mean- Field-Bethe/etc., and Sampling/Pruning, and hybrids between the above

9. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 9 Loopy BP

10. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 10 Recall general MP

11. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 11 Towards message passing MP

12. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 12 Loopy BP

13. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 13 Loopy BP

14. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 14 Loopy BP (or just BP)

15. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 15 Loopy BP

16. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 16 Loopy BP and variational

17. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 17 Loopy BP and variational

18. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 18 Variational inference

19. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 19 Variational EM

20. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 20 Variational EM

21. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 21 Variational EM

22. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 22 Mean-field

23. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 23 Bethe Free Energy

24. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 24 Bethe Free Energy

25. Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 25 Bethe Free Energy