1. Notes on Graphical Models Padhraic Smyth Department of Computer Science University of California, Irvine

2. Probabilistic Model Real World Data P(Data | Parameters)

3. Probabilistic Model Real World Data P(Data | Parameters) P(Parameters | Data)

4. Probabilistic Model Real World Data P(Data | Parameters) P(Parameters | Data) Generative Model, Probability Inference, Statistics

5. Part 1: Review of Probability

6. Notation and Definitions X is a random variable –Lower-case x is some possible value for X –“X = x” is a logical proposition: that X takes value x –There is uncertainty about the value of X e.g., X is the Dow Jones index at 5pm tomorrow p(X = x) is the probability that proposition X=x is true –often shortened to p(x) If the set of possible x’s is finite, we have a probability distribution and  p(x) = 1 If the set of possible x’s is infinite, p(x) is a density function, and p(x) integrates to 1 over the range of X

7. Example Let X be the Dow Jones Index (DJI) at 5pm Monday August 22 nd (tomorrow) X can take real values from 0 to some large number p(x) is a density representing our uncertainty about X –This density could be constructed from historical data, e.g., –After 5pm p(x) = 1 for some value of x (no uncertainty), once we hear from Wall Street what x is

8. Probability as Degree of Belief Different agents can have different p(x)’s –Your p(x) and the p(x) of a Wall Street expert might be quite different –OR: if we were on vacation we might not have access to stock market information we would still be uncertain about p(x) after 5pm So we should really think of p(x) as p(x | B I ) –Where B I is background information available to agent I –(will drop explicit conditioning on B I in notation) Thus, p(x) represents the degree of belief that agent I has in proposition x, conditioned on available background information

9. Comments on Degree of Belief Different agents can have different probability models –There is no necessarily “correct” p(x) –Why? Because p(x) is a model built on whatever assumptions or background information we use –Naturally leads to the notion of updating p(x | B I ) -> p(x | B I, C I ) This is the subjective Bayesian interpretation of probability –Generalizes other interpretations (such as frequentist) –Can be used in cases where frequentist reasoning is not applicable –We will use “degree of belief” as our interpretation of p(x) in this tutorial Note! –Degree of belief is just our semantic interpretation of p(x) –The mathematics of probability (e.g., Bayes rule) remain the same regardless of our semantic interpretation

10. Multiple Variables p(x, y, z) –Probability that X=x AND Y=y AND Z =z –Possible values: cross-product of X Y Z –e.g., X, Y, Z each take 10 possible values x,y,z can take 10 3 possible values p(x,y,z) is a 3-dimensional array/table –Defines 10 3 probabilities Note the exponential increase as we add more variables –e.g., X, Y, Z are all real-valued x,y,z live in a 3-dimensional vector space p(x,y,z) is a positive function defined over this space, integrates to 1

11. Conditional Probability p(x | y, z) –Probability of x given that Y=y and Z = z –Could be hypothetical, e.g., “if Y=y and if Z = z” observational, e.g., we observed values y and z –can also have p(x, y | z), etc –“all probabilities are conditional probabilities” Computing conditional probabilities is the basis of many prediction and learning problems, e.g., –p(DJI tomorrow | DJI index last week) –expected value of [DJI tomorrow | DJI index next week) –most likely value of parameter  given observed data

12. Computing Conditional Probabilities Variables A, B, C, D –All distributions of interest related to A,B,C,D can be computed from the full joint distribution p(a,b,c,d) Examples, using the Law of Total Probability –p(a) =  {b,c,d} p(a, b, c, d) –p(c,d) =  {a,b} p(a, b, c, d) –p(a,c | d) =  {b} p(a, b, c | d) where p(a, b, c | d) = p(a,b,c,d)/p(d) These are standard probability manipulations: however, we will see how to use these to make inferences about parameters and unobserved variables, given data

13. Two Practical Problems (Assume for simplicity each variable takes K values) Problem 1: Computational Complexity –Conditional probability computations scale as O(K N ) where N is the number of variables being summed over Problem 2: Model Specification –To specify a joint distribution we need a table of O(K N ) numbers –Where do these numbers come from?

14. Two Key Ideas Problem 1: Computational Complexity –Idea: Graphical models Structured probability models lead to tractable inference Problem 2: Model Specification –Idea: Probabilistic learning General principles for learning from data

15. Part 2: Graphical Models

16. “…probability theory is more fundamentally concerned with the structure of reasoning and causation than with numbers.” Glenn Shafer and Judea Pearl Introduction to Readings in Uncertain Reasoning, Morgan Kaufmann, 1990

17. Conditional Independence A is conditionally independent of B given C iff p(a | b, c) = p(a | c) (also implies that B is conditionally independent of A given C) In words, B provides no information about A, if value of C is known Example: –a = “reading ability” –b = “height” –c = “age” Note that conditional independence does not imply marginal independence

18. Graphical Models Represent dependency structure with a directed graph –Node random variable –Edges encode dependencies Absence of edge -> conditional independence –Directed and undirected versions Why is this useful? –A language for communication –A language for computation Origins: –Wright 1920’s –Independently developed by Spiegelhalter and Lauritzen in statistics and Pearl in computer science in the late 1980’s

19. Examples of 3-way Graphical Models ACB Marginal Independence: p(A,B,C) = p(A) p(B) p(C)

20. Examples of 3-way Graphical Models A CB Conditionally independent effects: p(A,B,C) = p(B|A)p(C|A)p(A) B and C are conditionally independent Given A e.g., A is a disease, and we model B and C as conditionally independent symptoms given A

21. Examples of 3-way Graphical Models A B C Independent Causes: p(A,B,C) = p(C|A,B)p(A)p(B)

22. Examples of 3-way Graphical Models ACB Markov dependence: p(A,B,C) = p(C|B) p(B|A)p(A)

23. Real-World Example Monitoring Intensive-Care Patients 37 variables 509 parameters …instead of 2 37 (figure courtesy of Kevin Murphy/Nir Friedman) PCWP CO HRBP HREKG HRSAT ERRCAUTER HR HISTORY CATECHOL SAO2 EXPCO2 ARTCO2 VENTALV VENTLUNG VENITUBE DISCONNECT MINVOLSET VENTMACH KINKEDTUBE INTUBATIONPULMEMBOLUS PAPSHUNT ANAPHYLAXIS MINOVL PVSAT FIO2 PRESS INSUFFANESTHTPR LVFAILURE ERRBLOWOUTPUT STROEVOLUMELVEDVOLUME HYPOVOLEMIA CVP BP

24. Directed Graphical Models A B C p(A,B,C) = p(C|A,B)p(A)p(B)

25. Directed Graphical Models A B C In general, p(X 1, X 2,....X N ) =  p(X i | parents(X i ) ) p(A,B,C) = p(C|A,B)p(A)p(B)

26. Directed Graphical Models A B C Probability model has simple factored form Directed edges => direct dependence Absence of an edge => conditional independence Also known as belief networks, Bayesian networks, causal networks In general, p(X 1, X 2,....X N ) =  p(X i | parents(X i ) ) p(A,B,C) = p(C|A,B)p(A)p(B)

27. Reminders from Probability…. Law of Total Probability P(a) =  b P(a, b) =  b P(a | b) P(b) –Conditional version: P(a|c) =  b P(a, b|c) =  b P(a | b, c) P(b|c) Factorization or Chain Rule –P(a, b, c, d) = P(a | b, c, d) P(b | c, d) P(c | d) P (d), or = P(b | a, c, d) P(c | a, d) P(d | a) P(a), or = …..

28. Graphical Models for Computation PCWP CO HRBP HREKG HRSAT ERRCAUTER HR HISTORY CATECHOL SAO2 EXPCO2 ARTCO2 VENTALV VENTLUNG VENITUBE DISCONNECT MINVOLSET VENTMACH KINKEDTUBE INTUBATIONPULMEMBOLUS PAPSHUNT ANAPHYLAXIS MINOVL PVSAT FIO2 PRESS INSUFFANESTHTPR LVFAILURE ERRBLOWOUTPUT STROEVOLUMELVEDVOLUME HYPOVOLEMIA CVP BP Say we want to compute P(BP|Press) Law of total probability: -> must sum over all other variables -> exponential in # variables Factorization: -> joint distribution factors into smaller tables Can now sum over smaller tables, can reduce complexity dramatically

29. Example D A B C F E G

30. D A B C F E G p(A, B, C, D, E, F, G) =  p( variable | parents ) = p(A|B)p(C|B)p(B|D)p(F|E)p(G|E)p(E|D) p(D)

31. Example D A B c F E g Say we want to compute p(a | c, g)

32. Example D A B c F E g Direct calculation: p(a|c,g) =  bdef p(a,b,d,e,f | c,g) Complexity of the sum is O(K 4 )

33. Example D A B c F E g Reordering (using factorization):  b p(a|b)  d p(b|d,c)  e p(d|e)  f p(e,f |g)

34. Example D A B c F E g Reordering:  b  p(a|b)  d p(b|d,c)  e p(d|e)  f p(e,f |g) p(e|g)

35. Example D A B c F E g Reordering:  b  p(a|b)  d p(b|d,c)  e p(d|e) p(e|g) p(d|g)

36. Example D A B c F E g Reordering:  b  p(a|b)  d p(b|d,c) p(d|g) p(b|c,g)

37. Example D A B c F E g Reordering:  b  p(a|b) p(b|c,g) p(a|c,g) Complexity is O(K), compared to O(K 4 )

38. Graphs with “loops” D A B C F E G Message passing algorithm does not work when there are multiple paths between 2 nodes

39. Graphs with “loops” D A B C F E G General approach: “cluster” variables together to convert graph to a tree

40. Reduce to a Tree D A B, E C F G

41. Probability Calculations on Graphs General algorithms exist - beyond trees –Complexity is typically O(m (number of parents ) ) (where m = arity of each node) –If single parents (e.g., tree), -> O(m) –The sparser the graph the lower the complexity Technique can be “automated” –i.e., a fully general algorithm for arbitrary graphs –For continuous variables: replace sum with integral –For identification of most likely values Replace sum with max operator

42. Part 3: Learning with Graphical Models Further Reading: M. Jordan, Graphical models, Statistical Science: Special Issue on Bayesian Statistics, vol. 19, no. 1, pp. 140-155, Feb. 2004 A.Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis (2 nd ed), Chapman and Hall, 2004

43. Probabilistic Model Real World Data P(Data | Parameters) P(Parameters | Data) Generative Model, Probability Inference, Statistics

44. The Likelihood Function Likelihood = p(data | parameters) = p( D |  ) = L (  ) Likelihood tells us how likely the observed data are conditioned on a particular setting of the parameters Details –Constants that do not involve  can be dropped in defining L (  ) –Often easier to work with log L (  )

45. Comments on the Likelihood Function Constructing a likelihood function L (  ) is the first step in probabilistic modeling The likelihood function implicitly assumes an underlying probabilistic model M with parameters  L (  ) connects the model to the observed data Graphical models provide a useful language for constructing likelihoods

46. Binomial Likelihood Binomial model –N memoryless trials, 2 outcomes –probability  of success at each trial Observed data –r successes in n trials –Defines a likelihood: L(  ) = p(D |  ) = p(successes) p(non-successes) =  r (1- ) n-r

47. Binomial Likelihood Examples

48. Multinomial Likelihood Multinomial model –N memoryless trials, K outcomes –Probability vector  for outcomes at each trial Observed data –n j successes in n trials –Defines a likelihood: –Maximum likelihood estimates:

49. Graphical Model for Multinomial w1w1   = [ p(w 1 ), p(w 2 ),….. p(w k ) ] w2w2 wnwn   Parameters  Observed data

50. “Plate” Notation wiwi i=1:n  Data = D = {w 1,…w n } Model parameters Plate (rectangle) indicates replicated nodes in a graphical model Variables within a plate are conditionally independent manner given parent

51. Learning in Graphical Models wiwi i=1:n  Data = D = {w 1,…w n } Model parameters Can view learning in a graphical model as computing the most likely value of the parameter node given the data nodes

52. Maximum Likelihood (ML) Principle (R. Fisher ~ 1922) wiwi i=1:n  L () = p(Data |  ) =  p(y i |  ) Maximum Likelihood:  ML = arg max{ Likelihood() } Select the parameters that make the observed data most likely Data = {w 1,…w n } Model parameters

53. The Bayesian Approach to Learning wiwi i=1:n  Fully Bayesian: p(  | Data) = p(Data |  ) p() / p(Data) Maximum A Posteriori:  MAP = arg max{ Likelihood() x Prior() }  Prior() = p(  )

54. Learning a Multinomial Likelihood: same as before Prior: p(   ) = Dirichlet ( 1,… K ) proportional to –Has mean Prior weight for  j Can set all  j =  for “uniform” prior

55. Dirichlet Shapes From: http://en.wikipedia.org/wiki/Dirichlet_distribution

56. Bayesian Learning P( | D, ) is proportional to p(data | ) p() = = Dirichlet( n 1 +  1,…, n K +  K ) Posterior mean estimate

57. Summary of Bayesian Learning Can use graphical models to describe relationships between parameters and data P(data | parameters) = Likelihood function P(parameters) = prior –In applications such as text mining, prior can be “uninformative”, i.e., flat –Prior can also be optimized for prediction (e.g., on validation data) We can compute P(parameters | data, prior) or a “point estimate” (e.g., posterior mode or mean) Computation of posterior estimates can be computationally intractable – Monte Carlo techniques often used