1. Local structures; Causal Independence, Context-sepcific independance COMPSCI 276 Fall 2007

2. Local structure2 Reducing parameters of families Determinizm Causal independence Context-specific independanc Continunous variables

3. Local structure3

4. 4 Causal Independence Event X has two possible causes: A,B. It is hard to elicit P(X|A,B) but it is easy to determine P(X|A) and P(X|B). Example: several diseases causes a symptom. Effect of A on X is independent from the effect of B on X  Causal Independence, using canonical models: Noisy-O, Noisy AND, noisy-max AB X

5. Local structure5 Binary OR AB X ABP(X=0|A,B) 001 P(X=1|A,B) 0 0101 1001 1101

6. Local structure6 Noisy-OR “noise” is associated with each edge described by noise parameter  [0,1] : Let q b =0.2, q a =0.1 P(x=0|a,b)= (1- a ) (1- b ) P(x=1|a,b)=1-(1- a ) (1- b ) AB X ABP(X=0|A,B) 001 P(X=1|A,B) 0 a b 010.10.9 100.20.8 110.020.98 qi=P(X=0|A_i=1,…else =0)

7. Local structure7 Noisy-OR with Leak Use leak probability 0  [0,1] when both parents are false: Let a =0.2, b =0.1, 0 = 0.0001 P(x=0|a,b)= (1- 0 )(1- a ) a (1- b ) b P(x=0|a,b)=1-(1- 0 )(1- a ) a (1- b ) b AB X ABP(X=0|A,B) 000.9999 P(X=1|A,B) 0.0001 a b 010.10.9 100.20.8 110.020.98

8. Local structure8 Formal Definition for Noisy-Or Definition 1 Let Y be a binary-valued random variable with k binary-valued parents X 1,…,X k. The CPT P(Y|X 1,…X k ) is a noisy-or if there are k+1 noise parameters 0, 1,… k such that P(y=0| X 1,…,X k ) = (1- 0 )  i,Xi=1 (1- i )

9. Local structure9 Closed Form Bel(X) - 1 Given: noisy-or CPT P(x|u) noise parameters i T u = {i: U i = 1} Define: q i = 1 - I, Then: q_i is the probability that the inhibitor for u_i is active while the

10. Local structure10 Closed Form Bel(X) - 2 Using Iterative Belief Propagation : Set pi ix = pi x (u k =1). Then we can show that:

11. Local structure11 Closed Form Bel(X) - 2 Using Iterative Belief Propagation : Set pi ix = pi x (u k =1). Then we can show that:

12. Local structure12 Causal Influence Defined Definition 2 Let Y be a random variable with k parents X 1,…,X k. The CPT P(Y|X 1,…X k ) exhibits independence of causal influence (ICI) if it is described via a network fragment of the structure shown in on the left where CPT of Z is a deterministic functions f. Z Y X1X1 X1X1 X1X1 Z0Z0 Z1Z1 Z2Z2 ZkZk

13. Local structure13

14. Local structure14

15. Local structure15

16. Local structure16

17. Local structure17

18. Local structure18 Context Specific Independence When there is conditional independence in some specific variable assignment

19. Local structure19

20. Local structure20

21. Local structure21

22. Local structure22

23. Local structure23 The impact during inference Causal independence in polytrees is linear during inference Causal independence in general can sometime be exploited but not always CSI can be exploited by using operation (product and summation) over trees.

24. Local structure24 Representing CSI Using decision trees Using decision graphs

25. Local structure25 IntelligenceDifficulty Grade Letter SAT Job Apply A student’s example

26. Local structure26 A S L (0.8,0.2) (0.9,0.1)(0.4,0.6) (0.1,0.9) s1s1 a0a0 a1a1 s0s0 l1l1 l0l0 Tree CPD If the student does not apply, SAT and L are irrelevant Tree-CPD for job

27. Local structure27 Definition of CPD-tree A CPD-tree of a CPD P(Z|pa_Z) is a tree whose leaves are labeled by P(Z) and internal nodes correspond to parents branching over their values.

28. Local structure28 C L2 (0.1,0.9) l2 1 c1c1 c2c2 l2 0 L1 (0.8,0.2)(0.3,0.7) l1 1 l1 0 (0.9,0.1) Letter1 Job Letter2 Choice Captures irrelevant variables

29. Local structure29 Multiplexer CPD A CPD P(Y|A,Z1,Z2,…,Zk) is a multiplexer iff Val(A)=1,2,…k, and P(Y|A,Z1,…Zk)=Z_a Letter1 Letter Letter2 Choice Job

30. Local structure30 A B C (0.3,0.7)(0.4,0.6) (0.1,0.9) b1b1 a0a0 a1a1 b0b0 c1c1 c0c0 C B (0.3,0.7)(0.5,0.5) (0.2,0.8) c1c1 c0c0 b1b1 b0b0 Rule-based representation A CPD-tree that correponds to rules.

31. Continuous Variables ICS 275b 2002

32. Local structure32 Gaussian Distribution N( ,  )

33. Local structure33 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 -3-20123 gaussian(x,0,1) gaussian(x,1,1) N( ,  )

34. Local structure34 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 -3-20123 gaussian(x,0,1) gaussian(x,0,2) N( ,  )

35. Local structure35 Multivariate Gaussian Definition: Let X 1,…,X n. Be a set of random variables. A multivariate Gaussian distribution over X 1,…,X n is a parameterized by an n-dimensional mean vector  and an n x n positive definitive covariance matrix . It defines a joint density via:

36. Local structure36 Linear Gaussian Distribution Definition: Let Y be a continuous node with continuous parents X 1,…,X k. We say that Y has a linear Gaussian model if it can be described using parameters  0, …,  k and  2 such that: P(y| x 1,…,x k )=N (  0 +  1 x 1 +…,  k x k ;  2 )

37. Local structure37 XY XYXY

38. Local structure38

39. Local structure39 Linear Gaussian Network Definition Linear Gaussian Bayesian network is a Bayesian network all of whose variables are continuous and where all of the CPTs are linear Gaussians. Linear Gaussian BN  Multivariate Gaussian =>Linear Gaussian BN has a compact representation

40. Local structure40 Hybrid Models Continuous Node, Discrete Parents (CLG) –Define density function for each instantiation of parents Discrete Node, Continuous Parents –Treshold –Sigmoid

41. Local structure41 Continuous Node, Discrete Parents Definition: Let X be a continuous node, and let U={U 1,U 2,…,U n } be its discrete parents and Y={Y 1,Y 2,…,Y k } be its continuous parents. We say that X has a conditional linear Gaussian (CLG) CPT if, for every value u  D(U), we have a a set of (k+1) coefficients a u,0, a u,1, …, a u,k+1 and a variance  u 2 such that:

42. Local structure42 CLG Network Definition: A Bayesian network is called a CLG network if every discrete node has only discrete parents, and every continuous node has a CLG CPT.

43. Local structure43 Discrete Node, Continuous Parents Threshold Model

44. Local structure44 Discrete Node, Continuous Parents Sigmoid Binomial Logit Definition: Let Y be a binary-valued random variable with k continuous-valued parents X 1,…X k. The CPT P(Y|X 1 …X k ) is a linear sigmoid (also called binomial logit) if there are (k+1) weights w 0,w 1,…,w k such that:

45. Local structure45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 05101520 sigmoid(0.1*x) sigmoid(0.5*x) sigmoid(0.9*x)

46. Local structure46 References Judea Pearl “Probabilistic Reasoning in Inteeligent Systems”, section 4.3 Nir Friedman, Daphne Koller “Bayesian Network and Beyond”