1. Selected Topics in Graphical Models Petr Šimeček

2. Independence  Unconditional Independence: Discrete r.v. Continuous r.v.  Conditional Independence: Discrete r.v. Continuous r.v.

3. List of Independence Relationships N random variables X 1, X 2, …, X N and their distribution P List of all conditional and unconditional independence relations between them

4. Representation by Graph X1 X2 X3 X4X5 X6 X1X1 X3X3 X2X2 X4X4

5. Example – Sprinkler Network Rain Wet Grass Sprink ler Cloudy

6. Example – Sprinkler Network Rain Wet Grass Sprink ler Cloudy CLO UDY TF 0.5

7. Example – Sprinkler Network Rain Wet Grass Sprink ler Cloudy SPRINKTF C=T0.10.9 C=F0.5 CLO UDY TF 0.5 RAINTF C=T0.80.2 C=F0.20.8

8. Example – Sprinkler Network Rain Wet Grass Sprink ler Cloudy SPRINKTF C=T0.10.9 C=F0.5 CLO UDY TF 0.5 WET GRASSTF R=TS=T0.990.01 R=TS=F0.90.1 R=FS=T0.90.1 R=FS=F01 RAINTF C=T0.80.2 C=F0.20.8

9. Example – Sprinkler Network R W S C The number of parameters needn’t grow exponentially with the number of variables! It depends on the number of parents of nodes.

10. Purpose 1– Propagation of Evidence Rain Wet Grass Sprink ler Cloudy What is the probability that it is raining if we know that grass is wet?

11. Propagation of Evidence In general: I have observed some variable(s). What is the probability of other variable(s)? What is the most probable value(s)? Why don’t transfer BN to contingency table? Marginalization does not work for N large: needs 2 N memory, much time, has low precision…

12. Propagation of Evidence In general: I have observed some variable(s). What is the probability of other variable(s)? What is the most probable value(s)? Why don’t transfer BN to contingency table? Marginalization does not work for N large: needs 2 N memory, much time, has low precision…

13. Purpose 2 – Parameter Learning Rain Wet Grass Sprink ler Cloudy SPRINKTF C=T?? C=F?? CLO UDY TF ?? WET GRASSTF R=TS=T?? R=TS=F?? R=FS=T?? R=FS=F?? RAINTF C=T?? C=F??

14. Parameter Learning We know:  graph (CI structure)  sample (observations) of BN We don’t know:  conditional probabilistic distributions (could be estimated by MLE, Bayesian stat.)

15. Purpose 3 – Structure Learning CLOUDYSPRINKLERRAINWET GRASS TRUEFALSE TRUEFALSE TRUEFALSETRUEFALSE TRUEFALSE TRUE FALSETRUE FALSETRUEFALSE …………

16. Structure Learning We know:  independent observations (data) of BN  sometimes, the casual ordering of vars We don’t know:  graph (CI structure)  conditional probabilistic distributions Solution:  CI tests  maximization of some criterion – huge s. space (AIC, BIC, Bayesian approach)

17. Example – Entry Examination

18. Markov Equivalence Some of arcs can be changed without changing CI relationships. The best one can hope to do is to identify the model up to Markov equivalence. Rain Wet Grass Rain Wet Grass

19. Structure Learning  Theory algorithms proved to be asymptotically right Janžura, Nielsen (2003) 1 000 000 observations for 10 binary variables  Practice in medicine – usually 50-1500 obs. BNs are often used in spite of that

20. Structure Learning - Simulation  3 variables, take m from 100 to 1000  for each m do 100 times generate of Bayesian network generate m samples use K2 structure learning algorithm  count the probability of successful selection for each m This should give an answer to the question: “Is it a chance to find the true model?”

22. To Do List:  software: free, open source, easy to use, fast, separated API  more simulation: theory x practice  popularization of structural learning  Czech literature: maybe my PhD. thesis

23. References:  Castillo E. et al. (1997): Expert Systems and Probabilistic Network Models, Springer Verlag.  Neapolitan R. E. (2003): Learning Bayesian Networks, Prentice Hall.  Janžura N., Nielsen J. (2003): A numerical method for learning Bayesian Networks from Statistical Data, WUPES.