1. 1 Knowledge Engineering for Bayesian Networks

2. 2 Probability theory for representing uncertainty l Assigns a numerical degree of belief between 0 and 1 to facts »e.g. “it will rain today” is T/F. »P(“it will rain today”) = 0.2 prior probability (unconditional) l Conditional probability (Posterior) »P(“it wil rain today” | “rain is forecast”) = 0.8 l Bayes’ Rule: P(H|E) = P(E|H) x P(H) P(E)

3. 3 Bayesian networks l Directed acyclic graphs l Nodes: random variables, »R: “it is raining”, discrete values T/F »T: temperature, continuous or discrete variable »C: color, discrete values {red, blue, green} l Arcs indicate dependencies (can have causal interpretation)

4. 4 Bayesian networks l Conditional Probability Distribution (CPD) –Associated with each variable –probability of each state given parent states “Jane has the flu” “Jane has a high temp” “Thermometer temp reading” X Flu Y Te Q Th Models causal relationship Models possible sensor error P(Flu=T) = 0.05 P(Te=High|Flu=T) = 0.4 P(Te=High|Flu=F) = 0.01 P(Th=High|Te=H) = 0.95 P(Th=High|Te=L) = 0.1

5. 5 Inference in Belief Networks l Main task of a belief network: Compute the conditional probability of a set of query variables given exact values for some evidence variables: P(query | evidence). l Belief networks are flexible enough so that any node can serve as either a query or an evidence variable.

6. 6 BN inference l Evidence: observation of specific state l Task: compute the posterior probabilities for query node(s) given evidence. Th Y Flu Te Diagnostic inference Th Flu Y Te Causal inference Intercausal inference Te FluTB Flu Mixed inference Th Flu Te

7. 7 Building a BN l Choose a set of random variables X i that describe the domain. »Missing variables may cause the BN unreliable.

8. 8 Building a BN l Choose a set of random variables X i that describe the domain. l Order the variables into a list L l Start with an empty BN. l For each variable X in L do »Add X into the BN »Choose a minimal set of nodes already in the BN which satisfy the conditional dependence property with X »Make these nodes the parents of X. »Fill in the CPT for X.

9. 9 The Alarm Example l Mr. Holmes’ security alarm at home may be triggered by either burglar or earthquake. When the alarm sounds, his two nice neighbors, Mary and John, may call him. causal DAG

10. 10 The Alarm Example l Variable order: »Burglary »Earthquake »Alarm »JohnCalls »MaryCalls BN

11. 11 The Alarm Example l Variable order: »MaryCalls »JohnCalls »Alarm »Burglary »Earthquake BN

12. 12 The Alarm Example l Variable order: »MaryCalls »JohnCalls »Earthquake »Burglary »Alarm BN

13. 13 Weakness of BN l Hard to obtain JPD (joint probability distribution) »Relative Frequency Approach: counting outcomes of repeated experiments »Subjective Approach: an individual's personal judgment about whether a specific outcome is likely to occur. l Worst time complexity is NP-hard.

14. 14 BN software l Commerical packages: Netica, Hugin, Analytica (all with demo versions) l Free software: Smile, Genie, JavaBayes, … http://HTTP.CS.Berkeley.EDU/~murphyk/Bayes/bnsoft. html l Example running Netica software

15. 15 What’s Netica? l Netica is a powerful, easy-to-use, complete program for working with belief networks and influence diagrams. It has an intuitive and smooth user interface for drawing the networks, and the relationships between variables may be entered as individual probabilities, in the form of equations, or learned from data files.belief networks

16. 16 Netica Screen Shot Priori probabilities are needed for each variables. Netica will compute CPT (conditional probability table).

17. 17 Netica Screen Shot P(Jewelry = yes | Age < 30, Sex = Male) P(Fraud = yes | Jewelry = yes, Age < 30, Sex = male)

18. 18 Netica Screen Shot P(Fraud = yes | Gas = yes, Jewelry = yes, Age < 30, Sex = male) P(Fraud = yes | Gas = yes, Jewelry = yes, Age > 50, Sex = female)

19. 19 Extensions of BN l Weaker requirement in a DAG: Instead of I(X, ND X | PA X ), ask I(X, ND X | MB X ), where MB X is called Markov Blanket of X, which is the set of neighboring nodes: its parents (PA X ), its children, and any other parents of X’s children. PA B = { H } MB B = { H, L, F } ND B = { L, X }

20. 20 Open Research Questions l Methodology for combining expert elicitation and automated methods »expert knowledge used to guide search »automated methods provide alternatives to be presented to experts l Evaluation measures and methods »may be domain depended l Improved tools to support elicitation »e.g. visualisation of d-separation l Industry adoption of BN technology