Bayesian Belief Networks (BBN) A Bayesian Belief Network is a method to describe the joint probability distribution of a set of variables. Let x1, x2, …, xn be a set of random variables. A Bayesian Belief Network or BBN will tell us the probability of any combination of x1, x2,.., xn.
Representation A BBN represents the joint probability distribution of a set of variables by explicitly indicating the assumptions of conditional independence through the following: a)Nodes representing random variables b)Directed links representing relations. c)Conditional probability distributions. d) The graph is a directed acyclic graph.
Representation Each variable is independent of its non-descendants given its predecessors. We say x1 is a descendant of x2 if there is a direct path from x2 to x1. Example: Predecessors of Alarm: Burglary, Earthquake.
Joint Probability Distribution To compute the joint probability distribution of a set of variables given a Bayesian Belief Network we simply use the following formula: P(x1,x2,…,xn) = Π P(xi | Parents(xi)) Where parents are the immediate predecessors of xi.
Joint Probability Distribution Example: P(John, Mary,Alarm,~Burglary,~Earthquake) : P(John|Alarm) P(Mary|Alarm) P(Alarm|~Burglary ^ ~Earthquake) P(~Burglary) P(~Earthquake) = 0.00062
Conditional Probabilities Alarm Burglary Earthquake B E P(A) t t 0.95 t f 0.94 f t 0.29 f f 0.001
Constructing Bayesian Networks Choose the right order from causes to effects. P(x1,x2,…,xn) = P(xn|xn-1,..,x1)P(xn-1,…,x1) = Π P(xi|xi-1,…,x1) -- chain rule Example: P(x1,x2,x3) = P(x1|x2,x3)P(x2|x3)P(x3)
How to construct BBN P(x1,x2,x3) x3 x2 x1 root cause leaf Correct order: add root causes first, and then “leaves”, with no influence on other nodes.
Compactness BBN are locally structured systems. They represent joint distributions compactly. Assume n random variables, each influenced by k nodes. Size BBN: n2 k Full size: 2 n
Noisy-OR Now the whole probability can be built: P(~fever | cold, ~flu, malaria) = 0.6 x 0.1 P(~fever | cold, flu, ~malaria) = 0.6 x 0.2 P(~fever | ~cold, flu, malaria) = 0.2 x 0.1 P(~fever | cold, flu, malaria) = 0.6 x 0.2 x 0.1 P(~fever | ~cold, ~flu, ~malaria) = 1.0
Continuous Variables Continuous variables can be discretized. Or define probability density functions Example: Gaussian distribution. A network with both variables is called a Hybrid Bayesian Network.
Bayesian networks are directed acyclic graphs that concisely represent conditional independence relations among random variables. BBN specify the full joint probability distribution of a set of variables. BBN can by hybrid, combining categorical variables with numeric variables.