1. Graphical Models - Inference -
Mainly based on F. V. Jensen, „Bayesian Networks and Decision Graphs“, Springer-Verlag New York, 2001.
Advanced
I WS 06/07
PCWP CO
HRBP
HREKG
HRSAT
ERRCAUTER HR
HISTORY
CATECHOL
SAO2
EXPCO2
ARTCO2
VENTALV
VENTLUNG
VENITUBE
DISCONNECT
MINVOLSET
VENTMACH
KINKEDTUBE
INTUBATION
PULMEMBOLUS
PAP
SHUNT
ANAPHYLAXIS
MINOVL
PVSAT
FIO2
PRESS
INSUFFANESTH
TPR
LVFAILURE
ERRBLOWOUTPUT
STROEVOLUME
LVEDVOLUME
HYPOVOLEMIA
CVP BP
Graphical Models - Inference -
Variable Elimination
Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel
Albert-Ludwigs University Freiburg, Germany

2. Reminder: Probability theory Basics of Bayesian Networks
Outline
Introduction
Reminder: Probability theory
Basics of Bayesian Networks
Modeling Bayesian networks
Inference
Excourse: Markov Networks
Learning Bayesian networks
Relational Models

3. Elimination in Chains Forward pass
Using definition of probability, we have A B C E D
- Inference (Variable Elimination)

4. Elimination in Chains By chain decomposition, we get A B C E D
- Inference (Variable Elimination)

5. Elimination in Chains Rearranging terms ... A B C E D
- Inference (Variable Elimination)

6. X Elimination in Chains Now we can perform innermost summation A B C E
A has been eliminated A B C E D
Now we can perform innermost summation
- Inference (Variable Elimination)

7. X X … Elimination in Chains Rearranging and then summing again, we get
B has been eliminated A B C E D
Rearranging and then summing again, we get
- Inference (Variable Elimination) …

8. Elimination in Chains with Evidence
Similar due to Bayes’ Rule
We write the query in explicit form A B C E D
- Inference (Variable Elimination)

9. Variable Elimination Write query in the form Iteratively
Move all irrelevant terms outside of innermost sum
Perform innermost sum, getting a new term
Insert the new term into the product
Eliminate irrelevant variables
Semantic Bayesian Network
- Inference (Variable Elimination)

10. Asia - A More Complex Example
Visit to
Asia
Smoking
Lung Cancer
Tuberculosis
Abnormality
in Chest
Bronchitis
X-Ray
Dyspnoea
- Inference (Variable Elimination)

11. We want to compute P(d) Need to eliminate: v,s,x,t,l,a,b
Initial factors
- Inference (Variable Elimination)

12. We want to compute P(d) Need to eliminate: v,s,x,t,l,a,b
Initial factors
Compute:
Eliminate: v
- Inference (Variable Elimination)
but in general the result of eliminating a variable is not necessarily a probability term !

13. We want to compute P(d) Need to eliminate: s,x,t,l,a,bV S L T A B X D
We want to compute P(d) Need to eliminate: s,x,t,l,a,b
Initial factors
Compute:
Eliminate: s
- Inference (Variable Elimination)
Summing on s results in a factor with two arguments fs(b,l). In general, result of elimination may be a function of several variables

14. We want to compute P(d) Need to eliminate: x,t,l,a,bV S L T A B X D
We want to compute P(d) Need to eliminate: x,t,l,a,b
Initial factors
- Inference (Variable Elimination)
Compute:
Eliminate: x
Note that fx(a)=1 for all values of a

15. We want to compute P(d) Need to eliminate: t,l,a,bV S L T A B X D
We want to compute P(d) Need to eliminate: t,l,a,b
Initial factors
- Inference (Variable Elimination)
Compute:
Eliminate: t

16. We want to compute P(d) Need to eliminate: l,a,bV S L T A B X D
We want to compute P(d) Need to eliminate: l,a,b
Initial factors
- Inference (Variable Elimination)
Compute:
Eliminate: l

17. We want to compute P(d) Need to eliminate: a,bV S L T A B X D
We want to compute P(d) Need to eliminate: a,b
Initial factors
- Inference (Variable Elimination)
Compute:
Eliminate: a,b

18. Variable Elimination (VE)
We now understand VE as a sequence of rewriting operations
Actual computation is done in elimination step
Exactly the same procedure applies to Markov networks
Computation depends on order of elimination
- Inference (Variable Elimination)

19. Dealing with Evidence How do we deal with evidence?S L T A B X D
Dealing with Evidence
How do we deal with evidence?
Suppose get evidence V = t, S = f, D = t
We want to compute P(L, V = t, S = f, D = t)
- Inference (Variable Elimination)

20. Dealing with Evidence We start by writing the factors: V S L T A B X D
- Inference (Variable Elimination)

21. Dealing with Evidence We start by writing the factors:X D
Dealing with Evidence
We start by writing the factors:
Since we know that V = t, we don’t need to eliminate V
Instead, we can replace the factors P(V) and P(T|V) with
- Inference (Variable Elimination)

22. Dealing with Evidence We start by writing the factors:X D
Dealing with Evidence
We start by writing the factors:
Since we know that V = t, we don’t need to eliminate V
Instead, we can replace the factors P(V) and P(T|V) with
This “selects” the appropriate parts of the original factors given the evidence
Note that fP(v) is a constant, and thus does not appear in elimination of other variables
- Inference (Variable Elimination)

23. Dealing with Evidence Initial factors, after setting evidence
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t ) V S L T A B X D
Dealing with Evidence
Initial factors, after setting evidence
- Inference (Variable Elimination)

24. Dealing with Evidence Initial factors, after setting evidence
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t ) V S L T A B X D
Dealing with Evidence
Initial factors, after setting evidence
Eliminating x, we get
- Inference (Variable Elimination)

25. Dealing with Evidence Initial factors, after setting evidence
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t ) V S L T A B X D
Dealing with Evidence
Initial factors, after setting evidence
Eliminating x, we get
Eliminating t, we get
- Inference (Variable Elimination)

26. Dealing with Evidence Initial factors, after setting evidence
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t ) V S L T A B X D
Dealing with Evidence
Initial factors, after setting evidence
Eliminating x, we get
Eliminating t, we get
Eliminating a, we get
- Inference (Variable Elimination)

27. Dealing with Evidence Initial factors, after setting evidence
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t ) V S L T A B X D
Dealing with Evidence
Initial factors, after setting evidence
Eliminating x, t, a, we get
Finally, when eliminating b, we get
- Inference (Variable Elimination)

28. Dealing with Evidence Initial factors, after setting evidence
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t ) V S L T A B X D
Dealing with Evidence
Initial factors, after setting evidence
Eliminating x, t, a, we get
Finally, when eliminating b, we get
- Inference (Variable Elimination)

29. Reminder: Probability theory Basics of Bayesian Networks
Outline
Introduction
Reminder: Probability theory
Basics of Bayesian Networks
Modeling Bayesian networks
Inference (VE, Junction Tree)
Excourse: Markov Networks
Learning Bayesian networks
Relational Models