1. Probabilistic Reasoning; Network-based reasoning
Set 7
ICS 179, Spring 2010

2. Propositional Reasoning
Example: party problem
= A
= B
If Alex goes, then Becky goes:
If Chris goes, then Alex goes:
Question:
Is it possible that Chris goes to the party but Becky does not?
= C
= A
To illustrate these concepts let me consider a simple example
Consider a simple scenario involving people going to parties. Lets assume that we have three individuals,
Alex, Becky and Chris. And we have some knowledge about their social relationships in regard to party-going.
Show……Query.
How can we do this kind of simple reasoning by a computer?
We have to first formalize the information in the sentences
and “represent this sentences” in some computer language .
We can associate symbols with fragments in the sentence which can either be true or false
(we call those “propositions”) and then describe the information In the sentence by logical rules (knowledge-representation).
Subsequently we need to be able to derive answers to queries, and to do it “automatically,” namely,
to engage in “automated reasoning.”
Automated reasoning is the study of algorithms and computer programs that answer such queries .
There are many disciplines that contributed, ideas, viewpoints and techniques to AI. Mathematical logic had a
tremendous influence on AI in the early days and even today.
In general to become a formal science AI relied heavily on three fundamental areas: logic, probability and computation.
Chavurah 5/8/2010

3. Probabilistic Reasoning
Party example: the weather effect
Alex is-likely-to-go in bad weather
Chris rarely-goes in bad weather
Becky is indifferent but unpredictable
Questions:
Given bad weather, which group of individuals is most likely to show up at the party?
What is the probability that Chris goes to the party but Becky does not?
P(A|W=bad)=.9 W A
P(C|W=bad)=.1 C
P(B|W=bad)=.5 B W A
P(A|W)
good
.01 1
.99
bad .1 .9
Lets go back now to the party example but have a factor we did not have before.
Lets say that on the day of the party the weather was bad. How would this affect our party-goers?
(Remember Drew’s holiday party and the weather that day? I think it affected some of us…)
Lets assume…
Query…
Well, in order to handle this we may use the field of probability theory and express the information in those sentences probabilistically.
We can put this together into a “probabilistic network” that has a node for each proposition and directed arcs signifying
Causal/probabilistic relationships. Although we express information just between several variables, we can argue that the information we have is sufficient to capture all the probabilistic relationship that the “simple” domain has. In other words, assuming that the individuals in the new story are affected only by the weather, we don’t need to specify P(a|b) because, when we know the
Weather a and b are conditionally independent. The local collection of function represent a joint probability on all the variables
That can be obtained by a product of all these functions. W
P(W)
P(A|W)
P(C|W)
P(B|W) B C A
P(W,A,C,B) = P(B|W) · P(C|W) · P(A|W) · P(W)
P(A,C,B|W=bad) = 0.9 · 0.1 · 0.5
Chavurah 5/8/2010

4. Mixed Probabilistic and Deterministic networksPN CN
P(C|W)
P(B|W)
P(W)
P(A|W) W B A C
P(C|W)
P(B|W)
P(W)
P(A|W) W B A C
A→B
C→A B A C
A→B
C→A B A C
Query:
Is it likely that Chris goes to the party if Becky does not but the weather is bad?
Semantics?
Algorithms?
Chavurah 5/8/2010

5. The problem All men are mortal T All penguins are birds …
Socrates is a man
Men are kind p1
Birds fly p2
T looks like a penguin
Turn key –> car starts
P_n
True
propositions
Uncertain
propositions
Q: Does T fly?
P(Q)?
Logic?....but how we handle exceptions
Probability: astronomical

14. Alpha and beta are events

17. Burglary is independent of Earthquake

18. Earthquake is independent of burglary

35. Bayesian Networks: Representation
P(D|C,B)
P(B|S)
P(S)
P(X|C,S)
P(C|S)
Smoking
lung Cancer
Bronchitis
CPD:
C B D=0 D=1
X-ray
Dyspnoea
P(S, C, B, X, D)
= P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)
Conditional Independencies
Efficient Representation

36. Chapter 14 , Russel and Norvig Section 1 – 2
Bayesian networks
Chapter 14 , Russel and Norvig
Section 1 – 2

37. Outline
Syntax
Semantics

38. Example
Topology of network encodes conditional independence assertions:
Weather is independent of the other variables
Toothache and Catch are conditionally independent given Cavity

39. Example
I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar?
Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls
Network topology reflects "causal" knowledge:
A burglar can set the alarm off
An earthquake can set the alarm off
The alarm can cause Mary to call
The alarm can cause John to call

40. Example contd.

41. Compactness
A CPT for Boolean Xi with k Boolean parents has 2k rows for the combinations of parent values
Each row requires one number p for Xi = true (the number for Xi = false is just 1-p)
If each variable has no more than k parents, the complete network requires O(n · 2k) numbers
I.e., grows linearly with n, vs. O(2n) for the full joint distribution
For burglary net, = 10 numbers (vs = 31)

42. Semantics
The full joint distribution is defined as the product of the local conditional distributions:
P (X1, … ,Xn) = πi = 1 P (Xi | Parents(Xi))
e.g., P(j  m  a  b  e)
= P (j | a) P (m | a) P (a | b, e) P (b) P (e) n

45. Constructing Bayesian networks
1. Choose an ordering of variables X1, … ,Xn
2. For i = 1 to n
add Xi to the network
select parents from X1, … ,Xi-1 such that
P (Xi | Parents(Xi)) = P (Xi | X1, ... Xi-1)
This choice of parents guarantees:
(chain rule)
(by construction)

46. Example
Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?

47. Example Suppose we choose the ordering M, J, A, B, E No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)?

48. Example Suppose we choose the ordering M, J, A, B, E No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)?
P(B | A, J, M) = P(B)?

49. Example Suppose we choose the ordering M, J, A, B, E No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)? Yes
P(B | A, J, M) = P(B)? No
P(E | B, A ,J, M) = P(E | A)?
P(E | B, A, J, M) = P(E | A, B)?

50. Example Suppose we choose the ordering M, J, A, B, E No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)? Yes
P(B | A, J, M) = P(B)? No
P(E | B, A ,J, M) = P(E | A)? No
P(E | B, A, J, M) = P(E | A, B)? Yes

51. Example contd.
Deciding conditional independence is hard in noncausal directions
(Causal models and conditional independence seem hardwired for humans!)
Network is less compact: = 13 numbers needed