1. Uncertainty in Expert Systems
CPS 4801

2. Uncertainty
Uncertainty is the lack of exact knowledge that would enable us to reach a fully reliable solution.
Classical logic assumes perfect knowledge exists:
IF A is true
THEN B is true
Describing uncertainty:
If A is true, then B is true with probability P

3. Sources of uncertainty
Weak implications: Want to be able to capture associations and correlations, not just cause and effect.
Imprecise language:
How often is “sometimes”?
Can we quantify “often,” “sometimes,” “always?”
Unknown data: In real problems, data is often incomplete or missing.
Differing experts: Experts often disagree, or have different reasons for agreeing.
Solution: attach weight to each expert

5. Two approaches Bayesian reasoning Certainty factors
Bayesian rule (Bayes’ rule) by Thomas Bayes
Bayesian network (Bayes network)
Certainty factors

6. Probability Theory
P(success) + P(failure) = 1
The probability of an event is the proportion of cases in which the event occurs
Numerically ranges from zero to unity (an absolute certainty) (i.e. 0 to 1)

7. Example Flip a coin P(head) = ½ P(tail) = ? P(head) = ¼ P(tail) = ?
Throw a dice
P(getting a 6) = ?
P(not getting a 6) = ?
P(A) = p  P(¬A) = 1-p

8. Example P(head) = ½ P(head head head) = ?
Xi = result of i-th coin flip Xi = {head, tail}
P(X1 = X2 = X3 = X4) = ?
Until now, events are independent and mutually exclusive.
P(X,Y) = P(X)P(Y) (P(X,Y) is joint probability.)

9. Example
P( {X1 X2 X3 X4} contains >= 3 head ) = ?

10. Conditional Probability
Suppose events A and B are not mutually exclusive, but occur conditionally on the occurrence of the other
The probability that event A will occur if event B occurs is called the conditional probability
probability of A given B

11. Conditional Probability
The probability that both A and B occur is called the joint probability of A and B, written p(A ∩ B)

12. Conditional Probability
Similarly, the conditional probability that event B will occur if event A occurs can be written as:

13. Conditional Probability

14. The Bayesian Rule
The Bayesian rule (named after Thomas Bayes, an 18th-century British mathematician):

15. Applying Bayes’ rule A = disease, B = symptom
P(disease|symptom) = P(symptom|disease) * P(disease) / P(symptom)

16. Applying Bayes’ rule
A doctor knows that the disease meningitis causes the patient to have a stiff neck for 70% of the time.
The probability that a patient has meningitis is 1/50,000.
The probability that any patient has a stiff neck is 1%.
P(s|m) = 0.7
P(m) = 1/50000
P(s) = 0.01

17. Applying Bayes’ rule P(s|m) = 0.7 P(m) = 1/50000 P(s) = 0.01
P(m|s) = P(s|m) * P(m) / P(s)
= 0.7 * 1/50000 / 0.01 =
= around 1/714
Conclusion: Less than 1 in 700 patients with a stiff neck have meningitis.

18. Example: Coin Flip P(X1 = H) = ½ 1) X1 is H: P(X2 = H | X1 = H) = 0.9
2) X1 is T: P(X2 = T | X1 = T ) = 0.8
P(X2 = H) = ?

19. What we learned from the example?
If event A depends on exactly two mutually exclusive events, B and ¬B, we obtain:
P(¬X|Y) = 1 – P(X|Y)
P(X|¬Y) = 1 – P(X|Y)?

20. Conditional probability
If event A depends on exactly two mutually exclusive events, B and ¬B, we obtain:
Similarly, if event B depends on exactly two mutually exclusive events, A and ¬A, we obtain:

21. The Bayesian Rule
Substituting p(B) into the Bayesian rule yields:

22. Bayesian reasoning
Instead of A and B, consider H (a hypothesis) and E (evidence for that hypothesis).
Expert systems use the Bayesian rule to rank potentially true hypotheses based on evidences

23. Bayesian reasoning
If event E occurs, then the probability that event H will occur is p(H|E)
IF E (evidence) is true
THEN H (hypothesis) is true with probability p

24. Bayesian reasoning Example: Cancer and Test
P(C) = P(¬C) = 0.99
P(+|C) = P(-|C) = 0.1
P(+|¬C) = 0.2 P(-|¬C) = 0.8
P(C|+) = ?

25. Simple Bayes Network from Example

26. Bayesian reasoning
Expert identifies prior probabilities for hypotheses p(H) and p(¬H)
Expert identifies conditional probabilities for:
p(E|H): Observing evidence E if hypothesis H is true
p(E|¬H): Observing evidence E if hypothesis H is false

27. Bayesian reasoning Experts provide p(H), p(¬H), p(E|H), and p(E|¬H)
Users describe observed evidence E
Expert system calculates p(H|E) using Bayesian rule
p(H|E) is the posterior probability that hypothesis H occurs upon observing evidence E
What about multiple hypotheses and evidences?

28. Bayesian reasoning with multiple hypotheses
p(A)

29. Bayesian reasoning with multiple hypotheses
Expand the Bayesian rule to work with multiple hypotheses (H1...Hm)

30. Bayesian reasoning with multiple hypotheses and evidences
Expand the Bayesian rule to work with multiple hypotheses (H1...Hm) and evidences (E1...En)

31. Bayesian reasoning with multiple hypotheses and evidences
Expand the Bayesian rule to work with multiple hypotheses (H1...Hm) and evidences (E1...En)
Assuming conditional independence among evidences E1...En

32. Summary

33. Bayesian reasoning Example
Expert is given three conditionally independent evidences E1, E2, and E3
Expert creates three mutually exclusive and exhaustive hypotheses H1, H2, and H3
Expert provides prior probabilities p(H1), p(H2), p(H3)
Expert identifies conditional probabilities for observing each evidence Ei for all possible hypotheses Hk

34. Bayesian reasoning Example
Expert data:

35. user observes E3

36. Bayesian reasoning Example
user observes E3
expert system computes
posterior probabilities

37. user observes E3 E1

38. Bayesian reasoning Example
user observes E1
expert system computes
posterior probabilities

39. user observes E3 E1 E2

40. Bayesian reasoning Example
expert system computes
posterior probabilities
user observes E2

41. Bayesian reasoning Example
Initial expert-based ranking:
p(H1) = 0.40; p(H2) = 0.35; p(H3) = 0.25
Expert system ranking after observing E1, E2, E3:
p(H1) = 0.45; p(H2) = 0.0; p(H3) = 0.55

42. Problems with the Bayesian approach
Humans are not very good at estimating probability!
In particular, we tend to make different assumptions when calculating prior and conditional probabilities
Reliable and complete statistical information often not available.
Bayesian approach requires evidences to be conditionally independent – often not the case.
One solution: certainty factors