Bayes’ Theorem Suppose we have estimated prior probabilities for events we are concerned with, and then obtain new information. We would like to a sound method to computed revised or posterior probabilities. Bayes’ theorem gives us a way to do this.

Probability Revision using Bayes’ Theorem Prior Probabilities New Information Application of Bayes’ Theorem Posterior Probabilities

Classical Probability 3T Expert Systems Probability is a quantitative way of dealing with uncertainty. Classical probability is also called a priory probability because it deals with ideal games or systems. The fundamental formula of classical probability is defined as the probability:  W is the number of wins  N is the number of equally possible events

Classical Probability 4T Expert Systems Theory of probability:  A certain event is assigned probability 1  An impossible event is assigned probability 0

Experimental and Subjective Probabilities 5T Expert Systems In contrast to the a priori approach, experimental probability defines the probability of an event, P(E), as the limit of a frequency distribution: where f(E) is the frequency of outcomes of an event for N observed total outcomes. This type of probability is also called a posteriori probability, which means “after the event” Another term for a posteriori is the posterior probability.

Experimental and Subjective Probabilities 6T Expert Systems Subjective probability deals with events that are not reproducible and have no historical basis on which to extrapolate, such as drilling an oil well at a new site. A subjective probability by an expert is better than no estimate at all and is usually very accurate (or the expert won’t be an expert for long). A subjective probability is actually a belief or opinion expressed as a probability rather than a probability based on axioms or empirical measurement.

Compound Probabilities 7T Expert Systems The probabilities of compound events can be computed from their sample spaces. This can be expressed in terms of a Venn diagram for the sets: A = { 2, 4, 6 }B = { 3, 6 } The compound probability of rolling an even number and a number divisible by three is:

Conditional Probabilities 8T Expert Systems Events that are not mutually exclusive influence one another. Knowing that one event has occurred may cause us to revise the probability that another event will occur. The probability of an event A, given that event B occurred, is called a conditional probability and is indicated by P (A|B). The conditional probability is defined as:

Conditional Probability “Chance” of an event given that something is true Notation: – –Probability of event a, given b is true Applications: –Diagnosis of medical conditions (Sensitivity/Specificity) –Data Analysis and model comparison –Markov Processes

Conditional Probability Example Diagnosis using a clinical test Sample Space = all patients tested –Event A: Subject has disease –Event B: Test is positive Interpret: –Probability patient has disease and positive test (correct!) –Probability patient has disease BUT negative test (false negative) –Probability patient has no disease BUT positive test (false positive) –Probability patient has disease given a positive test –Probability patient has disease given a negative test

Conditional Probability Example If only data we have is B or not B, what can we say about A being true? –Not as simple as positive = disease, negative = healthy –Test is not Infallible! Probability depends on union of A and B Must Examine independence –Does p(A) depend on p(B)? –Does p(B) depend on p(A)? –Events are dependant

Law of Total Probability & Bayes Rule Take events A i for I = 1 to k to be: –Mutually exclusive: for all i,j –Exhaustive: For any event B on S Bayes theorem follows

Return to Testing Example Bayes’ theorem allows inference on A, given the test result, using knowledge of the test’s accuracy and population qualities –p(B|A) is test’s sensitivity: TP/ (TP+FN) –p(B|A’) is test’s false positive rate: TP/ (TP+FN) –p(A) is occurrence of disease

Application of Bayes’ Theorem Consider a manufacturing firm that receives shipment of parts from two suppliers. Let A 1 denote the event that a part is received from supplier 1; A 2 is the event the part is received from supplier 2

We get 65 percent of our parts from supplier 1 and 35 percent from supplier 2. Thus: P(A 1 ) =.65 and P(A 2 ) =.35

Quality levels differ between suppliers Percentage Good Parts Percentage Bad Parts Supplier 1982 Supplier 2955 Let G denote that a part is good and B denote the event that a part is bad. Thus we have the following conditional probabilities: P(G | A 1 ) =.98 and P(B | A 2 ) =.02 P(G | A 2 ) =.95 and P(B | A 2 ) =.05

Tree Diagram for Two-Supplier Example Step 1 Supplier Step 2 Condition Experimental Outcome A1A1 A2A2 G B G B (A 1, G) (A 1, B) (A 2, G) (A 2, B)

Each of the experimental outcomes is the intersection of 2 events. For example, the probability of selecting a part from supplier 1 that is good is given by:

Probability Tree for Two-Supplier Example Step 1 Supplier Step 2 Condition Probability of Outcome P(A 1 ) P(A 2 ) P(G | A 1 ) P(B | A 2 )

A bad part broke one of our machines—so we’re through for the day. What is the probability the part came from suppler 1? We know from the law of conditional probability that: Observe from the probability tree that: (4.14) (4.15)

The probability of selecting a bad part is found by adding together the probability of selecting a bad part from supplier 1 and the probability of selecting bad part from supplier 2. That is: (4.16)

Bayes’ Theorem for 2 events By substituting equations (4.15) and (4.16) into (4.14), and writing a similar result for P(B | A 2 ), we obtain Bayes’ theorem for the 2 event case:

Do the Math

Bayes’ Theorem

Tabular Approach to Bayes’ Theorem— 2-Supplier Problem (1) Events A i (2) Prior Probabilities P(A i ) (3) Conditional Probabilities P(B | A 1 ) (4) Joint Probabilities P(A i ∩ B) (5) Posterior Probabilities P(A i | B) A1A /.0305 =.4262 A2A /.0305 = P(B)=

Using Excel to Compute Posterior Probabilities

Bayesian Network for Prediction Bayes nets may be used in any walk of life where modeling an uncertain reality is involved

Exercise 41, p A consulting firm submitted a bid for a large consulting contract. The firm’s management felt id had a change of landing the project. However, the agency to which the bid was submitted subsequently asked for additional information. Past experience indicates that that for 75% of successful bids and 40% of unsuccessful bids the agency asked for additional information. a.What is the prior probability of the bid being successful (that is, prior to the request for additional information). b.What is the conditional probability of a request for additional information given that the bid will be ultimately successful. c.Compute the posterior probability that the bid will be successful given a request for additional information.

Exercise 41, p. 187 Let S 1 denote the event of successfully obtaining the project. S 2 is the event of not obtaining the project. B is the event of being asked for additional information about a bid. a.P(S 1 ) =.5 b.P(B | S 1 ) =.75 c.Use Bayes’ theorem to compute the posterior probability that a request for information indicates a successful bid.