Decision Making Under Uncertainty CSE 495 Resources: –Russell and Norwick’s book

Some Examples 1. Suppose that you are in a TV show and you have already earned 1’ so far. Now, the presentator propose you a gamble: he will flip a coin if the coin comes up heads you will earn 3’ But if it comes up tails you will loose the 1’ What do you decide? 2. Considerations for sitting a new airport: Cost noise pollution Safety If there are two candidate sites, how to decide between them?

First Order Representation  p symptom(p,toothache)  disease(p, cavity)  p disease(p, cavity)  symptom(p,toothache) For the “dentist in you”, which one is true? None. A better rule could be something like:  p symptom(p,toothache)  disease(p, cavity)  disease(p, Gumdisease)  …

Problems with First Order Representation Laziness: it is too much work to list antecedents and consequents Theoretical Ignorance: not all known antecedents/consequents may be known Practical ignorance: even if we know all the rules, uncertainty in establishing antecedents/consequents may still occur Related to the knowledge-acquisition bottle neck & Fuzzy Logic

Handling Uncertain Knowledge We have only a degree of belief Probability theory can be used to deal with degree of belief  If the probability of an event is 1 does it mean that the event will occur? No. It means that we believe that will always occur In our example we may believe that with 0.8 probability the patient has cavities

Uncertainty and Rational Decisions In addition to probabilities there might be preferences Plan 1: Take the train from NYC to NC at 7:00AM (probability of missing a connection in WAS: 5%, waiting time in WAS: 4 hours) Plan 2: Take the train from NYC to NC at 10:00AM (probability of missing a connection in WAS: 35%, waiting time in WAS: 1 hour) Which would you choose? The point being that decisions are not made based only on the probability of events

Decision Theory Utility theory: represents and reasons with preferences Decision Theory = probability theory + utility theory Suppose that taken actions update probabilities of states/actions. Which actions should be taken? ? ? ? 1. Calculate probabilities of current state 2. Calculate probabilities of action 3. Select actions with the highest expected utility

Probability Distribution The events E 1, E 2, …, E k must meet the following conditions: One always occur No two can occur at the same time The probabilities p 1, …, p n are numbers associated with these events, such that 0  p i  1 and p 1 + … + p n = 1 A probability distribution assigns probabilities to events such that the two properties above holds

Expected Value In general, let Q be a quantity that has value v1 with probability p 1, …., v k with probability p k then the Expected value of Q is: p 1 * v 1 + p 2 * v 2 +…+ p k * v K

Selection of a Good Attribute: Information Gain Theory If the possible answers v i have probabilities p(v i ), then the information content of the actual answer is given by: I(p(v 1 ), p(v 2 ), …, p(v n )) = p(v 1 )I(v 1 ) + p(v 2 )I(v 2 ) +…+ p(v n )I(v n ) = p(v 1 )log 2 (1/p(v 1 )) + p(v 2 ) log 2 (1/p(v 2 )) +…+ p(v n ) log 2 (1/p(v n )) Examples:  Information content with the fair coin:  Information content with the totally unfair:  Information content with the very unfair: I(1/2,1/2) = 1 I(1,0) = 0 I(1/100,99/100) = 0.08

Uniform Distribution A probability distribution is uniform if there are k events each of which has probability 1/k Examples? Rolling a “fair” dice. The events being that the dice will comes up on each of the dice’s faces

Probability of Two Events Taken Place Two events are independent if the occurrence of one doesn’t affect the occurrence of the other one. If E1 and E2 are two independent events, then the probability that E1 and E2 occur is p(E1)*p(E2) If the probability of having a winning a lottery is.1, then the probability of winning the lottery two times in a row is.1*.1 =.01 If E1 and E2 are two independent events, then the probability that E1 or E2 occur is p(E1) + p(E2) – (p(E1)*p(E2))

Conditional Probability P(A  B) = P(B)P(A|B) = P(A)P(B|A) (product rule) P(A|B) = (P(A)P(B|A))/ P(B) (Bayes rule) Example: Suppose that the following is true: Meningitis cause stiff neck, 50% of the time Probability of patient having meningitis is 1/50000 Probability of patient having stiff neck is 1/120 P(M|S) = = P(S|M) = P(M) = P(S)

Axioms of Probability 1.For any event A, 0  P(A)  1 holds 2.If F can never occur then P(F) = 0 and if T always occurs then P(T) = 1 3.P(E1  E2) = p(E1) + p(E2) – p(E1  E2)

Axioms of Probability (2) Suppose there is a betting game between 2 persons for money. Suppose that person 1 has some degree of belief in an event A. “I bet $6 that it will occur. My degree of belief is 0.4” Suppose that person 2 bets for or against A consistent with the degree of belief of person 1. “I bet $4 that it will not occur” Theorem (Bruno de Finetti, 1931): If Person 1 sets of degrees violating the axioms of the probability, then there is a betting strategy for Person 2 that guarantees that Person 1 looses money