1. Probability and Independence 1 Krishna.V.Palem Kenneth and Audrey Kennedy Professor of Computing Department of Computer Science, Rice University

2. Probability 2 The classical definition of probability Pierre Simon Laplace The probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible when nothing leads us to expect that any one of these cases should occur more than any other, which renders them, for us, equally possible. Let us elaborate… Consider an event space Event 1 Event 2 Event 3 Event 4 If there is no reason believe that one event is more likely to occur than another If there is no reason believe that one event is more likely to occur than another Probability of favorable events Favorable events No. of favorable events Total no. of events

3. Generalizing Probability 3 Consider another event space Event 1 Event 2 Event 3 Event 4 Let us assume that each event is differently likely to occur Let us represent as a list of magnitude of “likeliness” EventLikeliness Event 1p1p1 Event 2p2p2 Event 3p3p3 Event 4p4p4

4. Generalizing Probability (Contd.) 4 EventLikeliness Event 1p1p1 Event 2p2p2 Event 3p3p3 Event 4p4p4 Let us look at these values. If these values have the following properties 1.All p i are in the range [0,1] 2.The sum of all p i = 1. then these values are called the probabilities of these events. For example, Event Probabilities Event 10.5 Event 20.3 Event 30.1 Event 40.1 Satisfies both the conditions This means that event 1 occurs 1 out of 4 times … etc.

5. Specification 5 Consider the event space Event 1 Event 2 Event 3 Event 4 Event Probabilities Event 1p1p1 Event 2p2p2 Event 3p3p3 Event 4p4p4 Complete specification of the behavior of the experiment Both these together

6. 6 Now we have this relationship Event Event 1 Event 2 Event 3 Event 4 Probabilities p1p1 p2p2 p3p3 p4p4 This can be defined concisely as a function whose independent variable represents the event The dependent variable is the value of the probability Let the variable ‘x’ represent the event xEvent 1Event 1 2Event 2 3Event 3 4Event 4 Probability (Event i) Here ‘x’ is called a random variable. Where i={1,2,3,4}

7. Analysis of the event space of a coin toss Can you list the event space of an experiment where an unbiased coin is being tossed ? To complete the specification we need the probabilities of these events. 7 Event H Event T Event Probabilities Event H½ Event T½

8. The method of union of events Joint Event space of two coins Event HH Event TT Event HT Event TH EventProbability Event HH1/4 Event HT1/4 Event TH1/4 Event TT1/4 Consider the outcome Heads in both the coins “OR” Tails in both the coins From the previous definition, Probability of the outcome = 2/4 = 1/2 Another way of representing the same probability is Probability of (Heads in both the coins “OR” Tails in both the coins) = Probability (Heads in both the coins) + Probability(Tails in both the coins) Why do you think that the “OR” of two events is translated to a + for their probabilities ? 8

9. Mutually Exclusive events and outcomes Joint Event space of two coins Event HH Event TT Event HT Event TH Two events are said to be mutually exclusive if (i)Given one event as the outcome of the experiment it is assumed that all other outcomes are discarded. For example, if Event HH is the outcome of the experiment then Event TT “cannot” the outcome of the experiment Let us consider another example, Joint Event space of two coins First coin is H Second coin is T First coin is T Second coin is H The event space of the two coins can also be represented like this because it covers all possible events But these two events are not mutually exclusive because if the event that the first coin is H occurs then the event that the second coin is H can also occur Hence these events are “not” mutually exclusive 9

10. The method of union of events Joint Event space of two coins Event HH Event TT Event HT Event TH The method of union of events states that “ If two events are mutually exclusive then the probability that either of them will occur is the sum of probabilities of the two events” 10 Let us play the experiment ∞ times Outcomes HH HT TH TT HH HH TT HH HT TH HT TH TH …. ∞ Relative Frequency = (HH, HH, HH) + (TT,TT,TT,….) All Outcomes 1/4 + = 1/2 For infinite trials

11. The method of union of events 11 Recall Consider an event space Event 1 Event 2 Event 3 Event 4 Favorable events Probability of favorable events No. of favorable events Total no. of events Event Probabilities Event 1p1p1 Event 2p2p2 Event 3p3p3 Event 4p4p4 ….Etc. Given all the above events are mutually exclusive Probability of favorable events ….. Etc.

12. The method of union of events Let us consider the case of two dice Joint Event Space of Die 1 & 2 Event 11 Event 12 Event 21 Event 23 ….. Etc. Using method of union of events, calculate the following (i)Probability that the first DIE and the second die rolled a ‘3’ OR both the DICE rolled a ‘6’ A) P(Event 33 OR Event 66) = 1/36 + 1/36 = 1/18 (ii) Probability that 3 and 4 were rolled be either one of them A) P(Event 34 OR Event 43) = 1/36 + 1/36 = 1/18 Are these events mutually exclusive ? 12

13. Mini-exercise 1 – Snakes and Ladders(S&L) Calculate the probability of landing in square 6 OR square 8 in the following scaled-down version of snakes and ladders (no snake or ladder)? 13 789 654 123 We will start at square 1 Roll the die 50 times and record the squares that you land in Calculate the chance of landing in square 6 OR square 8 Note that favorable events could be derived both from landing on square 6 OR on square 8 Instructions for Mini-Exercise 1

14. The method of intersection of events Joint Event space of two coins Event HH Event TT Event HT Event TH Event Space of coin 1 Event H Event T Event Space of coin 2 Event H Event T EventP H2/3 T1/3 EventP H½ T½ Consider this joint experiment again What is the probability of the outcome of the experiment having HEAD in the first die and TAIL in the second die ? All the events are not equally likely Cannot use the earlier definition of Favorable events Total number of events 14

15. The method of intersection of events Joint Event space of two coins Event HH Event TT Event HT Event TH Event Space of coin 1 Event H Event T Event Space of coin 2 Event H Event T EventP H2/3 T1/3 EventP H½ T½ Consider this joint experiment again What is the probability of the outcome of the experiment having HEAD in the first coin and TAIL in the second coin ? = Probability of first coin having HEAD * Probability of second coin having TAIL = ½ * 1/3 = 1/6 H in coin 1 = ½ of the time T in coin 2 = 1/3 of this = ½ * 1/3 = 1/6 of the time 15

16. The method of intersection of events Joint Event space of two coins Event HH Event TT Event HT Event TH Event Space of coin 1 Event H Event T Event Space of coin 2 Event H Event T EventP H2/3 T1/3 EventP H½ T½ The method of intersection of events says “The probability of a joint event is equal to the product of probability of elementary events” 16

17. The method of intersection of events Let us consider the case of two dice Using method of intersection of events, Calculate the following (i)Probability that the first die rolled a ‘1’ and the second die rolled a ‘4’ A)P(Event 14) = P(Event 1) * P(Event 4) = 1/12 * 1/12 = 1/144 (ii) Probability that the first die rolled a ‘6’ and the second die rolled a ‘6’ A)P(Event 66) = P(Event 6) * P(Event 6) = 1/3 * 1/12 = 1/36 Outcom e Die 1Die 2 11/121/3 21/121/3 31/12 4 51/31/12 61/31/12 Table of probabilities of the outcomes 17

18. Mini-exercise 2 – Snakes and Ladders(S&L) Calculate the probability of landing in square 2 AND square 8 in the following scaled-down version of snakes and ladders (no snake or ladder)? 18 789 654 123 We will start at square 1 Roll the die for 50 times and record the squares that you land in Calculate the chance of landing in square 6 AND 8 in one cycle In this case, favorable events requires that both events occur A cycle is completed when you go from square 9 to square 1 Instructions for Mini-Exercise 2

19. Building a model An important component of this course (including exercises and projects) would be building models, and these games are an example. An example of a model would be 19 4 5 6 1/6

20. 20 Solution: Probability of reaching 6 via 5 = 1/6 Probability of reaching 6 via 4 = 1/6 Total probability of reaching 6 from 4 OR 5 = 1/6 + 1/6 = 1/3 4 5 6 1/6

21. 21 4 5 6 1/6 You are at square 4 and want to compute the probability of reaching square 6 This can happen in two ways (i) You roll a 1 and reach 5 and then roll a 1 again (ii) You roll a 2 and reach 6 As we know the probabilities of the rolls we can calculate the probability of reaching 6 from 4 Solution: Probability of reaching 6 via 5 = (1/6 * 1/6 ) Probability of reaching 6 directly = 1/6 Total probability of reaching 6 from 4 = 7/36

22. Mini-Exercise 3 Can you build a model for the snakes and ladder game till square 6 ? In this game, if we cross square 6 with a die roll we stay at 6 That means, if you are at square 4 and you roll a 5, then you still stay at 6 22 654 123 Remember the elements of a model A node or a circle that indicates a particular square in the game An arrow or an edge that indicates the possibility of reaching another square from the current square A label on each arrow that indicates the probability of that transition taking place

23. 23 4 5 6 3 2 1 1/6 3/6 1/6 2/6 1/6

24. 24 This is a partial model of the scaled down snakes and ladder game 4 5 6 5/6 1/6 6/6 3 2 1 1/6 3/6 1/6 4/6 1/6 2/6 1/6

25. Mini-Exercise - 4 Using the model of the snakes and ladders, calculate the following values The probability of reaching square 4 from square 1 The probability of reaching square 5 OR square 6 from square 2 The probability of reaching square 3 AND square 5 from square 1 25

26. 26 654 123 4 5 6 5/6 1/6 6/6 3 2 1 1/6 3/6 1/6 4/6 1/6 2/6 1/6 Let us start with the transition model we built Hint: New edge(s) might be added as an extension

27. Mini-Exercise - 5 Using the new model of the snakes and ladders with the snake, calculate the following values The probability of reaching square 4 from square 1 The probability of reaching square 5 OR square 6 from square 2 The probability of reaching square 3 AND square 5 from square 1 27

28. Question ? 28 4 5 6 1/6 3 2 1 Say that you are at square 4 at a point in the game You roll a die and you reach 6

29. What can we do to get that information ? 29 4 5 6 1/6 3 2 1 What if we place some sort of a token at each square we land ? Let us say that we land at square 2 And then we reached square 4 So we leave a token at square 2 Thus after completing the game we have the additional “information” to determine where we have been

30. More information 30 4 5 6 1/6 3 2 1 So basically if we play this game multiple times, then we need more tokens (probably with different colors) to keep track of where we were each time. Let us say that an all-knowing “Oracle” tells us that the previous square we were in was odd and not even. Do we still need tokens to keep track ? Will we need more tokens or less ?

31. Answers 31 4 5 6 1/6 3 2 1 We will analyze more precisely the relationship between two events in the next class.

32. Independence and Dependence Till now, we have been looking at events individually. But in practice, events are interrelated. Some events are dependent on some other event taking place Let us consider the following example Let us play a small game. Bob is rolling a die. He asks Alice to guess what he rolled. 1.What is the probability that Bob is correct ? 2.Let us say that the Oracle told Danny that the outcome of the die is an even number or an odd number. Then what is the probability that Danny is correct ? 32

33. There is an underlying mathematical concept that can be explicitly stated to calculate the answer to the question Consider an experiment. The outcome of the experiment can be specified in terms of two different event spaces Event 1 Event 2 Event 3 … Event Even Event Odd Knowledge of the outcome of the roll of a die in terms of whether it is an even number or an odd number allows us to predict the actual outcome more precisely p p1p1 p2p2 p3p3 … p P even P odd 33 Let us calculate how much it improves our quality of our guess

34. There is an underlying mathematical concept that can be explicitly stated to calculate the answer to the question Consider an experiment. The outcome of the experiment can be specified in terms of two different event spaces Event 1 Event 2 Event 3 … Event Even Event Odd Knowledge of the outcome of the roll of a die in terms of whether it is an even number or an odd number allows us to predict the actual outcome more precisely p p1p1 p2p2 p3p3 … p P even P odd 34 Let us calculate how much it improves our quality of our guess