OKAN UNIVERSITY FACULTY OF ENGINEERING AND ARCHITECTURE Yrd. Doç. Dr. Didem Kivanc Tureli 14/10/2011Lecture 3 MATH 265 Probability and Random Processes 03 Conditional Probability and Independence Fall 2011

Conditional Probability Question Question : I throw a die once. I get a three. I am going to throw the die a second time. What is the probability that the sum of the two throws is 8? Experiment: Throw 2 dice Sample Space: – S = {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6), (2,1), (2,2), (2,3), (2,4), (2,5), (2,6), (3,1), (3,2), (3,3), (3,4), (3,5), (3,6), (4,1), (4,2), (4,3), (4,4), (4,5), (4,6), (5,1), (5,2), (5,3), (5,4), (5,5), (5,6), (6,1), (6,2), (6,3), (6,4), (6,5), (6,6)} 4/10/2011Lecture 32

Answer Define events E and F E = Getting a sum of 8 = {(1,7), (2,6), (3,5), (4,4), (5,3), (6,2), (7,1)} F = Getting a 3 on the first throw ={(3,1), (3,2), (3,3), (3,4), (3,5), (3,6)} Event of getting a sum of 8 when the result of the first throw is 3: EF={(3,5)} Probability of getting a sum of 8 when the result of the first throw is 3 = 4/10/2011Lecture 33

Graphical Interpretation of Conditional Probability You already know you are inside set E. What is the probability that you are inside set EF? 4/10/2011Lecture 34 S F E

Definition If P(F) > 0 then 4/10/2011Lecture 35 “Probability that event E will occur given that event F has occurred” “Probability of event E given event F” “Probability of E given F”

Examples A coin is flipped twice. If we assume that all four points in the sample space S = {(H,H), (H,T), (T,H), (T,T)}, are equally likely, what is the conditional probability that both flips result in heads, given that the first flip does? 4/10/2011Lecture 36

Examples 4/10/2011Lecture 37 E = Event that both flips result in heads = {(H, H)} F = Event that first flip results in heads = {(H, H), (H, T)} Notice that the intersection of sets E and F gives set E, that is, EF = E. Therefore, P(EF)=P(E) Conditional probability that both flips result in heads, given that the first flip does is P(E|F)

Examples You are a contestant on a quiz show. On this show, an urn contains 10 white, 5 yellow and 10 black marbles. A marble will be chosen at random from the urn. If you guess the color of the marble correctly, then you will win 1 million TL. The marble will be chosen from the bucket behind a curtain, in a darkened room. You have a friend in the production team who wants to help you to win the money. But he can’t see very well in the dark. All he can see is that the ball that came out was light colored, either white or yellow but definitely not black. Should you guess white or yellow? What is the probability that the ball chosen is yellow? 4/10/2011Lecture 38

Examples 4/10/2011Lecture 39 W2 W 10 W3 W6 W5 W1 W5 W3 W2 W4 W2 W5W6W7 W8W9 W3 W1 W 10 W1 W4 W7W8W9

Examples 4/10/2011Lecture 310 E = Event that chosen ball is not black – E = {W1, W2, W3, W4, W5, W6, W7, W8, W9, W10, Y1, Y2, Y3, Y4, Y5} F = Event that chosen ball is yellow = {Y1, Y2, Y3, Y4, Y5} As with the previous example, EF = F. Therefore, P(EF)=P(E) The conditional probability that the chosen ball is yellow given that the chosen ball is not black P(F|E)

Examples In the card game bridge, the 52 cards are dealt out equally to 4 players – called East, West, North and South. If North and South have a total of 8 spades among them, what is the probability that East has 3 of the remaining 5 spades? 4/10/2011Lecture 311

There are 52 cards, 13 of which are spades. All the cards are divided into 4 equal piles of 13 cards each, named north, south, east and west. Let (n, s, e, w) be the event that north has n spades, south has s spades, east has e spades and west has w spades. Then n + s + e + w = 13. From the previous lecture, there are possible values for (n, s, e, w) which fit this condition. 4/10/2011Lecture 312

Let F be the event that n + s = 8, therefore e + w = 5. Again from the previous lecture, the number of values for (n, s, e, w) in the event F is the number of quadruplets (n, s, e, w) which satisfy both the above equations, that is, 4/10/2011Lecture 313

We can also find this result by realizing that we need to choose – numbers (e, w) from {(0,5), (1,4), (2,3), (3,2), (4,1), (5,0)}, – numbers (n, s) from {(0,8), (1,7), (2,6), (3,5), (4,4), (5,3), (6,2), (7,1), (8,0)}, So the set F is F ={(0,8,0,5), (0,8,1,4), (0,8,2,3), (0,8,3,2), (0,8,4,1), (0,8,5,0), (1,7,0,5), (1,7,1,4), (1,7,2,3), (1,7,3,2), (1,7,4,1), (1,7,5,0), (2,6,0,5), (2,6,1,4), (2,6,2,3), (2,6,3,2), (2,6,4,1), (2,6,5,0), …. (8,0,0,5), (8,0,1,4), (8,0,2,3), (8,0,3,2), (8,0,4,1), (8,0,5,0)} 4/10/2011Lecture 314

Let E be the event that e ≥ 3. E is a very large set, so let us list only the elements of the event EF : EF ={(0,8,3,2), (0,8,4,1), (0,8,5,0), (1,7,3,2), (1,7,4,1), (1,7,5,0), (2,6,3,2), (2,6,4,1), (2,6,5,0), …. (8,0,3,2), (8,0,4,1), (8,0,5,0)} The number of such elements is 4/10/2011Lecture 315

Finally then the probability of east having 3 or more spades when it is given that north and south have 8 spades between them is: 4/10/2011Lecture 316

Examples Celine is undecided as to whether to take a French course or a chemistry course. She estimates that her probability of receiving an A grade would be ½ in a French course, and 2/3 in a chemistry course. If Celine decides to base her decision on the flip of a fair coin, what is the probability that she gets an A in chemistry? 4/10/2011Lecture 317

Examples The sample space for this example is S = {(French, A), (French, no A), (Chemistry, A), (Chemistry, no A)} But the events do not have equal probability. In general, use the formula P(AB)=P(A|B)P(B) as below: 4/10/2011Lecture 318

Find the probability that Celine gets an A There are two ways that Celine can get an A: either she takes French and gets an A, or she takes Chemistry and gets an A: 4/10/2011Lecture 319 It is also true that: In general the following holds: Therefore:

Multiplication Rule The probability that events E1 and E2 and E3 and … and En occur is equal to the product of – The probability that event E1 occurs – The probability that event E2 occurs given that event E1 occurs – The probability that event E3 occurs given that event E1 occurs and event E2 occurs, – … – The probability that event En occurs given that event E1 occurs and event E2 occurs, …, and event En occurs. 4/10/2011Lecture 320

Example An ordinary deck of 52 playing cards is randomly divided into 4 piles of 13 cards each. Compute the probability that each pile has exactly 1 ace E1 = Event that ace of hearts is in any one of the piles.  E2 = Event that ace of spades and ace of hearts are in different piles  E2 = Event that ace of spades, ace of hearts and ace of clubs are in different piles  E4 = Event that ace of spades, ace of hearts, ace of clubs and ace of diamonds are in different piles P(E1) = 1 4/10/2011Lecture 321

4/10/2011Lecture 322 Pile 1 Pile 4 Pile 3 Pile 2 Ace of spades can go into any one of Piles 2,3,4. There are 13 cards in every one of those piles, so 12 Card Slots Left 13 Card Slots Left

4/10/2011Lecture 323 Pile 1 Pile 4 Pile 3 Pile 2 Ace of diamonds can go into any one of Piles 3,4. There are 13 cards in every one of those piles, so 12 Card Slots Left 13 Card Slots Left

4/10/2011Lecture 324 Pile 1 Pile 4 Pile 3 Pile 2 Ace of diamonds can go into any one of Piles 3,4. There are 13 cards in every one of those piles, so 12 Card Slots Left 13 Card Slots Left

4/10/2011Lecture 325 Now using the multiplication rule:

Bayes’ Formula (Version 1) 4/10/2011Lecture 326 S A B Your sample space is now just B.

Bayes’ Formula (Version 2) 4/10/2011Lecture 327 S F1F1 F2F2 F3F3 FnFn E Take the sample space S and divide it into n pieces.

Example An insurance company believes that people can be divided into two classes – those that are accident prone and those that are not. Their statistics show that an accident prone person will have an accident at some time within a fixed 1 year period with probability 0.4, whereas this probability decreases to 0.2 for a non accident prone person. If we assume that 30 percent of the population is accident prone, what is the probability that a new policyholder will have an accident within a year of purchasing a policy? 4/10/2011Lecture 328

4/10/2011Lecture 329 Event A = New policy holder is accident prone Event B = New policy holder has an accident within a year of purchasing the policy S B A AB

4/10/2011Lecture 330 Event A: Policy holder is Accident Prone P(A)=0.3P(A c )= 0.7 P(B|A)= 0.4P(B c |A)= 0.6 Event B|A: Accident prone policy holder has an accident Event A c : Policy holder is NOT Accident Prone Event B c |A: Accident prone policy holder DOES NOT have an accident Event B|A c : NON-accident prone policy holder has an accident Event B c | A c : NON-accident prone policy holder DOES NOT have an accident Universal Set: Everything may happen P(B|A c )= 0.2P(B c |A c )= 0.8

Example Suppose that a new policy holder has an accident within a year of purchasing a policy. What is the probability that he/she is accident prone? 4/10/2011Lecture 331 Event A = New policy holder is accident prone Event B = New policy holder has an accident within a year of purchasing the policy P(A|B)= ?

Example In a criminal investigation the inspector in charge is 60 percent convinced that Ahmet is guilty. Now a new piece of evidence shows that the criminal is left handed. Ahmet is also left handed. Given that about 5 percent of the population is left handed, what is the probability that Ahmet is guilty given the new evidence? 4/10/2011Lecture 332 G=Event that Ahmet is guilty L=Event that Ahmet is left handed P(G|L)=?

Independent Events The two events E and F are said to be independent if 4/10/2011Lecture 333 Two events E and F that are not independent are said to be dependent. The three events E, F and G are said to be independent if

Independent Events Proposition: If E and F are independent, then so are E and F c. 4/10/2011Lecture 334

Example A card is selected at random from an ordinary deck of 52 playing cards. If E is the event that the selected card is an ace and F is the event that it is a spade, then E and F are independent. This follows because P(EF)=1/62, whereas P(E)=4/52 and P(F)=13/52. 4/10/2011Lecture 335

Example S = {1, 2, 3, …, 9, 10, J, Q, K, 1 , 2 , 3 , …, 9 , 10 , J , Q , K , 1 , 2 , 3 , …, 9 , 10 , J , Q , K , 1 , 2 , 3 , …, 9 , 10 , J , Q , K  } E = {1, 1 , 1 , 1  } F = {1 , 2 , 3 , …, 9 , 10 , J , Q , K  } EF= {1 } 4/10/2011Lecture 336

Independent trials resulting in a success with probability p and a failure with probability 1 − p are performed. [You perform and infinite number of trials, just start doing trials and never stop. ] What is the probability that n successes occur before m failures? If we think of A and B as playing a game such that A gains 1 point when a success occurs and B gains 1 point when a failure occurs, then the desired probability is the probability that A would win if the game were to be continued in a position where A needed n and B needed m more points to win. 4/10/2011Lecture 337

Pascal’s Solution P n,m is the probability that n successes occur before m failures. 4/10/2011Lecture 338 The last period we had n – 1 successes m failures, and then we were successful again. The last period we had n successes m – 1 failures, and then we failed again. P n,0 =0 (why? Because m=0 means no failures, only successes occur. There is almost no chance that in all of your infinite number of trials, you never fail. ) P 0,m =1 (why? This is the probability that at least 0 successes occur before m failures. So you don’t need any successes at all, although if there is a success that is fine too. So anything fits this option.)

Fermat’s Solution If you solve Pascal’s equations, you will get the answer. But Fermat had a better way to think about it: He argued that the necessary and sufficient condition for n successes to occur before m failures is that there be at least n successes in the first n+m − 1 trials. The probability of exactly k successes in n+m − 1 trials is 4/10/2011Lecture 339 So the probability of n successes before m failures is