1. Probability

3. The Birthday Paradox

4. How many people would you need in a room to have at least two of them share the same birthday. The answer is less a paradox and more a surprise, and points up the trouble people have with probability – they think linearly when they should be thinking exponentially. So here’s the answer, then I’ll discuss why: Number of people to be 100% sure: 367*. (*If we account for leap years) Number of people to be 99.99997% sure: 100. Number of people to be 97% sure: 50. Number of people to be 50.7% sure: 23. Surprised? You should be.

5. The Power of Powers in Probability Only 23 people to be over 50% sure? And only 50 to be 97%? Come on! So how does that work. Let me start with another question with a surprising answer: in a coin toss, what are the odds of getting 10 heads in a row? You might reason that since the odds are 50% of getting one head then it must be 50%/10=5% and you'd be wrong. You’re wrong because you thought linearly – in probability you don’t divide – you multiply and that means exponents. So the answer is.5 10 = 0.00097% As long as you remember the power of powers then you can understand why probability works.

6. The Birthday Paradox – By The Numbers The mistake people make (and why they never believe you about the birthday paradox) is that they always think of the question in the wrong way. It’s NOT the chance of someone sharing YOUR birthday. It’s the chance of ANY TWO PEOPLE sharing A birthday. But let’s start with you.

7. The Birthday Paradox – By The Numbers The probability of someone sharing your birthday is 1/365 th or 0.27% - very small. The arithmetic is: 1-(364/365) n Where ‘n’ would be the number of people for a given probability of having one of them share. Thus to have 50% odds of someone sharing YOUR birthday would require 253 people: 1-(364/365) 253 = 0.50047 or 50.05% YOUR Even in a room of 365 people the odds are only 63% someone shares YOUR birthday, and you’d need 1,000 people to get a 93% chance. If the 58 people who are still in this course were all here, the odds would be about 15% that two of you share a birthday. [1-(364/365) 58 = 0.15009 or 15%]

8. The Birthday Paradox – By The Arithmetic Let’s start the arithmetic by still asking the wrong question: if there were 23 people in a room including you, what’s the chance one would share your birthdate? Easy answer: compare your birthday with the other 22 people. That’s 22 comparisons. Since you don’t care if two other people share, the arithmetic is easy: 1-(364/365) 22 or @ 6% Now ask the correct question: Do any two in the room share a birthday? That’s a little more complicated.

9. There are 253 pairs of people to compare: (23*22)/2=253. We divide by 2 to eliminate the duplicate pairs (you on her and her on you) and use 22 to eliminate non- pairs (you on you, etc). Duplicate Pairs. Person on PersonCountable Pairs. Number of Countable Pairs Among 23 people

10. The chance of any 2 people having the same birthday: 1/365 = 0.00274 = 0.274% And thus having different birthdays: 1-1/365 = 1 - 0.00274 = 0.99726 = 99.726% But that’s just for one pair of people and one day. So, to finish the calculation… Chance of Any 2 People Sharing Birthday

11. Chance of Any 2 People Out of 23 Sharing A Same Birthday

12. Some other factoids deriving from the birthday surprise: The sqrt(n) is roughly the number you need to have a 50% chance of a match with n items. Sqrt(365) is about 20. This comes into play in cryptography for the birthday attack. Example You only need @ 5 people picking letters from the alphabet [sqrt(26)=5.1] to have a 50% chance of a match. Moral Exponentiation (multiplying) rapidly decreases the chance of picking unique items (that is, it increases the chances of a match). Remember: exponents (powers) are non- intuitive and are the heart of probability! The Birthday Paradox

14. Bayesian statistics (named after Thomas Bayes) has been around since the 1700s and recently has seen a surge in interest because it offers a different perspective to doing statistics than the so called frequentist approach. In fact, Bayesians would say that the results of using their approach can be opposite to the results gained from a frequentist approach. But it is more intuitively difficult to understand than the frequentist approach normally used. Bayesian Versus Frequentist Statistics

15. Bayesian stats is based on Bayes' Law and is all about relating prior knowledge or evidence to current (or posteriori) knowledge or evidence about a situation. That is: That the probability of something is related to the probability of something else. This is called the prior condition. Once you know the probability of that “something else”, it changes the way you judge the probability of the something. This is called the current (or posteriori) condition. Bayesian: probability measures degree of belief. Frequentist: probability measures proportion of outcomes. Bayes’ Law (or Theorem, or Rule)

16. Another way to look at the two types of stats is to look at the way they treat uncertainty because statistics is all about quantifying uncertainty. can’t know’ uncertainty Frequentists use ‘can’t know’ uncertainty, which results from random processes: e.g. you can’t know which number will come up when you throw a die. But if you can quantify the system (the die – it has six faces) you can estimate the chances of something happening (each face has 1/6 th chance of coming up). You still can’t know which number will come up but you know the chance of it coming up. “Can’t Know” and “Don’t Know” Uncertainty

17. don’t know’ uncertainty Bayesians use ‘don’t know’ uncertainty. That is, they are happy to change prior probabilities if they receive subsequent information that might lead them to do so. That is, they believe the same things about throwing a die – until it is thrown, and then their beliefs can change if the die looks unfair. In fact, they call the terms in their equations a priori and a posteriori knowledge. Before we go there here’s an example: “Can’t Know” and “Don’t Know” Uncertainty

18. A horse wins 5 out of 12 races, and therefore its chance of winning its next race is 5/12 or 41.6%. For frequentists, this is where the analysis stops. They can’t know what’s behind the curtain. But 3 of its 5 wins are when its raining, so now 3/5 = 60%. Yet when its raining it wins 3 out of 4, so 3/4 = 75%. (Note that P of horse winning given that it is raining is not the same as the P of it raining when horse wins!) For Bayesians it depends on the “don’t know (yet) probabilities under the curtain. “Can’t Know” and “Don’t Know” Uncertainty RainingNot rainingTOTAL Horse #1 wins325 Horse #1 loses167 TOTAL4812 The don’t know/can’t know curtain.

19. If research question is about how often the horse wins, or even how often it wins in the rain, Bayesians would also want to account for: Wins overall Loses overall Wins in rains Wins not rain Loses in rain Loses in not rain Bayes is as much about how many times the horse loses as wins. Winning, Losing, Not Winning, Not Losing

20. Bayes’ Law Formula Where: P(A) is what you know about A at first (called the prior probability). P(A│B) is what you know about A after learning something about B (called the posteriori probability). P(B│A)/P(B) is the support B provides for making decisions about A. It usually involves something called the Bayes’ Factor (or ratio), expressed in terms such as: A:B

21. You visit Arizona and you want to go hiking tomorrow. In recent years, it has rained only 5 days each year. Unfortunately, the weatherperson has predicted rain for tomorrow., and… When it actually rains, the she correctly forecasts rain 90% of the time, but… When it doesn't rain, she incorrectly forecasts rain 10% of the time. What is the probability that it will rain tomorrow? Example of Bayes’ Law

22. Event A 1 : It will rain tomorrow. P( A 1 ) = 5/365 =0.0137 [It rains 5 days out of the year.] Event A 2 : It will not rain tomorrow. P( A 2 ) = 360/365 = 0.986 [It does not rain 360 days out of the year.] Event B. The weatherperson predicts rain, but… P( B | A 1 ) = 0.9 [When it rains, she is right 90% of the time.] P( B | A 2 ) = 0.1 [When it does not rain, she is wrong 10% of the time.] Example of Bayes’ Law

23. P( A 1 | B ) = P( A 1 ) P( B | A 1 ) _____________________________ P( A 1 ) P( B | A 1 ) + P( A 2 ) P( B | A 2 ) P( A 1 | B ) =[(0.014)(0.9) / [ (0.014)(0.9) + (0.986)(0.1)] P( A 1 | B ) =0.111 Plug in the numbers and calculate through: And the real chance of rain tomorrow? Once again counterintuitive - 11% and not the 90% the weatherperson (and frequentist stats) led you to believe. Example of Bayes’ Law

24. Here’s another puzzle, a famous one in conditional Bayesian probabilities, completely non-intuitive and with a very surprising answer. It’s a game show and you’re offered three doors. Behind one is a million dollars. Behind the other two are goats. You choose door #1 but DO NOT open it. The host of the show opens one of the other doors and there is a goat. He now offers you the chance to change your mind about your choice of door. The question is, should you change your choice and switch to the other unopened door? The Monty Hall Puzzle

25. Once again the answer is surprising – you should switch because now your odds have doubled that the money is behind the other door! How can that be? Most people calculate that with two doors left, the odds are 50/50 so why switch? But the real odds are now 33/66 that the other door (the one you didn’t choose) is the correct one. It works using Bayes’ Law. The Monty Hall Puzzle

26. Door #1 Odds 1/3 Door #2 Odds 1/3 Door #3 Odds 1/3 Cumulative Odds 2/3 Door #2 Odds 2/3 Cumulative Odds 2/3 Using Bayes’ Theorem Probabilities before door 3 is opened (prior condition): 1:1:1 Probabilities after door 3 is opened (current condition): 1:2:0 Therefore your probability of winning doubles if you switch.

27. The Monty Hall Puzzle - Conditionality Your choice of door is not conditional on Monty’s choice of door (you chose first without prior knowledge of his choice). But your decision to switch or not is now conditional on your current knowledge of his choice. Monty’s choice of door is conditional on your choice of door (he chose after, with the current knowledge of your choice). X Door #1 Odds 1/3 Door #2 Odds 1/3 Door #3 Odds 1/3

28. Choose a Door Door 1 Goat Door 2 Goat Door 3 Car Stick Change Goat Car This diagram clearly shows that if you change your selection, 2 times out of 3 you will win the car. If you don’t change your selection, you will only win the car 1 time in 3. You are therefore twice as likely to win if you change your selection If you choose this door and you …

29. Choose one red ace and two black aces from a deck. Lay them face down, making note of where the red ace is. Do not show the cards. Ask someone to point to a card. Now flip one of the remaining cards that has the black ace. Ask them to bet you whether their chosen card is the red one The probability is 66% that it is not, but they will think it is 50/50. Here’s another simulator where you can test Bayes’ Law: SIMULATOR: http://www.curiouser.co.uk/monty/montygame.htm Bayes’ Law Simulation (aka how to win money at cards – no, seriously)

30. Statistics and Probability

31. The formal study of the laws of chance is called probability theory. Quantifying uncertainty is an essential part of statistics and this requires an understanding of probability. There are three areas where uncertainty enters into statistics.

32. Statistics and Probability First Inferential statistics deals with samples that only represent a population and thus there will always be uncertainty about how closely your sample does represent the population. Second Sampling theory is based on the idea that sample selection is regulated by chance. Third You are dealing with a limited number of variables, data from those variables, and information from those data. Thus there are many places where uncertainty exists.

33. Some Definitions A random experiment is whatever it is you are doing to answer a research question – for example, surveying a sample of the population or throwing a pair of dice. Within a random experiment you will create a sample space comprised of a limited set of elementary outcomes. These in turn combine into an unlimited set of events.

34. Some Definitions Examples of sample space: Sample of a population. (E.G. The numbers 1 to 6 on each of the die in a pair of dice.) Examples of elementary outcomes: Results of your questions or observations. (E.G. The probability of each of the 36 combinations of pairs of numbers.) Examples of events: A particular response to a question. (E.G. A particular number from the dice – say a 7 or a 3.)

35. Number on the die. 123456 Chance of getting it. 1/6th or 16.6% or.166 1/6 th or 16.6% or.166 1/6 th or 16.6% or.166 1/6th or 16.6% or.166 Definitions Example: throwing one die SAMPLE SPACE ELEMENTARY OUTCOMES Probabilities are expressed as a decimal between 0 and 1. In this example the probability of any number would be expressed as P(Oi) = 0.166 or in words, the probability ( P ) of an outcome ( O ) for a number ( i ) is equal to.166 or about 16.6%. RANDOM EXPERIMENT

36. Expressing Probabilities The formal language used to express probabilities is defined by (1) the sample space with which you are working, and (2) the finite range of probabilties. It all takes place within a finite range between 0 and 1. 0.0  --------------------------0.5---------------------------  1.0 No chance 50% chanceCertainty (0%) (100%) The reasons for this are: All random experiments, by definition, end up with a result or they wouldn’t be experiments. The chance of a result has to be somewhere between 0 and 1. That result will be between one end of your sample space and the other….

37. Expressing Probabilities In the single die random experiment you know that the probability of getting a number is one sixth or 0.166, which in percentage terms is about 16.6%. That is written as: P(Oi) =.166 where Oi is a number between and including 1 to 6. Since there cannot be any other number, the sample space can be written as: P(Oi) ≥ 1 ≤ 6 : that is Oi is a number between and including 1 to 6 Since probability is as much about what’s not going to happen, then the P of a number not coming up is 1-the probability of it coming up, or 1- P(Oi).

38. Number of the die. 123456 Chance of getting it. P(Oi) 1/6th or 16.6% or.166 Chance of not getting it. 1-P(Oi) 5/6th or 83.4% or.830 All probabilities associated with throwing one die SAMPLE SPACE ELEMENTARY OUTCOMES Note that these also include the probability of not getting a specific elementary outcome, expressed as 1-P(O i )

39. Probabilities when using a pair of dice Principles are the same, but the table has to have dimensions of 6X6 or 36 cells each comprised of a pair of numbers in combination. The probability of any given combination is 1/36th or about 0.028 (1 divided by 36) or 2.8%. The odds of not getting a given combination are 1-0.028 or 0.972 or 97.2%. The following table illustrates all of the combinations of numbers you can get with a throw of a pair of dice, with the shaded columns representing the second die.

40. Shaded numbers are the second die There is 1 chance in 36 that any given combination will come up This is the sample space These are examples of elementary outcomes: 5+6=11 5+2=7 3+4=7 2+2=4

41. Expressing Probabilities – Two Dice In the two dice random experiment the probability of getting a number is 1/36th or about 0.028 (1 divided by 36) or 2.8%. That is written as: P(Oi) =.028 where Oi is a number between and including 2 to 12. Since there cannot be any other numbers, the sample space can be written as: P(Oi) ≥ 2 ≤ 12 : that is Oi is a number between and including 2 to 12. Since probability is as much about what’s not going to happen, then the P of a number not coming up is 1-the probability of it coming up, or 1- P(Oi).

42. Summary Rules of Probability First, you cannot have a negative probability. Second, every elementary outcome (e.g. pair of numbers) has the same probability of occurring. Third, the chance of an event not happening will be 1 minus the chance it will happen. Fourth, (the corollary of the third point), the sum of all the probabilities of a sample set has to equal one. These rules are usually written as the following probability expressions:

43. SOME PROBABILITY RULES The RuleThe ExpressionThe Meaning The Non-negative RuleP(Oi) ≥ 0 Probabilities cannot be less than zero. That is, they cannot be negative. The Equality RuleP(O 1 ) = P(O 2 ) =….. P(O n ) Each elementary outcome has an equal chance of occurring, as in throwing a die or pair of dice. The probability of events (actual numbers) may certainly differ. The Non-occurrence Rule (or NOT rule) P(NOT O i ) = 1- P(O i ) The probability of an elementary outcome not occurring is 1 minus the probability that it will occur. The Unity RuleP(O 1 ) + P(O 2 ) +…+ P(O n )=1 All the probabilities of all the elementary outcomes in an experiment must sum to 1 (unity is another word for 1 (or whole) in mathematics).

44. Events So far we have dealt only with the elementary outcomes (Oi) of an experiment – for example, all possible pairs of numbers. But you really want to know what’s the probability of getting a specific number with the dice – say a 7? You might also want to ask other questions, such as getting specific combinations, or adding conditions such as “or” and “and”, or even not getting a number at all. To do this you use events: An event is a set of elementary outcomes where the probability of the event will be the sum of the probabilities of its elementary outcomes.

45. But there are only 11 different numbers (2- 12), each with a different probability of occurring. There are 36 different pairs of numbers that could come up, each with the same probability of occurring. For example, there are 6 pairs on the diagonal each with 1/36 th chance of occurring, but they all add up to the number 7, which has 6*1/36 th or 1/6 th or 0.16 or 16% chance of occurring.

46. Probabilities of Different Events Occurring With Two Dice Event ( Ei ) Description Event’s Elementary Outcomes Probability of Event Occurring Throwing a 7 [1,6],[2,5],[3,4],[4,3],[5,2], [6,1] [.028]+[.028]+[.028]+[.028 ]+[.028]+ [.028]=0.167 or 16.7% Throwing a 3 [1,2],[2,1] [.028]+[.028]=.056 or 5.6% White die shows 1 [1,1],[1,2],[1,3],[1,4],[1,5],[1,6] [.028]+[.028]+[.028]+[.028 ]+[.028]+ [.028]=0.167 or 16.7% Black die shows 4 [1,4],[2,4],[3,4],[4,4],[5,4],[6,4] [.028]+[.028]+[.028]+[.028 ]+[.028]+ [.028]=0.167 or 16.7% Getting a 7, black high [1,6],[2,5],[3,4] [.028]+[.028]+[.028]=.083 or 8.3%

47. Logical Operators OR, AND, NOT Some other questions are: What is the probability that the black die will equal 1 OR the white die will equal 1? What is the probability that the black die will equal 1 AND the white die will equal 1? What is the probability that the black die will NOT equal 1? Using the dice combinations table we can answer the questions above.

48. (Bottom row (blue numbers plus the green number) = white die showing 1). (Right column (red numbers plus green number) = black die showing 1). (Green combination has been double counted so must be subtracted once). Probability of the black die OR the white die showing a 1.

49. Event table for OR condition Event Description Event’s Elementary Outcome Probability of Event Occurring White die shows 1 OR black die shows 1 [1,1],[1,2],[1,3],[1,4],[1, 5],[1,6] OR [6,1],[5,1],[4,1],[3,1],[2, 1],[1,1] [.028]+[.028]+[.028]+[.028 ]+[.028]+[.028] + [.028]+[.028]+[.028]+[.028 ]+[.028]+[.028]-[.028] = 0.31 or 31% Subtract one of the double counted 1,1s

50. Mutually Exclusive OR Statements Formally the OR function for this event would be written: P(E1 OR E2) = P(E1) + P(E2) – P(E1 AND E2) If there are no overlapping events (i.e. the double counting of 1,1) the term P(E1 AND E2) equals zero so that with such mutually exclusive events as they are called, the expression is simply: P(E1 OR E2) = P(E1) + P(E2) An example of this would be E1 = the dice add to 6, E2 = the dice add to 3 - there are no combinations of elementary outcomes that overlap for these two events.

51. Event Table for Mutually Exclusive OR condition Event Description Event’s Elementary Outcome Probability of Event Occurring E1 Two dice add to 6 [5,1],[4,2],[3,3],[4,2],[5, 1] [.028]+[.028]+[.028]+[.028 ]+[.028]=0.14 or 14.0% E2 Two dice add to 3 [2,1], [1,2] [.028]+[.028]=.056 or 5.6% No double counted numbers to be subtracted.

52. What is the probability of throwing a 1 AND a 1?

53. Event table for AND condition Event Description Event’s Elementary Outcome Probability of Event Occurring E 2 White die shows 1 AND black die shows 1 [1,1][.028]=.028 or 2.8% There is only one elementary outcome that can give a double one (or a double of any number between 2 and 6). Note that OR functions always produce larger probabilities than AND functions.

54. AND, the Special Multiplication Rule, and Joint Probability The AND function can be used to calculate the probability of a specific series of events occurring (such as 3,4,5 or 7,7,7). The expression for the AND function follows what is called the special multiplication rule, stated as: The probability of ‘n’ specific events occurring is the product of the probabilities of each event ‘n’. That is, the individual probabilities are multiplied to get the joint probability thus: P(E1 AND E2) = P(E1)P(E2)

55. Example of AND and Joint Probability What would be the probability of throwing three 7’s in a row? The probability of throwing one 7 (called event 1 or E1) is 0.167 or about 16.7%, and the probability of throwing three of them in a row is: P(E1 AND E1 AND E1) = P(E1)*P(E1)*P(E1) 0.167*0.167*0.167 = 0.0047 or about 0.47% POINT: The chance of a random event repeating itself decreases dramatically even in the case of individually fairly common events such as throwing a 7.

56. The NOT Logical Operator The NOT logical operator is a handy tool for when you want to know the probability of something not happening. Assume that E1 is that a double-1 will not be thrown. Now the expression becomes: P(E1) =1- P(NOT E1) Probability of E1 (a double 1) = 1/36 th Probability of NOT E1 = 1-1/36th =35/36ths or 0.972 (35 divided by 36) or about 97.2% And for not throwing three 7s in a row? 1-0.0047 or 99.53%.

57. When To Bet NOT The thing about the NOT operator is you can rarely lose with it, if someone is careless enough to take a NOT bet. Even with throwing a 7, you have an 83% chance of winning if you bet someone they will not throw one. Do not bet? Do bet NOT.

58. The Power of Powers in Probability What is the probability of guessing the correct answer to a multiple choice question with 5 possible responses? 1 in 5 or 0.2 or 20% (Hmmm. Not bad. Maybe I’ll try that.) Now, what is the probability of guessing them all correctly in a 40 question test? 0.0000000000000000000000000001099 Of not getting any? 0.00013292279957849187 Half correct? 0.00001666462278347505

59. LESSON FOR THE DAY Do not bet unless you can bet NOT (but not in multiple choice quizzes)

60. What is the probability that you will win Lotto 649 this week? 1 in 13,983,816 (or about 0.000000007%. What is the probability that someone will win Lotto 649 this week? 1 in 1.75 (or about 57%)