Introduction to Data Analysis

Presentation on theme: "Introduction to Data Analysis"— Presentation transcript:

Introduction to Data Analysis
Probability Confidence Intervals

Today’s lecture Some stuff on probability
Confidence intervals (A&F 5). Standard error (part 2) and efficiency. Confidence intervals for means. Confidence intervals for proportions.

Last week’s aide memoire
Population distribution We don’t know this, but we want to know about it. In particular we normally want to know the mean. Sample distribution We know this, and calculate the statistics such as the sample mean and the sample standard deviation from it. Sampling distribution This describes the variability in value of the sample means amongst all the possible samples of a certain size. We can work this out from information about the sample distribution and the fact that the sampling distribution is normal if sample size is large (ish).

Random Variables Flip a coin. Will it be heads or tails?
The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How many will be heads and how many will be tails? Over time, patterns emerge from seemingly random events. These allow us to make probability statements.

Heads or Tails? A coin toss is a random event [H or T] unpredictable on each toss but a stable pattern emerges of 50:50 after many repetitions. The French naturalist, Buffon ( ) tossed a coin 4040 times; resulting in 2048 heads for a relative frequency of /4040 = .5069

Heads or Tails? The English mathematician John Kerrich, while imprisoned by Germans in WWII, tossed a coin 5,000 times, with result 2534 heads . What is the Relative Frequency? 2,534 / 5,000 = .5068

Heads or tails? A computer simulation of 10,000 coin flips yields 5040 heads. What is the relative frequency of heads? 5040 / 10,000 = .5040

Each of the tests is the result of a sample of fair coin tosses.
Sample outcomes vary. Different samples produce different results. True, but the law of large numbers tells us that the greater the number of repetitions the closer the outcomes come to the true probability, here .5. A single event may be unpredictable but the relative frequency of these events is lawful over an infinite number of trials\repetitions.

Random Variables "X" denotes a random variable. It is the outcome of a sample of trials. “X,” some event, is unpredictable in the short run but lawful over the long run. This “Randomness” is not necessarily unpredictable. Over the long run X becomes probabilistically predictable. We can never observe the "real" probability, since the "true" probability is a concept based on an infinite number of repetitions/trials. It is an "idealized" version of events

To figure the odds of some event occurring you need 2 pieces of information:
A list of all the possibilities – all the possible outcomes (sample space) The number of ways to get the outcome of interest (relative to the number of possible outcomes).

Take a single Dice Roll Assuming an evenly-weighted 6-sided dice, what are the odds of rolling a 3? How do you know? 6 possible outcomes (equally likely) 1 way to get a 3 p(Roll=3) = 1 / 6

What are the chances of rolling numbers that add up to “4” when rolling two six-sided dice?
What do we need to know? All Possible Outcomes from rolling two dice Outcomes that would add up to 4

How Many Ways can the Two Dice Fall?
Let’s say the dice are different colors (helps us keep track. The White Dice could come out as: We know how to figure out probabilities here, but What about the other dice?

When the white die shows , there are six possible outcomes.
When the white die shows , there are six more possible outcomes. We then just do that for all six possible outcomes on the white die

Remember the Question: What is the probability of Rolling numbers that sum to 4?
What do we need to know? All Possible Outcomes from rolling two dice (36--Check Previous Slide) How many outcomes would add up to 4? Our Probability is 3/36 =

Probability = Frequency of Occurrence Total # outcomes
Frequency of occurrence = # of ways this one event could happen Total # outcomes = # ways all the possible events could happen Probability of a 7 is 6 ways out of 36 possibilities p=.166

--+----+----+----+----+----+----+----+----+----+----+
Frequency of Sum of 2 Dice * p = .167 - p = .139 * * p = .139 F R E p =.111 * * p = .111 Q U E N p=.083 * * p=.083 C Y * p= p=.056 * 1 - * p= p= * SUM OF 2 DICE

Review of Set Notation Capital Letters sets of points
Lower case letters represent elements of the set For example: A = {a1, a2, a3}

Subsets Let S denote the full sample space (the set of all possible elements) For two sets A and B, if every element of A is also an element in B, we say that A is a subset of B S B A

Union The union of two arbitrary sets of points is the set of all points that are in at least one of the sets S A B

Intersection The intersection of two arbitrary sets of points is the set of all points that are in both of the sets

Mutual Exclusivity Two events are said to be disjoint or mutually exclusive if none of the elements in set A appear in set B.

Independence We will give a more rigorous definition later, but…
Two events are independent if the occurrence of A is unaffected by the occurrence or nonoccurrence of B. Example: You flip a coin—what is the probability of heads? You flip it 10 times, getting heads each time. What is the probability of getting heads again?

Axioms for Probabilities
The conventional rules for probabilities are named the Kolmogorov Axioms. They are: 1. 2. 3. If A1, A2, A3, … are pairwise mutually exclusive events in S, then:

Rules for Calculating Probabilities
Simple Additive rule for disjoint events a.k.a. the “or” rule S A B

Example: One community is 75% white (non-hispanic), 10% black (non-hispanic), and 15% hispanic. They choose their mayor at random to maximize equality. What is the probability that the next mayor will be non-white?

Rules for Calculating Probabilities
Simple Multiplicative rule for independent events a.k.a. the “and” rule

Example: Suppose in that same mythical community (75% white, 10% black, 15% Hispanic) there was an even division of males and females. What is the probability of a white male mayor?

Rules for Calculating Probabilities
The Complement Rule

Rules for Calculating Probabilities
Additive rule for events that are not mutually exclusive events

Rules for Calculating Probabilities
Multiplicative rule for conditional events

But wait… Now we know the rules for manipulating probabilities mathematically, but to get them, we need to calculate the sample space and the number of ways to get the outcome

Tools for Counting Sample Spaces
Listing The mn rule Permutations (Pnr) Combinations (Cnr)

Listing You are flipping 2 coins (this is one trial). List the possible outcomes: HH TH HT TT S = 4 You are flipping 3 coins: THH HTT HTH THT HHH HHT TTH TTT S = 8

Easy (with a small sample space) Disadvantages Hard to do with a large sample space

The mn rule Think of the coin example—there are 2 possible outcomes for each coin, and 2 coins. Thus, there are 2 * 2 = 4 possible outcomes Think of the dice example—there are 6 possible outcomes for each dice, and 2 dice. Thus, there are 6 * 6 = 36 possible outcomes

The mnp rule ? It works for more than 2 dice/coins or whatever too. Consider the 3 coin flips: 2 * 2 * 2 = 23 = 8 This can be thought of as successive applications of the mn rule. First, you get the combinations of 2 coin flips mn = 2 * 2 = 4 Then you get the combination of that sample space with the 3rd coin (mn)*p = 4 * 2 = 8

Complex Example Assume that leap years don’t exist and there are thus just 365 possible birthdays. We want to know the sample space for possible birthdays for 20 randomly drawn people. How do we get it? 36520 = 1.76*1051

Complex Example Assuming that each birthday is equally likely, what is the probability of everyone having a January 1 Birthday? What is the probability that everyone has the same birthday?

Complex Example What is the probability that everyone has a different birthday? For the first drawn person, there are 365 possible b-days because no one else has been drawn For the second drawn person, there are 364 possible b-days because only 1 birthday has been drawn; for the third person it’s 363; fourth is 362…

Permutations In some situations, we will be concerned with the order in which sequences occur. An ordered arrangement of r objects is called a permutation The number of possible permutations is denoted as Our proof is based on our extension of the mn rule in the last problem…

Permutations We are interested in filling r positions with n distinct objects—no repeated values (selecting 20 people with 365 possible birthdays)

Permutation Example Think of a chain lock with a combination that requires 4 digits (0-9) How many possible combinations are there? 4 6 2

Combinations In some situations, the ordering of the symbols in a set is unimportant—we only care what is included (not its position) There will be fewer combinations than permutations—permutation counts HHT, HTH, and THH as being separate; combinations don’t consider the order, just the contents.

Example A company selects five applicants for a “short list” for a job. They will actually interview just 2 candidates. How combinations of two applicants can be selected from the pool?

Example 2 A company receives 10 applications for a single position. Being short on time, but good at combinatorics, the boss considers drawing two applications at random for interviews. The boss wants to know the probability of getting exactly 1 of the two best applicants in his sample by random chance. The probability of choosing one of the two best is given by The probability of choosing one of the three worst is given by

Example 2 We can then use the mn rule to figure out the number of ways we can get both In the last example we computed the number of possible combinations to be Thus, the probability is 6 / 10 = .6

Conditional Probability
Under some circumstances the probability of an event depends on another event. An unconditional probability asks what the chances are of rain tomorrow (event A). A conditional probability says, “Given that rained today (event B), what are the chances of rain tomorrow? (event A)” P(A|B)

Computing Conditional Probabilities

Independence Two events are said to be independent if
Otherwise, the events are dependent

Bayes’ Rule Suppose we knew P(B|A) but wanted to know P(A|B)?

Example Suppose you have been tested positive for a disease; what is the probability that you actually have the disease? Suppose the probability of having the disease is The test is 95% accurate, and you tested positive. Do you have the disease? We know: The probability of anyone having the disease (.01) The probability of testing positive for the disease conditional on having the disease (.95) We want to know the probability of having the disease if you tested positive for it…

Bayes’ Rule

What? .161? Why so low? Out of 100 people who take this test, we expect only 1 would have the disease. However, 5 people would test positive even if they didn’t have the disease. Out of those 6 people, only 1 actually has the disease…

Political Application
In a certain population of voters, 40% are Republican and 60% are Democrats. It is reported that 30% of Republicans and 70% of Democrats support a particular issue. A randomly selected person is found to favor the issue—what is the conditional probability that they are a Democrat?

Work it out We want to know P(Dem | F_issue)

Standard errors continued…
Last week we managed to work out some info about the distribution of sample means. If we have lots of samples then: Mean of all the sample means = population mean. Shape of the distribution of sample means = normal. Standard error tells us how dispersed the distribution of sample means is. Put all these together, and we can finally work out how likely our sample mean is to be near the population mean.

Churchgoing example Let’s take a less contrived example than my car managing to break any speed limits. This week I’m interested in the average number of times a year people go to church (or synagogue, or whatever). I take a sample of 100 people and record the number of times they went to church last year. Some went a lot but most went infrequently. So, from that sample I get a sample mean and a sample standard deviation.

Church-going sample Sample mean = 8.5 Sample s.d. = 20

Standard error Since we know the s.d. and mean of the sample, we can calculate the standard error. Because we know the standard error and the fact that the sampling distribution is normal, we are able to calculate how ‘likely’ it is that a specific range around our estimate contains the population mean. We can calculate what’s called a confidence interval.

Confidence intervals (1)
A confidence interval for an estimate is a range of numbers within which the parameter is ‘likely’ to fall. Remember a parameter is something about the population, like the population mean. We can use the standard error (and the fact that the sampling distribution is normal) to produce such a range.

Confidence intervals (2)
k is chosen to determine what is meant by ‘likely’ to contain the actual value of the population mean. We want to pick a k that is meaningful to us. … and is not so large that the interval is useless. … but not so small that the interval is very unlikely to contain the true population value.

Confidence intervals (3)
To return to our example, we have: An estimate of the population mean (the mean number of days of church attendance) which is the sample mean (8 ½ days). A standard error (2 days) which allows us to put a range around our estimate.

How do we pick k? Imagine sampling repeatedly, and therefore getting lots of sample means. Given what we know about the sampling distribution (i.e. it’s normal if n is large enough), we know what proportion of these sample means will fall within a certain number of standard errors of the actual population mean.

Confidence coefficient
Call this proportion the confidence coefficient. If we picked .95 then we know that if we repeatedly sampled the population, that interval around each of the sample means would include the true value 95% of the time. The confidence coefficient is a number that is chosen by the researcher which is close to 1, like 0.95 or 0.99. Since the sampling distribution is normal, we know the values of k that correspond to the probability of any proportion. So we know that approximately 95% of confidence intervals that are 2 standard errors either side of the sample mean will include the population mean.

Back to church For our particular sample we calculate a 95% confidence interval (i.e. k=2). Imagine the population mean for churchgoing in Britain is actually 6 days a year. So our particular sample is off by 2 ½ days a year. The mean of all the possible sample means is equal to the population mean so the centre of the sampling distribution is 6. k SE

Sampling distribution (1)
Population mean = 6 Sample mean=8.5

More samples But that’s just our one sample, let’s imagine we took many samples, and then calculated 95% confidence intervals for all the sample means.

Sampling distribution (2)
Population mean = 6

Sampling distribution (3)
Of my 7 samples, all the confidence intervals around the sample mean enclosed the actual true population mean apart from one. If we repeated this lots of times, we would expect 95% of the confidence intervals to enclose the actual population mean. 95% because that’s the confidence coefficient that we picked. If we had picked a confidence coefficient of 0.99, then it would be 99% and the confidence intervals would be larger.

Confidence coefficients
To make things easier I’ve been rounding the numbers off, the exact figures for k at each confidence coefficient are slightly different. Confidence coefficient k 68% 1.00 95% 1.96 99% 2.58 99.9% 3.29

Smaller confidence intervals?
We could just be less certain. If I was willing to pick a 90% confidence interval then the range of the interval would be narrower as k would be lower. We would be wrong more often though… We could increase the sample size. The bigger the sample size, the lower the standard error and therefore the smaller the confidence interval for a given probability. This isn’t always practical of course…

Why 95%? Generally speaking in social science we pick 95% as an appropriate confidence coefficient. A 1 in 20 chance of being wrong is generally felt to be okay. Path dependency—Fisher integrated the normal PDF for the 95% level, which is REALLY hard to do without a computer. This isn’t always the case… Sometimes we want to be really sure we’ve enclosed the mean. Other times we want a narrower range and are willing to accept that we are wrong more often.

CIs for proportions Since calculating the standard error is similar for proportions, so are producing confidence intervals. Remember we need binary data coded a 0 and 1 (yes/no, men/women etc.) Remember the standard error for proportions depends on the n and the sample proportion. So the CI is as below:

Proportions example Let’s take a political opinion poll in the US, we’re interested in the population proportion voting Democrat (call this π). Sample is 1000 people. Sample proportion is 0.45 (or 45%).

Opinion polls A lot of opinion polls use a sample size of about 1000 people, and aim to estimate proportions that are between 30% and 50%. This is why you often see ‘margins of error’ that are 3% either side of an estimate quoted in newspapers; these are 95% confidence intervals. Note that all these confidence intervals ignore non-sampling error. What we call sampling bias, this could be due to non-response, badly worded questions and so forth.

Exercise If you were taking a political opinion poll, roughly how big a sample would you need to achieve a ‘margin of error’ (i.e. a 95% confidence interval) of: 2% either side of the estimate? 1% either side of the estimate? (Assume that the sample proportion of interest is around 40%).

Similar presentations