Chapter 221 What Is a Test of Significance?

Chapter 222 Thought Question 1 The defendant in a court case is either guilty or innocent. Which of these is assumed to be true when the case begins? The jury looks at the evidence presented and makes a decision about which of these two options appears more plausible. Depending on this decision, what are the two types of errors that could be made by the jury? Which is more serious?

Chapter 223 Thought Question 2 Suppose 60% (0.60) of the population are in favor of new tax legislation. A random sample of 265 people results in 175, or 0.66, who are in favor. From the Rule for Sample Proportions, we know the potential sample proportions in this situation follow an approximately normal distribution, with a mean of 0.60 and a standard deviation of Find the standardized score for the observed value of 0.66; then find the probability of observing a standardized score at least that large or larger.

Chapter 224 Thought Question 2: Bell-Shaped Curve of Sample Proportions (n=265) % mean = 0.60 S.D. = 0.03

Chapter 225 Thought Question 3 Suppose that in the previous question we do not know for sure that the proportion of the population who favor the new tax legislation is 60%. Instead, this is just the claim of a politician. From the data collected, we have discovered that if the claim is true, then the sample proportion observed falls at the percentile (about the 98th percentile) of possible sample proportions for that sample size. Should we believe the claim and conclude that we just observed strange data, or should we reject the claim? What if the result fell at the 85th percentile? At the 99.99th percentile?

Chapter 226 Thought Question 3: Bell-Shaped Curve of Sample Proportions (n=265) th 98th 99.99th

Chapter 227 Case Study Brown, C. S., (1994) “To spank or not to spank.” USA Weekend, April 22-24, pp Parental Discipline What are parents’ attitudes and practices on discipline?

Chapter 228 Case Study: Survey Parental Discipline u Nationwide random telephone survey of 1,250 adults. –474 respondents had children under 18 living at home –results on behavior based on the smaller sample u reported margin of error –3% for the full sample –5% for the smaller sample

Chapter 229 Case Study: Results Parental Discipline “The 1994 survey marks the first time a majority of parents reported not having physically disciplined their children in the previous year. Figures over the past six years show a steady decline in physical punishment, from a peak of 64 percent in 1988” –The 1994 sample proportion who did not spank or hit was 51% ! –Is this evidence that a majority of the population did not spank or hit?

Chapter 2210 The Five Steps of Hypothesis Testing u Determining the Two Hypotheses u Computing the Sampling Distribution u Collecting and Summarizing the Data (calculating the observed test statistic) u Determining How Unlikely the Test Statistic is if the Null Hypothesis is True (calculating the P-value) u Making a Decision/Conclusion (based on the P-value, is the result statistically significant?)

Chapter 2211 The Null Hypothesis: H 0 u population parameter equals some value u status quo u no relationship u no change u no difference in two groups u etc. u When performing a hypothesis test, we assume that the null hypothesis is true until we have sufficient evidence against it

Chapter 2212 The Alternative Hypothesis: H a u population parameter differs from some value u not status quo u relationship exists u a change occurred u two groups are different u etc.

Chapter 2213 The Hypotheses for Proportions u Null: H 0 : p = p 0 u One sided alternatives H a : p > p 0 H a : p < p 0 u Two sided alternative H a : p   p 0

Chapter 2214 Case Study: The Hypotheses u Null: The proportion of parents who physically disciplined their children in the previous year is the same as the proportion [p] of parents who did not physically discipline their children. [H 0 : p =.5] u Alt: A majority of parents did not physically discipline their children in the previous year. [H a : p >.5]

Chapter 2215 If numerous simple random samples of size n are taken, the sample proportions from the various samples will have an approximately normal distribution with mean equal to p (the population proportion) and standard deviation equal to Sampling Distribution for Proportions Since we assume the null hypothesis is true, we replace p with p 0 to complete the test.

Chapter 2216 Test Statistic for Proportions To determine if the observed proportion is unlikely to have occurred under the assumption that H 0 is true, we must first convert the observed value to a standardized score:

Chapter 2217 Case Study: Test Statistic Based on the sample: u n=474 (large, so proportions follow normal distribution) u no physical discipline: 51% – – standard error of : (where.50 is p 0 from the null hypothesis) ustandardized score (test statistic) z = ( ) / = 0.43

Chapter 2218 P-value u The P-value is the probability of observing data this extreme or more so in a sample of this size, assuming that the null hypothesis is true. u A small P-value indicates that the observed data (or relationship) is unlikely to have occurred if the null hypothesis were actually true –The P-value tends to be small when there is evidence in the data against the null hypothesis –The P-value is NOT the probability that the null hypothesis is true

Chapter 2219 P-value for Testing Proportions u H a : p > p 0 u When the alternative hypothesis includes a greater than “>” symbol, the P-value is the probability of getting a value as large or larger than the observed test statistic (z) value.  look up the percentile for the value of z in the standard normal table (Table B)  the P-value is 1 minus this probability

Chapter 2220 P-value for Testing Proportions u H a : p < p 0 u When the alternative hypothesis includes a less than “<” symbol, the P-value is the probability of getting a value as small or smaller than the observed test statistic (z) value.  look up the percentile for the value of z in the standard normal table (Table B)  the P-value is this probability

Chapter 2221 P-value for Testing Proportions u H a : p  p 0 u When the alternative hypothesis includes a not equal to “  ” symbol, the P-value is found as follows:  make the value of the observed test statistic (z) positive (absolute value)  look up the percentile for this positive value of z in the standard normal table (Table B)  find 1 minus this probability  double the answer to get the P-value

Chapter 2222 Alternative Method for P-value 1. Make the value of the observed test statistic (z) negative 2. Look up the percentile for this negative value of z in the standard normal table (Table B)  if the alternative hypothesis includes a greater than “>” or less than “<“ symbol, the P-value is this probability in step 2  if the alternative hypothesis includes a not equal to “  ” symbol, double this probability in step 2 to get the P-value Caution:Use this method only when H a has a “>” sign and Z is positive, or when H a has a “<“ sign and Z is negative, or when Ha has a “  ” sign.

Chapter Case Study: P-value P-value = From Table B, z=0.4 is the th percentile. z= z: H a : p >.50

Chapter 2224 * Decision * u If we think the P-value is too low to believe the observed test statistic is obtained by chance only, then we would reject chance (reject the null hypothesis) and conclude that a statistically significant relationship exists (accept the alternative hypothesis). u Otherwise, we fail to reject chance and do not reject the null hypothesis of no relationship (result not statistically significant).

Chapter 2225 Typical Cut-off for the P-value u Commonly, P-values less than 0.05 are considered to be small enough to reject chance (reject the null hypothesis). u Some researchers use 0.10 or 0.01 as the cut-off instead of u This “cut-off” value is typically referred to as the significance level  of the test

Chapter 2226 Case Study: Decision u Since the P-value (.3446) is not small, we cannot reject chance as the reason for the difference between the observed proportion (0.51) and the (null) hypothesized proportion (0.50). u We do not find the result to be statistically significant. u We fail to reject the null hypothesis. It is plausible that there was not a majority (over 50%) of parents who refrained from using physical discipline.

Chapter 2227 Decision Errors: Type I u If we decide there is a relationship in the population (reject null hypothesis) –This is an incorrect decision only if the null hypothesis is true. –The probability of this incorrect decision is equal to the cut-off (  ) for the P-value. u If the null hypothesis is true and the cut-off is 0.05 –There really is no relationship and the extremity of the test statistic is due to chance. –About 5% of all samples from this population will lead us to wrongly reject chance.

Chapter 2228 Decision Errors: Type II u If we decide not to reject chance and thus allow for the plausibility of the null hypothesis –This is an incorrect decision only if the alternative hypothesis is true. –The probability of this incorrect decision depends on v the magnitude of the true relationship, v the sample size, v the cut-off for the P-value.

Chapter 2229 Power of a Test u This is the probability that the sample we collect will lead us to reject the null hypothesis when the alternative hypothesis is true. u The power is larger for larger departures of the alternative hypothesis from the null hypothesis (magnitude of difference) u The power may be increased by increasing the sample size.

Chapter 2230 Case Study: Decision Error? u Decision: fail to reject H 0 u If in the population there truly was a majority of parents who did not physically discipline their children, then we have committed a Type II error. u Could we have committed a Type I error with the decision that we made? [No! Why?]

Chapter 2231 Key Concepts (1 st half of Ch. 22) u Decisions are often made on the basis of incomplete information. u Five Steps of Hypothesis Testing u P-values and Statistical Significance u Decision Errors u Power of a Test

Chapter 2232 Inference for Population Means Hypothesis Testing The remainder of this chapter discusses the situation when interest is completing hypothesis tests about population means rather than population proportions

Chapter 2233 A study showed that the difference in sample means for the heights of tomato plants when using a nutrient rich potting soil versus using ordinary top soil was 6.82 inches. The corresponding standard error was 3.10 inches. Suppose the means are actually equal, so that the mean difference in heights for the populations is actually zero. What is the standardized score (z) corresponding to the observed difference of 6.82 inches? How often would you expect to see a standardized score that large or larger? Thought Question 4

Chapter 2234 Thought Question 4: Answer standardized score: [(sample mean diff.)  (population mean diff.)] divided by [standard error of the mean difference] z = (6.82  0) / 3.10 = 2.2 This is the percentile for a standard normal curve; so the probability of seeing a z-value this large or larger is 1.39% (.0139)

Chapter 2235 Thought Question 5 One of the conclusions made by researchers from a study comparing the amount of bacteria in carpeted and uncarpeted rooms was, “The average difference [in mean bacteria colonies per cubic foot] was 3.48 colonies (95% CI: -2.72, 9.68; P-value = 0.29).” What are the null and alternative hypotheses being tested here? Is there a statistically significant difference between the means of the two groups?

Chapter 2236 Thought Question 5: Answer u Null: The mean number of bacteria for carpeted rooms is equal to the mean number of bacteria for uncarpeted rooms. u Alt: The mean number of bacteria for carpeted rooms is different from the mean number of bacteria for uncarpeted rooms. u P-value is large (>.05), so there is not a significant difference (fail to reject the Null hypothesis) [Also, the confidence interval for the difference contains 0.]

Chapter 2237 Case Study (cont. from Ch. 21) Hypothetical Exercise and Pulse Rates Is the mean resting pulse rate of adult subjects who regularly exercise different from the mean resting pulse rate of those who do not regularly exercise? Use Hypothesis Testing for means

Chapter 2238 Case Study: Results Exercise and Pulse Rates u Observed difference in means = 75  66 = 9 u Standard error of the difference = 2.26 (given)

Chapter 2239 The Five Steps of Hypothesis Testing u Determining the Two Hypotheses u Computing the Sampling Distribution u Collecting and Summarizing the Data (calculating the observed test statistic) u Determining How Unlikely the Test Statistic is if the Null Hypothesis is True (calculating the P-value) u Making a Decision/Conclusion (based on the P-value, is the result statistically significant?)

Chapter 2240 The Hypotheses for a Single Mean  Null: H 0 :  =  0 u One sided alternatives H a :     0 H a :    0 u Two sided alternative H a :    0

Chapter 2241 The Hypotheses for a Difference in Two Means  Null: H 0 :  diff =  diff,0 (usually = 0) u One sided alternatives H a :  diff   diff,0 H a :  diff   diff,0 u Two sided alternative H a :  diff   diff,0

Chapter 2242 Case Study: The Hypotheses  Null: The mean resting pulse rate of adult subjects who regularly exercise is the same as the mean resting pulse rate of those who do not regularly exercise? [H 0 :  diff = 0]  Alt: The mean resting pulse rate of adult subjects who regularly exercise is different from the mean resting pulse rate of those who do not regularly exercise? [H a :  diff  0]

Chapter 2243 Also, we again replace the unknown value of  with the sample value s Sampling Distribution for Means Since we assume the null hypothesis is true, we replace  with  0 to complete the test. If numerous simple random samples of size n are taken, the sample means from the various samples will have an approximately normal distribution with mean equal to  (the population mean) and standard deviation equal to

Chapter 2244 Test Statistics for Means where is usually 0.

Chapter 2245 Case Study: Test Statistic Based on the samples: u large, random samples, so sample means follow normal distribution u observed difference in means = 9 u standard error of the difference = 2.26 u standardized score (test statistic) z = (9  0) / 2.26 = 3.98

Chapter 2246 P-value for Testing Means  H a :  >  0 u When the alternative hypothesis includes a greater than “>” symbol, the P-value is the probability of getting a value as large or larger than the observed test statistic (z) value.  look up the percentile for the value of z in the standard normal table (Table B)  the P-value is 1 minus this probability

Chapter 2247 P-value for Testing Means  H a :  <  0 u When the alternative hypothesis includes a less than “<” symbol, the P-value is the probability of getting a value as small or smaller than the observed test statistic (z) value.  look up the percentile for the value of z in the standard normal table (Table B)  the P-value is this probability

Chapter 2248 P-value for Testing Means  H a :    0 u When the alternative hypothesis includes a not equal to “  ” symbol, the P-value is found as follows:  make the value of the observed test statistic (z) positive (absolute value)  look up the percentile for this positive value of z in the standard normal table (Table B)  find 1 minus this probability  double the answer to get the P-value

Chapter Case Study: P-value < z=3.98 From Table B, 3.98 is beyond the 99.97th percentile. < z: P-value < 2  = diff: H a :  diff  0

Chapter 2250 Case Study: Decision u Since the P-value (<.0006) is small, we reject chance as the reason for the difference between the observed means (9). u We find the result to be statistically significant. u We reject the null hypothesis. The data provide evidence that the two population means (resting pulse rates) are not the same.

Chapter 2251 Case Study: Decision Error? u If in the population the mean resting pulse rate does not differ with exercise, then we have committed a Type I error. u Could we have committed a Type II error with the decision that we made? [No! Why?]

Chapter 2252 Key Concepts (2 nd half of Ch. 22) u Hypothesis Testing for a Single Mean u Hypothesis Testing for a Difference in Two Means