1. Chapter 16: Inference in Practice STAT 1450

2. Connecting Chapter 16 to our Current Knowledge of Statistics ▸ Chapter 14 equipped you with the basic tools for confidence interval construction. ▸ Chapter 15 equipped you with the basic tools for tests of significance. ▸ Chapter 16 addresses some of the nuances associated with inference (our owner’s manual of sorts). 16.0 Inference in Practice

3. Conditions for Inference ▸ Random sample:  Do we have a random sample?  If not, is the sample representative of the population?  If not, was it a randomized experiment? ▸ Large enough population: sample ratio  Is the population of interest at least 20 times larger than the sample? ▸ Large enough sample:  Are the observations from a population that has a Normal distribution, or one where we can apply principles from a Normal distribution? Look at the shape of the distribution and whether there are any outliers present. 16.1 Conditions for Inference

4. Cautions about Confidence Intervals The margin of error covers only sampling errors. ▸ Undercoverage, nonresponse, or other biases are not reflected in margins of error. ▸ The source of the data is of utmost importance. ▸ Consider the details of a study before completely trusting a confidence interval. 16.2 Cautions about Confidence Intervals

5. Example: Parental Monitoring Software ▸ Many parents elicit the use of various software and passwords to monitor the ways children use their computers. In a survey of a random sample of high school students, 16.7% with 3.45% margin of error expressed an ability to circumvent their parent’s security efforts. Would you trust a confidence interval based upon this data? Explain. 16.2 Cautions about Confidence Intervals

6. Example: Parental Monitoring Software ▸ Many parents elicit the use of various software and passwords to monitor the ways children use their computers. In a survey of a random sample of high school students, 16.7% with 3.45% margin of error expressed an ability to circumvent their parent’s security efforts. Would you trust a confidence interval based upon this data? Explain. The Confidence Interval would be (.1325,.2015). Yes, it is from a random sample. But, there is likely some under-reporting by the teens. 16.2 Cautions about Confidence Intervals

7. Example: Parental Monitoring Software ▸ Many parents elicit the use of various software and passwords to monitor the ways children use their computers. In a survey of a random sample of high school students, 16.7% with 3.45% margin of error expressed an ability to circumvent their parent’s security efforts. Would you trust a confidence interval based upon this data? Explain. The Confidence Interval would be (.1325,.2015). Yes, it is from a random sample. But, there is likely some under-reporting by the teens. As mentioned in Chapter 8, people tend to provide conservative answers to provocative questions. 16.2 Cautions about Confidence Intervals

8. Cautions about Significance Tests ▸ When H 0 represents an assumption that is widely believed, small p-values are needed. ▸ Be careful about conducting multiple analyses for a fixed .  It is preferred to just run a single test and reach a decision. 16.3 Cautions about Significance Tests

9. Cautions about Significance Tests ▸ When there are strong consequences of rejecting H 0 in favor of H A, we need strong evidence. ▸ Either way, strong evidence of rejecting H 0 requires small p-values. ▸ Depending on the situation, p-values that are below 10% can lead to rejecting H 0. ▸ Unless stated otherwise, researchers assume the de-facto significance level of 5%. 16.3 Cautions about Significance Tests

10. Cautions about Significance Tests ▸ The P-value for a one-sided tests is half of the P-value for the two-sided test of the same null hypothesis and of the same data. ▸ The two-sided case combines two equal areas. The one-sided case has one of those areas PLUS an inherent supposition by the researcher of the direction of the possible deviation from H 0. 16.3 Cautions about Significance Tests

11. A Connection between Confidence Intervals and Significance Tests ▸ Analogous to how we use high levels of confidence for confidence intervals, we need strong evidence (and very small p-values) to reject null hypotheses. ▸ Standard levels of confidence are 90%, 95%, and 99%. ▸ Standard levels of significance are 10%, 5%, and 1%. Recall from last chapter: more than 10% was a “likely” event. 5% to 10% was an “unlikely” event. < 5% was an “extremely unlikely” event. 16.3 Cautions about Significance Tests

12. Sample Size affects Statistical Significance ▸ Very large samples can yield small p-values that lead to rejection of H 0. ▸ Phenomena that are “statistically significant” are not always “practically significant.” 16.3 Cautions about Significance Tests

13. Example: Carry-on luggage ▸ Airlines are now monitoring the amount of carry-on luggage passengers bring with them. It is believed that the mean weight of carry-on luggage for passengers on multiple hour flights is 30 lbs. with a standard deviation of 7.5 lbs. A random sample of 500,000 passengers who had recently flown on multiple hour flights had an average carry-on luggage weight of 29.9 lbs. ▸ The test statistic is -9.43 with a P-value of 0. ▸ There is a statistically significant reason to reject the H 0 and believe that the mean weight of carry-on luggage is not 30 lbs. But, practically, the sample mean (29.9) and the population mean (30.0) are quite comparable. 16.3 Cautions about Significance Tests

14. Example: Carry-on luggage 16.3 Cautions about Significance Tests

15. Example: Carry-on luggage 16.3 Cautions about Significance Tests

16. Example: Carry-on luggage 16.3 Cautions about Significance Tests

17. Cautions about Significance Tests ▸ Be advised that it is better to design a single study and conduct one test of significance - (yielding one conclusion) than to design 1 study, and perform multiple analyses until a desired result is achieved. 16.3 Cautions about Significance Tests

18. Sample Size for Confidence Intervals 16.4 Planning Studies: Sample Size for Confidence Intervals

19. Example: Carry-on luggage ▸ Example: In the carry-on luggage example from earlier, a random sample of 500,000 passengers yielded a standard deviation for the sample mean that was extremely small; resulting in |z| ≈ 9.43. (a) Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? (b)Explicitly determine the sample size. 16.4 Planning Studies: Sample Size for Confidence Intervals

20. Example: Carry-on luggage (a)Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? Larger sample sizes, yield smaller margins of error. Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse. 16.4 Planning Studies: Sample Size for Confidence Intervals

21. Example: Carry-on luggage (a)Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? Larger sample sizes, yield smaller margins of error. Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse. If we desire to double the m (m=.20) we need one-fourth the sample size. 16.4 Planning Studies: Sample Size for Confidence Intervals

22. Example: Carry-on luggage (a)Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? Larger sample sizes, yield smaller margins of error. Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse. If we desire to double the m (m=.20) we need one-fourth the sample size. 500,000*(1/4) = 125,000. 16.4 Planning Studies: Sample Size for Confidence Intervals

23. Example: Carry-on luggage (a)Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? Larger sample sizes, yield smaller margins of error. Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse. If we desire to double the m (m=.20) we need one-fourth the sample size. 500,000*(1/4) = 125,000. The |z| from earlier95% Conf. 16.4 Planning Studies: Sample Size for Confidence Intervals

24. Example: Carry-on luggage (a)Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? Larger sample sizes, yield smaller margins of error. Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse. If we desire to double the m (m=.20) we need one-fourth the sample size. 500,000*(1/4) = 125,000. The |z| from earlier95% Conf. Take another fraction of 125,000. 16.4 Planning Studies: Sample Size for Confidence Intervals

25. Example: Carry-on luggage (a)Should we expect to sample more or fewer passengers if we now desire a margin of error of 0.2 lbs. with 95% confidence? Larger sample sizes, yield smaller margins of error. Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse. If we desire to double the m (m=.20) we need one-fourth the sample size. 500,000*(1/4) = 125,000. The |z| from earlier95% Conf. Take another fraction of 125,000. Further decreasing n from 500,000 to 125,000 now to a number that is a fraction of 125,000. 16.4 Planning Studies: Sample Size for Confidence Intervals

26. Example: Carry-on luggage (b)Explicitly determine the sample size. 16.4 Planning Studies: Sample Size for Confidence Intervals

27. Example: Carry-on luggage (b)Explicitly determine the sample size. 16.4 Planning Studies: Sample Size for Confidence Intervals

28. Example: Carry-on luggage (b)Explicitly determine the sample size. 16.4 Planning Studies: Sample Size for Confidence Intervals

29. Example: Carry-on luggage (b)Explicitly determine the sample size. ▸ Resulting in a much smaller sample size. 16.4 Planning Studies: Sample Size for Confidence Intervals Sorry, no TI-83 short-cuts.

30. Example: The Justice System 16.5 Errors in Significance Testing Jury Verdict Truth about the Defendant InnocentGuilty Not Guilty

31. Example: The Justice System 16.5 Errors in Significance Testing Jury Verdict Truth about the Defendant InnocentGuilty Correct decision Not Guilty Correct decision

32. Example: The Justice System 16.5 Errors in Significance Testing Jury Verdict Truth about the Defendant InnocentGuilty ErrorCorrect decision Not Guilty Correct decisionError

33. Power, Type I Error, and Type II Error Decision based on data Truth about a hypothesis Ho is trueHa is true Reject Ho Fail to reject HoCorrect decision 16.5 Errors in Significance Testing

34. Power, Type I Error, and Type II Error Decision based on data Truth about a hypothesis Ho is trueHa is true Reject HoCorrect Decision Fail to reject HoCorrect decision 16.5 Errors in Significance Testing

35. Power, Type I Error, and Type II Error Decision based on data Truth about a hypothesis Ho is trueHa is true Reject HoCorrect Decision Fail to reject HoCorrect decision 16.5 Errors in Significance Testing

36. Power, Type I Error, and Type II Error 16.5 Errors in Significance Testing ▸ Type I Error – the maximum allowable “error” of a falsely rejected H 0 (also the significance level, a). ▸ Type II Error – the probability of not rejecting H 0, when we should have rejected it. ▸ Power – the probability that the test will reject H 0 when the alternative value of the parameter is true.  Note: Increasing the sample size increases the power of a significance test. ▸ Effect size – the departure from a null hypothesis that results in practical significance.

37. Example: Coffee consumption 16.5 Errors in Significance Testing ▸ Recall the coffee consumption example from last chapter with standard deviation of 9.2 oz. A random sample of 48 people drank an average of 26.31 oz. of coffee daily. A significance test of the mean being different from our original estimate is conducted. Provide examples of , and power.

38. Example: Coffee Consumption 16.5 Errors in Significance Testing

39. Five-Minute Summary ▸ List at least 3 concepts that had the most impact on your knowledge of inference in practice. _______________________________________