Using Inference to Make Decisions Lesson 11 - 4 Using Inference to Make Decisions

Knowledge Objectives Define what is meant by a Type I error. Define what is meant by a Type II error. Define what is meant by the power of a test. Identify the relationship between the power of a test and a Type II error. List four ways to increase the power of a test.

Construction Objectives Describe, given a real situation, what constitutes a Type I error and what the consequences of such an error would be. Describe, given a real situation, what constitutes a Type II error and what the consequences of such an error would be. Describe the relationship between significance level and a Type I error. Explain why a large value for the power of a test is desirable.

Vocabulary Power of the test – value of 1 – β Power curve – a graph that shows the power of the test against values of the population mean that make the null hypothesis false. Level of Significance – probability of making a Type I error, α

Hypothesis Testing: Four Outcomes Reality H0 is True Ha is True Conclusion Do Not Reject H0 Correct Conclusion Type II Error Reject H0 Type I Error H0: the defendant is innocent H1: the defendant is guilty Type I Error (α): convict an innocent person Type II Error (β): let a guilty person go free Note: a defendant is never declared innocent; just not guilty decrease α  increase β increase α  decrease β

Hypothesis Testing: Four Outcomes We reject the null hypothesis when the alternative hypothesis is true (Correct Decision) We do not reject the null hypothesis when the null hypothesis is true (Correct Decision) We reject the null hypothesis when the null hypothesis is true (Incorrect Decision – Type I error) We do not reject the null hypothesis when the alternative hypothesis is true (Incorrect Decision – Type II error)

Example 1 You have created a new manufacturing method for producing widgets, which you claim will reduce the time necessary for assembling the parts. Currently it takes 75 seconds to produce a widget. The retooling of the plant for this change is very expensive and will involve a lot of downtime. Ho : Ha: TYPE I: TYPE II:

Example 1 Ho : µ = 75 (no difference with the new method) Ha: µ < 75 (time will be reduced) TYPE I: Determine that the new process reduces time when it actually does not. You end up spending lots of money retooling when there will be no savings. The plant is shut unnecessarily and production is lost. TYPE II: Determine that the new process does not reduce when it actually does lead to a reduction. You end up not improving the situation, you don't save money, and you don't reduce manufacturing time.

Example 2 A potato chip producer wants to test the hypothesis H0: p = 0.08 proportion of potatoes with blemishes Ha: p < 0.08 Let’s examine the two types of errors that the producer could make and the consequences of each Type I Error: Type II Error: Description: producer concludes that the p < 8% when its actually greater Consequence: producer accepts shipment with sub-standard potatoes; consumers may choose not to come back to the product after a bad bag Description: producer concludes that the p > 8% when its actually less Consequence: producer rejects shipment with acceptable potatoes; possible damage to supplier relationship and to production schedule

Example 3 A city manager’s staff takes a random sample of 400 emergency call response times that yielded x-bar = 6.48 minutes with a standard deviation of 2 minutes. The manager wants to know if the response time decreased from last year’s mean of 6.7 min? Parameter to be tested: Test Type: H0: Ha: mean response time in min left-tailed test Mean response time,  = 6.7 minutes Mean response time,  < 6.7 minutes

Example 3 H0: Mean response time,  = 6.7 minutes Ha: Mean response time,  < 6.7 minutes Give the description and consequences of the two error types: Type I: Type II: The manager concludes that the response times have improved, when they really have not. No additional funding for improvement; possible additional lives lost. The manager concludes that the response times still need to be improved, when they have improved already. Additional funds spent unnecessarily and morale might be lowered.

Graphical View of Error Types Area to the left of critical value under the right most curve is the Type I error Area to the right of critical value under the left most curve is the Type II error As the critical value of x-bar moves right α increases and β decreases As the critical value of x-bar moves left α decreases and β increases Need to identify the differences in errors and their consequences in a given problem

Finding P(Type II Error) Determine the sample mean that separates the rejection region from the non-rejection region x-bar = μ0 ± zα · σ/√n Draw a normal curve whose mean is a particular value from the alternative hypothesis, with the sample mean(s) found in step 1 labeled. The area described below represents β, the probability of not rejecting the null hypothesis when the alternative hypothesis is true. a. Left-tailed Test: Find the area under the normal curve drawn in step 2 to the right of x-bar b. Two-tailed Test: Find the area under the normal curve drawn in step 2 between xl and xu c. Right-tailed Test: Find the area under the normal curve drawn in step 2 to the left of x-bar

Example 3 The current wood preservative (CUR) preserves the wood for 6.40 years under certain conditions. We have a new preservative (NEW) that we believe is better, that it will in fact work for 7.40 years Ho : Ha: TYPE I: TYPE II: μ = 6.40 (our preservative is same as the current) μ = 7.40 (new is significantly better than the current)

Example 3 Our hypotheses H0: μ = 6.40 (our preservative is the same as the current one) H1: μ = 7.40 (our preservative is significantly better than the current one) H0 H1

Example 3 Type I error Assumes that H0 is true (that NEW is no better) Our experiment leads us to reject H0 H1 H0 This area is the P(Type I error) Critical Value

Example 3 Type I errors Assumes that H0 is true (that NEW is no better) Our experiment leads us to reject H0 Result – we conclude that NEW is significantly better, when it actually isn’t This will lead to unrealistic expectations from our customers that our product actually works better

Example 3 Type II errors Assumes that H1 is true (that NEW is better) Our experiment leads us to not reject H0 This area is the P(Type II error) H1 H0 Critical Value (the same as before)

Example 3 Type II errors Assumes that H1 is true (that NEW is better) Our experiment leads us to not reject H0 Result – we conclude that NEW is not significantly better, when it actually is This will lead to customers not getting a better treatment because we didn’t realize that it was better

Example 3 Test Details The standard error of the mean is We test our product on n = 60 wood planks We have a known standard deviation σ = 3.2 We use a significance level of α = 0.05 The standard error of the mean is The critical value (for a right-tailed test) is 6.40 + 1.645  0.41 = 7.08 3.2 ----- = 0.41 60

Example 3 The Type II error, β, is the probability of not rejecting H0 when H1 is true H1 is that the true mean is 7.40 The area where H0 is not rejected is where the sample mean is 7.08 or less The probability that the sample mean is 7.08 or less, given that it’s mean is 7.40, is Thus β, the Type II error, is 0.22 Power of the test is 1 – β = 0.78 7.08 – 7.40 β = P( z < ------------------) = P(z < -0.77) = 0.22 0.41

Power and Type II Error Probability of a Type II error is β Power of the test is 1 – β P-value describes what would happen supposing the null hypothesis is true Power describes what would happen supposing that a particular alternative is true

Increasing the Power of a Test Four Main Methods: Increase significance level,  Consider a particular alternative that is farther away from  Increase the sample size, n, in the experiment Decrease the population (or sample) standard deviation, σ Only increasing the sample size and the significance level are under the control of the researcher

Comparisons P-value, (compared to ) assumes that H0 is true Power, (1 - ) assumes that some alternative Ha is true

Summary and Homework Summary Homework A Type I error occurs if we reject H0 when in fact its true; P(Type I) = α A Type II error occurs if we fail to reject HO when in fact its false; P(Type II) = β Power of a significance test measures its ability to detect an alternative hypothesis and is = 1 - β Increasing the power of a test can be done by increasing sample size and by using a higher α Homework pg 727 – 735; 11.50, 51, 59-61