1. STAT 3120 Statistical Methods I Lecture 4 Hypothesis Testing

2. STAT3120 – Hypothesis Testing  Some Limitations of Confidence Interval Analysis –Descriptive –Provides limited analysis of potential errors (Impact/likelihood)  Hypothesis Testing is Decision Oriented –Is a Population Parameter (or Proportion) less than, Equal to, or Greater than a specific Value (With Decision Making Implications) –Guides user to a particular choice –Provides User with quantitative information regarding probabilities of different outcomes (both good and bad).  Highlights that Two Different Decision Making Errors Possible –TYPE I or  -Error –TYPE II or  -Error   -Level of significance predefines standard of proof (risk)  1-  – indicating the “power” of the test (the probability of finding a significant effect)  p-Value (Prob-Value) Aids in Interpreting Results –Strength of the Evidence

3. STAT3120 – Hypothesis Testing The first step in Hypothesis Testing is to develop a claim to be tested. For example: Drug A will lower an individual’s cholesterol level by a minimum of 10 points. Trashbag A has greater tensile strength than Trashbag B. Automobile A will average at least 35 miles to the gallon. Note that every possible claim has an opposite statement – the two statements together must be mutually exclusive and collectively exhaustive.

4. STAT3120 – Hypothesis Testing When conducting Hypothesis Testing, the process of testing includes five distinct parts: 1. Statement of the Claim – This is referred to as the Alternative Hypothesis and is designated as H 1 or H a ; 2. Statement of the Opposite of the Claim – This is referred to as the Null Hypothesis and is designated as H 0. You can think of this as the “base case” or “do nothing” scenario; 3. Calculation of the appropriate Test Statistic; 4. Identification of the Rejection Region (H 0 ); 5. Assess Results and Draw Conclusions.

5. STAT3120 – Hypothesis Testing Note that Hypothesis Statements can take one of three forms (these are the Null Statements): Two Sided Test:  ≠ H 0 One Sided Test:  < H 0 One Sided Test:  >H 0

6. Develop the appropriate Hypothesis Statements to test the claims (try to develop the statements using mathematical operators): 1.The Coca Cola Marketing Department wants to run a TV ad – “Diet Coke tastes better than Diet Pepsi”. Assume some kind of taste scale of 1-10 (10 is the best). 2.A Pharmaceutical company wants to claim in their marketing materials that a particular drug will lower cholesterol by at least 30%. 3.A manufacturer of PVC pipes wants to become a supplier to a large civil engineering firm. The manufacturer claims that they can manufacture pipes to within 1mm of the engineering firms specifications. 4.The Milemaster Tire company has a new tire that they claim will go 100,000 before the treads wear out. STAT3120 – Hypothesis Testing

7. True State of nature Decision Ho is true Ho is false Reject Ho Do not reject Ho When conducting a hypothesis test, you should always develop the 2x2 matrix below – which compares the statistically-supported decision to the “true” state of nature:

8. Descriptions of errors:  Type I Error: Reject Null Hypothesis When Null is actually true (  - Error)  Type II Error: Accept Null Hypothesis which is false (  -Error)  Significance level, , is Maximum Risk of making Type I error that we are prepared to “Live With”. In other words, Probability of Type I Error =  (usually set at.05 or Less)  A Type II Error =  (not typically controlled)  Type I Error and Type II error cannot be controlled simultaneously Type of ErrorDECISIONCONSEQUENCES/COSTS TYPE IReject Null; Null is true Market tire, but should not have done so; may cause customer dissatisfaction, loss, claims for refund TYPE IIAccept Null; Null is false; Fail to market a good product; opportunity cost STAT3120 – Hypothesis Testing

9. Tire is no good Ho is true Tire meets expectations Ho is false Market Tire Reject Ho Don’t Market Tire Do not reject Ho customer dissatisfaction, lose market share Type 1 Error -  Gain market share Valid Decision* No change Valid Decision Fail to capitalize on good product, opportunity cost Type II Error -  STAT3120 – Hypothesis Testing * The calculated probability of this outcome is 1- . This is known as the “Power” of the test.

10. STAT3120 – Hypothesis Testing Ho is trueHo is false Reject Ho Typically the Worst Possible Mistake*…this decision asserts that an effect is present when it is not; a false positive. The “Power” of the test. There was an effect present and it was detected. Do not reject Ho A “Push”. There was no effect present and this was correctly determined. Lost opportunity. There was an effect present and it was not detected; a false negative* Descriptions of each outcome “in English”: * In certain contexts, the False Negative is may be the worse mistake.

11. STAT3120 – Hypothesis Testing After the Hypothesis Statements have been developed, and the Type I and Type II errors have been evaluated, we establish the alpha level – the highest probability we are willing to assume of committing a Type I error. This alpha value corresponds to a “Critical” or cut off Z-score*: One tailed tests alpha/Z-score.01/2.33.05/1.645.10/1.28 Two tailed tests alpha/Z-score.01/2.575.05/1.96.10/1.645 *Note that if the sample sizes are less than 100, a t-stat should be used over a Z-stat.

12. At this point, we perform the calculation of the ACTUAL Z-score (or t-stat) and compare this to the CRITICAL Z-score (or t-stat): STAT3120 – Hypothesis Testing For Example… A corporation manages a fleet of company cars. A random sample of 40 cars is examined. The mean and std for the sample are 2,752 and 350 miles, respectively. Records for previous years indicated that the average miles driven was 2,600. Use the sample data to test the claim that the current mean is different from the previous mean. Use alpha =.05. t = (2752-2600)/(350/SQRT(40)) = 2.75

13. Now, consider these questions… 1.How does this compare to a Critical t of 2.02? 2.What is your decision? 3.What is the implication if you are wrong? 4.What if the claim was testing if “the current mean is greater than the previous mean”? How would your answer change (if at all)? STAT3120 – Hypothesis Testing

14. Sample Conditions p-ValueSignificance level Z-Score/t-stat Associated with the Actual or Calculated Z- score/t-stat Associated with the Critical Z- score/t-stat Range 0 < p < 1 0 <  < 1 Definition Actual Probability of Making a Type I Error Maximum Probability of Making a Type I Error How Determined and When Known From Sample Data After Analysis Set Before Analysis STAT3120 – Hypothesis Testing The Z-scores can be compared directly. However, we typically translate these Z-scores into probabilities:

15. STAT3120 – Hypothesis Testing Decision Rule: If the p-value is less than the established alpha value, REJECT the null hypothesis and proceed with the claim. The potential error is a Type I (Type II is not possible and therefore not relevant). If the p-value is greater than the established alpha value, we FAIL TO REJECT the null hypothesis and maintain the status quo. The potential error is a Type II (Type I is not possible and therefore not relevant).

16. STAT3120 – Hypothesis Testing Fun Exercise to do by Hand (yeah!): A dealer in recycled paper places empty trailers behind Whole Foods locations. The trailers are gradually filled by customers stopping by with their old newspapers and cardboard. The dealer currently picks up the trailers every other week. This schedule works as long as the average amount of recycled paper is more than 1600 cubic feet (the amount needed to justify operating costs of the trailer). The dealer’s records for 18 2-week periods show the following volumes for the Whole Foods on Briarcliff Road and La Vista: 1660 1820 1590 1440 1730 1680 1750 1720 1900 1570 1700 1900 1800 1770 2010 1580 1620 1690

17. STAT3120 – Hypothesis Testing 1.What are the Hypothesis Statements? 2.Complete the testing Matrix. 3.Calculate the Appropriate test statistic (since there are less than 30 observations, lets use a t-stat rather than a z-stat). a)For fun, build the 95% CI – is the target value included? 4.Identify the rejection region for the testing distribution. 5.Draw your conclusion using alpha =.05. 6.What are the implications of making the wrong choice?