1. Hypothesis Testing ESM 206 6 Feb. 2002

2. Example: Gas Mileage SMALLCOMPACT Eagle SummitAudi 80 Ford EscortBuick Skylark Ford FestivaChevrolet LeBaron Honda CivicFord Tempo Mazda ProtégéHonda Accord Mercury TracerMazda 626 Nissan SentraMitsubishi Galant Pontiac LeMansMitsubishi Sigma Subaru LoyaleNissan Stanza Subary JustyOldsmobile Calais Toyota CorollaPeugeot 405 Toyota TercelSubaru Legacy Volkswagen JettaToyota Camry Do “Small” cars have a different average gas mileage than “Compact” cars? Data on mileage of 13 small and 15 compact cars.

3. Example: gas consumption Which coefficients are different from zero? Data from 36 years in US.

4. Hypothesis testing Define null hypothesis (H 0 ) Does direction matter? Choose test statistic, T Distribution of T under H 0 Calculate test statistic, S Probability of obtaining value at least as extreme as S under H 0 (P) P small: reject H 0

5. The null hypothesis Statement about underlying parameters of the population We will either reject or fail to reject H 0 Usually a statement of no pattern or of not exceeding some criterion Examples

6. The alternate hypothesis Written H A Is the logical complement of H 0 Examples

7. One- and two-sided tests One-sided test: direction matters Pick a direction based on regulatory criteria or knowledge of processes Direction must be chosen a priori Two-sided: all that matters is a difference One-sided has greater power Must make decision before analyzing data

8. Comparing means: the t-test Compare sample mean to fixed value (eqs. 1-4) Compare regression coefficient to fixed value (eq. 5) Compare the difference between two sample means to a fixed value (usually 0) (eqs. 6-7)

9. Assumptions of the t-test The data in each sample are normally distributed The populations have the same variance Can correct for violations of this with the Welch modification of df Test for difference among variances with F-test

10. The P-value P is the probability of observing your data if the null hypothesis is true P is the probability that you will be in error if you reject the null hypothesis P is not the probability that the null hypothesis is true

11. Critical values of P Reject H 0 if P is less than threshold P < 0.05 commonly used Arbitrary choice Other values: 0.1, 0.01, 0.001 Always report P, so others can draw own conclusions

12. Example: Gas Mileage SMALLCOMPACT Eagle SummitAudi 80 Ford EscortBuick Skylark Ford FestivaChevrolet LeBaron Honda CivicFord Tempo Mazda ProtégéHonda Accord Mercury TracerMazda 626 Nissan SentraMitsubishi Galant Pontiac LeMansMitsubishi Sigma Subaru LoyaleNissan Stanza Subary JustyOldsmobile Calais Toyota CorollaPeugeot 405 Toyota TercelSubaru Legacy Volkswagen JettaToyota Camry Do “Small” cars have a different average gas mileage than “Compact” cars? Data on mileage of 13 small and 15 compact cars.

13. Gas mileage: variances are unequal

14. Gas mileage Test Name: Welch Modified Two-Sample t-Test Estimated Parameter(s): mean of x = 31 mean of y = 24.13333 Data: x: Small in DS2, and y: Compact in DS2 Test Statistic: t = 5.905054 Test Statistic Parameter: df = 16.98065 P-value: 0.00001738092 95 % Confidence Interval: LCL = 4.413064 UCL = 9.32027

15. Example: gas consumption Which coefficients are different from zero? Data from 36 years in US.

16. Gas consumption Value Std. Error t value Pr(>|t|) (Intercept) -0.0898 0.0508 -1.7687 0.0868 GasPrice -0.0424 0.0098 -4.3058 0.0002 Income 0.0002 0.0000 23.4189 0.0000 New.Car.Price -0.1014 0.0617 -1.6429 0.1105 Used.Car.Price -0.0432 0.0241 -1.7913 0.0830

17. Interpreting model coefficients Is there statistical evidence that the independent variable has an effect? Is the parameter estimate significantly different from zero? Is the coefficient large enough that the effect is important? Must take into account the variation in the independent variable Use linear measure of variation – SD, IQ range, etc.

18. Types of error Type I: reject null hypothesis when it’s really true Desired level:  Type II: fail to reject null hypothesis when it’s really false Desired level:  Is associated with a given effect size E.g., want a probability 0.1 of failing to reject when true difference between means is 0.35.

19. Types of error In reality, H 0 is TrueFalse Your test says that H 0 should be: AcceptedCorrect conclusion Type II error RejectedType I error Correct conclusion

21. Controlling error levels  is controlled by setting critical P-value  is controlled by , sample size, sample variance, effect size Tradeoff between  and  Need to balance costs associated with type I and type II errors Power is 1- 