Hypothesis Testing ESM Feb. 2002

Example: Gas Mileage SMALLCOMPACT Eagle SummitAudi 80 Ford EscortBuick Skylark Ford FestivaChevrolet LeBaron Honda CivicFord Tempo Mazda ProtégéHonda Accord Mercury TracerMazda 626 Nissan SentraMitsubishi Galant Pontiac LeMansMitsubishi Sigma Subaru LoyaleNissan Stanza Subary JustyOldsmobile Calais Toyota CorollaPeugeot 405 Toyota TercelSubaru Legacy Volkswagen JettaToyota Camry Do “Small” cars have a different average gas mileage than “Compact” cars? Data on mileage of 13 small and 15 compact cars.

Example: gas consumption Which coefficients are different from zero? Data from 36 years in US.

Hypothesis testing Define null hypothesis (H 0 ) Does direction matter? Choose test statistic, T Distribution of T under H 0 Calculate test statistic, S Probability of obtaining value at least as extreme as S under H 0 (P) P small: reject H 0

The null hypothesis Statement about underlying parameters of the population We will either reject or fail to reject H 0 Usually a statement of no pattern or of not exceeding some criterion Examples

The alternate hypothesis Written H A Is the logical complement of H 0 Examples

One- and two-sided tests One-sided test: direction matters Pick a direction based on regulatory criteria or knowledge of processes Direction must be chosen a priori Two-sided: all that matters is a difference One-sided has greater power Must make decision before analyzing data

Comparing means: the t-test Compare sample mean to fixed value (eqs. 1-4) Compare regression coefficient to fixed value (eq. 5) Compare the difference between two sample means to a fixed value (usually 0) (eqs. 6-7)

Assumptions of the t-test The data in each sample are normally distributed The populations have the same variance Can correct for violations of this with the Welch modification of df Test for difference among variances with F-test

The P-value P is the probability of observing your data if the null hypothesis is true P is the probability that you will be in error if you reject the null hypothesis P is not the probability that the null hypothesis is true

Critical values of P Reject H 0 if P is less than threshold P < 0.05 commonly used Arbitrary choice Other values: 0.1, 0.01, Always report P, so others can draw own conclusions

Example: Gas Mileage SMALLCOMPACT Eagle SummitAudi 80 Ford EscortBuick Skylark Ford FestivaChevrolet LeBaron Honda CivicFord Tempo Mazda ProtégéHonda Accord Mercury TracerMazda 626 Nissan SentraMitsubishi Galant Pontiac LeMansMitsubishi Sigma Subaru LoyaleNissan Stanza Subary JustyOldsmobile Calais Toyota CorollaPeugeot 405 Toyota TercelSubaru Legacy Volkswagen JettaToyota Camry Do “Small” cars have a different average gas mileage than “Compact” cars? Data on mileage of 13 small and 15 compact cars.

Gas mileage: variances are unequal

Gas mileage Test Name: Welch Modified Two-Sample t-Test Estimated Parameter(s): mean of x = 31 mean of y = Data: x: Small in DS2, and y: Compact in DS2 Test Statistic: t = Test Statistic Parameter: df = P-value: % Confidence Interval: LCL = UCL =

Example: gas consumption Which coefficients are different from zero? Data from 36 years in US.

Gas consumption Value Std. Error t value Pr(>|t|) (Intercept) GasPrice Income New.Car.Price Used.Car.Price

Interpreting model coefficients Is there statistical evidence that the independent variable has an effect? Is the parameter estimate significantly different from zero? Is the coefficient large enough that the effect is important? Must take into account the variation in the independent variable Use linear measure of variation – SD, IQ range, etc.

Types of error Type I: reject null hypothesis when it’s really true Desired level:  Type II: fail to reject null hypothesis when it’s really false Desired level:  Is associated with a given effect size E.g., want a probability 0.1 of failing to reject when true difference between means is 0.35.

Types of error In reality, H 0 is TrueFalse Your test says that H 0 should be: AcceptedCorrect conclusion Type II error RejectedType I error Correct conclusion

Controlling error levels  is controlled by setting critical P-value  is controlled by , sample size, sample variance, effect size Tradeoff between  and  Need to balance costs associated with type I and type II errors Power is 1- 