Regression Examples

Gas Mileage 1993 SOURCES: Consumer Reports: The 1993 Cars - Annual Auto Issue (April 1993), Yonkers, NY: Consumers Union. PACE New Car & Truck 1993 Buying Guide (1993), Milwaukee, WI: Pace Publications Inc. Specifications are given for 93 new car models for the 1993 year.

Gas Mileage vs Weight Several measures are available, such as price, mpg ratings, engine size, cylinders, weight, horsepower, etc. We consider the relationship between weight and highway mpg. Since more fuel is needed to move more weight, an increase in weight should result in a decrease in mpg.

Pearson Correlation Coefficients, N = 93 Prob > |r| under H0: Rho=0 mpg weight mpg <.0001 weight <.0001 SAS Output

The REG Procedure Model: MODEL1 Dependent Variable: mpg Number of Observations Read 93 Number of Observations Used 93 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model <.0001 Error Corrected Total Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept <.0001 weight <.0001

Interpretation From the graph it is evident that there is a fairly strong negative correlation between weight and mpg. The correlation output tells us that r=-.81. The regression output, under parameter estimates, tells us that the equation for the least squares line (best fit line) is mpg= *weight. Also, there are standard errors for the estimates that can be used to build confidence intervals, and there are t- values and p-values for a test of the hypothesis H o : Parameter=0 (vs not =0). Since the p-value for weight is.0001, we can reject H o and conclude that there is a linear relationship between weight and mpg (weight contributes information about mpg, or helps to predict mpg).

Gas Mileage vs Engine Size Generally speaking, larger engines burn more fuel (but is this due to weight?). We can check the relationship between liters (engine displacement) and mpg. We expect a negative relationship, since a larger engine would tend to decrease mpg.

Pearson Correlation Coefficients, N = 93 Prob > |r| under H0: Rho=0 mpg liters mpg <.0001 liters <.0001

The REG Procedure Model: MODEL1 Dependent Variable: mpg Number of Observations Read 93 Number of Observations Used 93 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model <.0001 Error Corrected Total Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept <.0001 liters <.0001

Interpretation From the first graph it is evident that there is a negative relationship between weight and mpg. However, the pattern is not purely linear. It seems to be some kind of curve. Thus we will not expect a linear analysis to tell the whole story. The correlation output tells us that r=-.63. The regression output, tells us that the equation for the least squares line is mpg=37.68 – 3.22*liters. The p-value for liters is.0001, so we conclude that mpg has a linear relationship with engine size. Which is a better predictor of mpg, engine size or weight? We can use the R-square value to determine that. For weight it is.6572, and for liters it is.3929.

More Advanced Interpretation Which is a better predictor of mpg, engine size or weight? We can use the R-square value to determine that. For weight it is.6572, and for liters it is R-square is an indication of the proportion of changes in y that are accounted for by x, so a larger value corresponds to a better predictor. Thus weight is a better predictor than engine size. The graphs show the best fit line and a best fit parabola (quadratic equation). The latter is provided for comparison purposes only. Note that the quadratic equation, even though it fits the points better, may not be a better model, because it shows mpg rising as engine sizes get very large. This does not make sense.

Brief Look at Multiple Regression Now you might think, what if we wanted to use both weight and engine size to predict mpg? This idea is called multiple regression, and it involves making an equation with two or more “x” variables to predict y. The next regression output shows this. Compare the R-square and p-values to previous results.

The REG Procedure Model: MODEL1 Dependent Variable: mpg Number of Observations Read 93 Number of Observations Used 93 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model <.0001 Error Corrected Total Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept <.0001 liters weight <.0001

Interpretation The least squares equation is mpg= *liters *weight. The R-square for weight alone is In the new model, it is It has gone up, but not much. This means that adding engine size to the equation does not improve predicted mpg very much. The p-value for weight is still very small, but the p-value for liters is now suspiciously large. Using alpha=.05, we would not reject that the coefficient of liters is zero, which means we are not able to detect a contribution to mpg.