1. Let’s Get It Straight! Re-expressing Data Curvilinear Regression
Chapter 9
Let’s Get It Straight!
Re-expressing Data Curvilinear Regression

2. Straight to the Point
We cannot use a linear model unless the relationship between the two variables is (approximately) linear.
Often re-expression is necessary to straighten curved relationships so that we can fit and use a simple linear model.
Ways to re-express data involve using logarithms, powers, and reciprocals.
Re-expressions can be seen in everyday life—everybody does it.

3. Straight to the Point (cont.)
The relationship between fuel efficiency (in miles per gallon) and weight (in pounds) for late model cars looks fairly linear at first:

4. Straight to the Point (cont.)
A look at the residuals plot shows a problem:

5. Straight to the Point (cont.)
We can re-express fuel efficiency as gallons per hundred miles (a reciprocal) and eliminate the bend in the original scatterplot:

6. Straight to the Point (cont.)
A look at the residuals plot for the new model seems more reasonable:

7. Goals of Re-expression
Goal 1: Make the distribution of a variable (as seen in its histogram, for example) more symmetric.

8. Goals of Re-expression (cont.)
Goal 2: Make the spread of several groups (as seen in side-by-side boxplots) more alike, even if their centers differ.

9. Goals of Re-expression (cont.)
Goal 3: Make the form of a scatterplot more nearly linear.

10. Goals of Re-expression (cont.)
Goal 4: Make the scatter in a scatterplot spread out evenly rather than thickening at one end.
This can be seen in the two scatterplots we just saw with Goal 3:

11. The Ladder of Powers
There is a family of simple re-expressions that move data toward our goals in a consistent way. This collection of re-expressions is called the Ladder of Powers.
The Ladder of Powers orders the effects that the re-expressions have on data.

12. The Ladder of Powers -1 -1/2 “0” ½ 1 2 Comment Name Power
Ratios of two quantities (e.g., mph) often benefit from a reciprocal.
The reciprocal of the data -1
An uncommon re-expression, but sometimes useful.
Reciprocal square root
-1/2
Measurements that cannot be negative often benefit from a log re-expression.
We’ll use logarithms here
“0”
Counts often benefit from a square root re-expression.
Square root of data values
Data with positive and negative values and no bounds are less likely to benefit from re-expression.
Raw data 1
Try with unimodal distributions that are skewed to the left.
Square of data values 2
Comment
Name
Power

13. Tukey’s Rule of Thumb for Re-Expression

14. Example

15. Plan B: Attack of the Logarithms
When none of the data values is zero or negative, logarithms can be a helpful ally in the search for a useful model.
Try taking the logs of both the x- and y-variable.
Then re-express the data using some combination of x or log(x) vs. y or log(y).

16. Plan B: Attack of the Logarithms (cont.)

17. Power: log(y) vs log(x)
Size matters: size of mammals and their metabolic rate
Slope < 1. Indicates that the nonlinear effect of mass on metabolic rate lessens as mass increases. Average percentage increase is about 70%

18. Multiple Benefits
We often choose a re-expression for one reason and then discover that it has helped other aspects of an analysis.
For example, a re-expression that makes a histogram more symmetric might also straighten a scatterplot or stabilize variance.

19. Why Not Just a Curve?
If there’s a curve in the scatterplot, why not just fit a curve to the data?

20. Why Not Just a Curve? (cont.)
The mathematics and calculations for “curves of best fit” are considerably more difficult than “lines of best fit.”
Besides, straight lines are easy to understand.
We know how to think about the slope and the y-intercept.

21. What Can Go Wrong? Don’t expect your model to be perfect.
Don’t choose a model based on R2 alone:

22. What Can Go Wrong? (cont.)
Beware of multiple modes.
Re-expression cannot pull separate modes together.
Watch out for scatterplots that turn around.
Re-expression can straighten many bent relationships, but not those that go up and down.

23. What Can Go Wrong? (cont.)
Watch out for negative data values.
It’s impossible to re-express negative values by any power that is not a whole number on the Ladder of Powers or to re-express values that are zero or negative powers.
Watch for data far from 1.
Data values that are all very far from 1 may not be much affected by re-expression unless the range is very large. If all the data values are large (e.g., years), consider subtracting a constant to bring them back near 1.
Don’t stray too far from the ladder.

24. What have we learned?
When the conditions for regression are not met, a simple re-expression of the data may help.
A re-expression may make the:
Distribution of a variable more symmetric.
Spread across different groups more similar.
Form of a scatterplot straighter.
Scatter around the line in a scatterplot more consistent.

25. What have we learned? (cont.)
Taking logs is often a good, simple starting point.
To search further, the Ladder of Powers or the log-log approach can help us find a good re-expression.
Our models won’t be perfect, but re-expression can lead us to a useful model.

26. Re-expressing Data Curvilinear Regression (aka Polynomial Regression)
Chapter 9 (cont.)
Let’s Get It Straight!
Re-expressing Data Curvilinear Regression
(aka Polynomial Regression)
Previous slides
Next

27. Polynomial Regression Model
To model this behavior we include additional terms that have higher powers of the explanatory variable x
Second-Order Model
Third-Order Model
We will not go beyond degree 3

28. Example: Fast Food Revenue
You are asked to develop a regression model for a fast food restaurant. The primary market is middle-income families and their children, particularly those between the ages of 5 and 12.
Response variable —gross restaurant revenue
Explanatory variable — family income (median family income in “neighborhood” of restaurant)

29. Analysis of variance table for regression model: Parameter Estimate
Simple linear regression results: Dependent Variable: Revenue (000's) Independent Variable: Income (000's) Revenue (000's) = Income (000's) Sample size: 25 R (correlation coefficient) = R-sq = Estimate of error standard deviation: Parameter estimates:
Analysis of variance table for regression model:
Parameter
Estimate
Std. Err.
Alternative DF
T-Stat
P-value
Intercept
≠ 0 23
<0.0001
Slope
0.0296
Source DF SS MS
F-stat
P-value
Model 1
0.0296
Error 23
Total 24

30. Residual Plot (Oh-Oh)

31. Scatterplot Indicates 2nd Order Term Needed

32. Regression Statistics
Excel Output
Regression Statistics
Multiple R
R Square
Adjusted R Sq
Standard Error
Observations 25
ANOVA df SS MS F
Significance F
Regression 2
163035
E-08
Residual 22
3637.5
Total 24
Coefficients
t Stat
P-value
Lower 95%
Upper 95%
Intercept
-5.195
3.3E-05
Income (000's)
8.7118
1.4E-08
Income sq
-8.275
3.4E-08

33. Residual Plots
We improved, but can we do even better?

34. Scatterplots

35. Expanded Model
Where
x1 is the median family income in the “neighborhood”
x2 is the average child age in the “neighborhood”
Should we include the interaction term in the model? When in doubt, it’s probably best to include it

36. Regression Statistics
Multiple R
R Square
Adjusted R Sq.
Standard Error
Observations 25
ANOVA df SS MS F
Significance F
Regression 5
73628
E-09
Residual 19
1997.7
Total 24
Coefficients
t Stat
P-value
Lower 95%
Upper 95%
Intercept
-3.543
Income (000's)
6.1411
6.66E-06
Income sq
-6.873
1.48E-06
Age
0.7306
Age sq
-3.281
(Income)( Age)
2.0838
Multicollinearity

37. Residual Plot