1. Statistical Methods Lecture 28

2. Extrapolation and Prediction
QTM1310/ Sharpe
Extrapolation and Prediction
Extrapolating – predicting a y value by extending the regression model to regions outside the range of the x-values of the data. 2

3. Extrapolation and Prediction
QTM1310/ Sharpe
Extrapolation and Prediction
Why is extrapolation dangerous?
It introduces the questionable and untested assumption that the relationship between x and y does not change. 3

4. Extrapolation and Prediction
QTM1310/ Sharpe
Extrapolation and Prediction
Cautionary Example: Oil Prices in Constant Dollars
Price = – Time
Model Prediction (Extrapolation):
On average, a barrel of oil will increase $7.39 per year from 1983 to 1998. 4

5. Extrapolation and Prediction
QTM1310/ Sharpe
Extrapolation and Prediction
Cautionary Example: Oil Prices in Constant Dollars
Price = – Time
Actual Price Behavior
Extrapolating the model to the ’80s and ’90s lead to grossly erroneous forecasts. 5

6. Extrapolation and Prediction
QTM1310/ Sharpe
Extrapolation and Prediction
Remember: Linear models ought not be trusted beyond the span of the x-values of the data.
If you extrapolate far into the future, be prepared for the actual values to be (possibly quite) different from your predictions. 6

7. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Outliers, Leverage, and Influence
In regression, an outlier can stand out in two ways. It can have…
1) a large residual: 7

8. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Outliers, Leverage, and Influence
In regression, an outlier can stand out in two ways. It can have…
2) a large distance from :
“High-leverage
point”
A high leverage point is influential if omitting it gives a regression model with a very different slope. 8

9. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Outliers, Leverage, and Influence
Tell whether the point is a high-leverage point, if it has a large residual, and if it is influential.
Not high-leverage
Large residual
Not very influential 9

10. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Outliers, Leverage, and Influence
Tell whether the point is a high-leverage point, if it has a large residual, and if it is influential.
High-leverage
Small residual
Not very influential 10

11. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Outliers, Leverage, and Influence
Tell whether the point is a high-leverage point, if it has a large residual, and if it is influential.
High-leverage
Medium (large?) residual
Very influential (omitting the red point will change the slope dramatically!)
large? comment—I would expect many students to say large, because they would be answering “large” based on the residual from a line without that point included 11

12. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Outliers, Leverage, and Influence
What should you do with a high-leverage point?
Sometimes, these points are important. They can indicate that the underlying relationship is in fact nonlinear.
Other times, they simply do not belong with the rest of the data and ought to be omitted.
When in doubt, create and report two models: one with the outlier and one without. 12

13. Unusual and Extraordinary Observations
QTM1310/ Sharpe
Unusual and Extraordinary Observations
Example: Hard Drive Prices
Prices for external hard drives are linearly associated with the Capacity (in GB). The least squares regression line without a 200 GB drive that sold for $ was found to be
The regression equation with the original data is
How are the two equations different?
The intercepts are different, but the slopes are similar.
Does the new point have a large residual? Explain.
Yes. The hard drive’s price doesn’t fit the pattern since it pulled the line up but didn’t decrease the slope very much. 13

14. Working with Summary Values
QTM1310/ Sharpe
Working with Summary Values
Scatterplots of summarized (averaged) data tend to show less variability than the un-summarized data.
Example:
Wind speeds at two locations, collected at 6AM, noon, 6PM, and midnight.
Raw data:
R2 = 0.736
Daily-averaged
data:
R2 = 0.844
Monthly-averaged data:
R2 = 0.942
Next 3 slides are merely cautionary—these (summarized) data don’t violate any of the conditions of doing linear regression per se; however, you must take into account that the association you are describing is among summarized data 14

15. Working with Summary Values
QTM1310/ Sharpe
Working with Summary Values
WARNING:
Be suspicious of conclusions based on regressions of summary data.
Regressions based on summary data may look better than they really are!
In particular, the strength of the correlation will be misleading. 15

16. QTM1310/ Sharpe
Autocorrelation
Time-series data are sometimes autocorrelated, meaning points near each other in time will be related.
First-order autocorrelation: Adjacent measurements are related
Second-order autocorrelation: Every other measurement is related
etc…
Autocorrelation violates the independence condition. Regression analysis of autocorrelated data can produce misleading results. 16

17. Transforming (Re-expressing) Data
QTM1310/ Sharpe
Transforming (Re-expressing) Data
An aside
On using technology:
Make sure to point out Residuals, and Residuals plots on the 3rd thing to “pop up”. 17

18. Transforming (Re-expressing) Data
QTM1310/ Sharpe
Transforming (Re-expressing) Data
Linearity
Some data show departures from linearity.
Example: Auto Weight vs. Fuel Efficiency
Linearity condition is not satisfied. 18

19. Transforming (Re-expressing) Data
QTM1310/ Sharpe
Transforming (Re-expressing) Data
Linearity
In cases involving upward bends of negatively-correlated data, try analyzing –1/y (negative reciprocal of y) vs. x instead.
Linearity condition now appears satisfied. 19

20. Transforming (Re-expressing) Data
QTM1310/ Sharpe
Transforming (Re-expressing) Data
The auto weight vs. fuel economy example illustrates the principle of transforming data.
There is nothing sacred about the way x-values or y-values are measured. From the standpoint of measurement, all of the following may be equally-reasonable:
x vs. y
x vs. –1/y
x2 vs. y
x vs. log(y)
One or more of these transformations may be useful for making data more linear, more normal, etc. 20

21. Transforming (Re-expressing) Data
QTM1310/ Sharpe
Transforming (Re-expressing) Data
Goals of Re-expression
Goal 1 Make the distribution of a variable more symmetric. 21

22. Transforming (Re-expressing) Data
QTM1310/ Sharpe
Transforming (Re-expressing) Data
Goals of Re-expression
Goal 2 Make the spread of several groups more alike.
We’ll see methods later in the book that can be applied only to groups with a common standard deviation. 22

23. Looking back
Make sure the relationship is straight enough to fit a regression model.
Beware of extrapolating.
Treat unusual points honestly. You must not eliminate points simply to “get a good fit”.
Watch out when dealing with data that are summaries.
Re-express your data when necessary.
Assumptions: quantitative data from incr. normal popn as sample gets smaller, r. sample
Particular form of the test statistic
Two-tailed test: effect on the p-value (have to double tail prob) or rejection region (have to take given alpha & divide it between two tails)