1. Prediction, Correlation, and Lack of Fit in Regression (§11. 4, 11
Outline
Confidence interval and prediction interval.
Regression Assumptions.
Checking Assumptions (model adequacy).
Correlation.
Influential observations.

2. Prediction Our regression model is
Number Repair
of components time
i xi yi
so that the average value of the response at X=x is

3. The estimated average response at X=x is therefore
The expected value!
This quantity is a statistic, a random variable, hence it has a sampling distribution.
Regression Assumptions
Normal Distribution for 
Sample estimate, and associated variance:
A (1-a)100% CI for the average response at X=x is therefore:

4. Prediction and Predictor Confidence
The best predictor of an individual response y at X=x, yx,pred, is simply the average response at X=x.
Random variables -- they vary from sample-to-sample.
Variance associated with an individual prediction is larger than that for the mean value! Why?
Hence the predicted value
is also a random variable.
A (1-a)100% CI for an individual response at X=x:

5. Prediction band - what would we expect for one new observation.
Confidence band - what would we expect for the mean of many observations taken at the value of X=x.

7. Regression Assumptions and Lack of Fit Regression Model Assumptions
Effect additivity (multiple regression)
Normality of the residuals
Homoscedasticity of the residuals
Independence of the residuals

8. Additivity Additivity assumption.
“The expected value of an observation is
a weighted linear combination of a number of factors.”
Which factors? (model uncertainty)
number of factors in the model
interactions of factors
powers or transformations of factors

9. Homoscedasticity and Normality
Observations never equal
their expected values.
No systematic biases.
Homoscedasticity assumption.
The unexplained component
has a common variance
for all values i.
Normality assumption.
The unexplained
component has a
normal distribution.

10. Independence
Independence assumption.
Responses in one experimental unit are not correlated with, affected by, or related to, responses for other experimental units.

11. Correlation Coefficient
A measure of the strength of the linear relationship between two variables.
Product Moment
Correlation Coefficient
In SLR, r is related to the slope of the fitted regression equation.
r2 (or R2) represents that proportion of total variability of the Y-values that is accounted for by the linear regression with the independent variable X.
R2: Proportion of variability in Y explained by X.

12. Properties of r 1. r lies between -1 and +1.
r > 0 indicates a positive
linear relationship.
r < 0 indicates a negative
r = 0 indicates no linear relationship.
r = 1 indicates perfect linear relationship.
2. The larger the absolute value of r, the stronger the linear relationship.
3. r2 also lies between 0 and 1.

13. Checking Assumptions How well does the model fit?
Do predicted values seem to be placed in the middle of observed values?
Do residuals satisfy the regression assumptions? (Problems seen in plot of X vs. Y will be reflected in residual plot.) y
Constant variance?
Regularities suggestive of lack of independence or more complex model?
Poorly fit observations? x

14. Studentized residuals (ei)
Model Adequacy
Studentized residuals (ei)
Allows us to gauge whether the residual is too large. It should have a standard normal distribution, hence it is very unlikely that any studentized residual will be outside the range [-3,3].
MSE(I) is the calculated MSE leaving observation i out of the computations.
hi is the ith diagonal of the projection matrix for the predictor space (ith hat diagonal element).

15. Normality of residuals
Kolmogorov-Smirnov Test
Shapiro-Wilks Test (n<50)
D’Agostino’s Test (n³50)
Formal Goodness of fit tests:
All quite conservative - they fail to reject the hypothesis of normality more often than they should.
Graphical Approach:
Quantile-quantile plot (qq-plot)
1. Compute and sort the simple residuals e[1],e[2],…e[n].
2. Associate to each residual a standard normal quantile
[z[i]=normsinv((i-.5)/n)].
3. Plot z[I] versus e[I]. Compare to 45o line.

17. Influence Diagnostics (Ways to detect influential observations)
Does a particular observation consisting of a pair of (X,Y) values (a case) have undue influence on the fit of the regression model? i.e. what cases are greatly affecting the estimates of the p regression parameters in the model. (For simple linear regression p=2.)
Standardized/Studentized Residuals. The ei are used to detect cases that are outlying with respect to their Y values. Check cases with
|ei| > 2 or 3.
Hat diagonal elements. The hi are used to detect cases that are outlying with respect to their X values. Check cases with
hi > 2p/n.

18. Dffits. Measures the influence that the ith case has on the ith fitted value. Compares the ith fitted value with the ith fitted value obtained by omitting the ith case. Check cases for which
|Dffits|>2Ö(p/n).
Cook’s Distance. Similar to Dffits, but considers instead the influence of the ith case on all n fitted values. Check when
Cook’s Dist > Fp,n-p,0.50.
Covariance Ratio. The change in the determinant of the covariance matrix that occurs when the ith case is deleted. Check cases with
|Cov Ratio  1| ³ 3p/n.
Dfbetas. A measure of the influence of the ith case on each estimated regression parameter. For each regression parameter, check cases with
|Dfbeta| > 2/Ön.

19. Cutoffs: Hat=0.29, CovRatio=0.43, Dffits=0.76, Dfbetas=0.53

21. Obs 5
Obs 1
Obs 2