Residual Analysis

Purposes –Examine Functional Form (Linear vs. Non- Linear Model) –Evaluate Violations of Assumptions Graphical Analysis of Residuals Residual Analysis

(X 1,  Y1 ) For one value X 1, a population contains may Y values. Their mean is  Y1. X1X1 Y

Y X A Population Regression Line  Y =  X

Y x A Sample Regression Line The sample line approximates the population regression line. y = a + bx

Histogram of Y Values at X = X 1 Y f(e) X X 1  Y =  X  Y1 =  X 1

Normal Distribution of Y Values when X = X 1 Y f(e) X X 1  Y1 =  X 1  Y =  X The standard deviation of the normal distribution is the standard error of estimate.

Normality & Constant Variance Assumptions Y f(e) X X 1 X 2

A Normal Regression Surface Y f(e) X X 1 X 2 Every cross-sectional slice of the surface is a normal curve.

Analysis of Residuals A residual is the difference between the actual value of Y and the predicted value.

Linear Regression and Correlation Assumptions The independent variables and the dependent variable have a linear relationship. The dependent variable must be continuous and at least interval-scale.

Linear Regression Assumptions Normality  Y Values Are Normally Distributed with a mean of Zero For Each X.   he  residuals (  )  are normally distributed with a mean of Zero. Homoscedasticity (Constant Variance)  The variation in the residuals must be the same for all values of Y.  The standard deviation of the residuals is the same regardless of the given value of X. Independence of Errors  The residuals are independent for each value of X  The residuals (  ) are independent of each other  The size of the error for a particular value of x is not related to the size of the error for any other value of x

Evaluating the Aptness of the Fitted Regression Model Does the model appear linear?

Residual Plot for Linearity (Functional Form) Aptness of the Fitted Model Correct Specification X e Add X 2 Term X e

Residual Plots for Linearity of the Fitted Model Scatter Plot of Y vs. X value Scatter Plot of residuals vs. X value

Using SPSS to Test for Linearity of the Regression Model Analyze/Regression/Linear –Dependent - Sales –Independent - Customers –Save Predicted Value (Unstandardized or Standardized) Residual (Unstandardizedor Standardized) Graphs/Scatter/Simple Y-Axis: residual [ res_1 or zre_1 ] X-Axis: Customer (independent variable)

ELECTRONIC FIRMS

The Linear Regression Assumptions 1. Normality of residuals (Errors) 2. Homoscedasticity (Constant Variance) 3. Independence of Residuals (Errors) Need to verify using residual analysis.

Residual Plots for Normality Construct histogram of residuals –Stem-and-leaf plot –Box-and-whisker plot –Normal probability plot Scatter Plot residuals vs. X values –Simple regression Scatter Plot residuals vs. Y –Multiple regression

Residual Plot 1 for Normality Construct histogram of residuals Nearly symmetric Centered near or at zero Shape is approximately normal

Using SPSS to Test for Normality Histogram of Residuals Analyze/Regression/Linear –Dependent - Sales –Independent - Customers –Plot/Standardized Residual Plot: Histogram –Save Predicted Value (Unstandardized or Standardized) Residual (Unstandardizedor Standardized) Graphs/Histogram –Variable - residual (Unstandardized or Standardedized)

Histogram of Residuals of Sales and Customer Problem from regression output

Histogram of Residuals of Sales and Customer Problem from graph output

Residual Plot 2 for Normality Plot residuals vs. X values Points should be distributed about the horizontal line at 0 Otherwise, normality is violated X Residuals 0

Using SPSS to Test for Normality Scatter Plot Simple Regression –Graph/Scatter/Simple Y-Axis: residual [ res_1 or zre_1 ] X-Axis: Customers [independent variable ] Multiple Regression –Graph/Scatter/Simple Y-Axis: residual [ res_1 or zre_1 ] X-Axis: predicted Y values

An accounting standards board investigating the treatment of research and development expenses by the nation’s major electronic firms was interested in the relationship between a firm’s research and development expenditures and its earnings. The Electronic Firms Earnings = (rdexpend)

ELECTRONIC FIRMS List of Data, Predicted Values and Residuals Data Predicted Residual Standardized Standardized Value Predicted Value Residual

ELECTRONIC FIRMS

Residual Plot for Homoscedasticity Constant Variance Correct Specification X SR 0 Heteroscedasticity X SR 0 Fan-Shaped. Standardized Residuals Used.

Simple Regression –Graphs/Scatter/Simple Y-Axis: residual [ res_1 or zre_1 ] X-Axis: rdexpend[independent variable ] Multiple Regression –Graphs/Scatter/Simple Y-Axis: residual [ res_1 or zre_1 ] X-Axis: predicted Y values Using SPSS to Test for Homoscedasticity of Residuals

Test for Homoscedasticity DUNTON’S WORLD OF SOUND

Test for Homoscedasticity ELECTRONIC FIRMS

Residual Plot for Independence Correct Specification X SR Not Independent X SR Plots Reflect Sequence Data Were Collected.

Two Types of Autocorrelation Positive Autocorrelation: successive terms in time series are directly related Negative Autocorrelation: successive terms are inversely related

Residual y - y Time Period, t Positive autocorrelation: Residuals tend to be followed by residuals with the same sign

Residual y - y Time Period, t Negative Autocorrelation: Residuals tend to change signs from one period to the next

Problems with autocorrelated time-series data s y.x and s b are biased downwards Invalid probability statements about regression equation and slopes F and t tests won’t be valid May imply that cycles exist May induce a falsely high or low agreement between 2 variables

Using SPSS to Test for Independence of Errors Graphs/Sequence –Variables: residual (res_1) Durbin-Watson Statistic

DUNTON’S WORLD OF SOUND

ELECTRONIC FIRMS

CustomersSales($000) Customers and sales for period of 15 consecutive weeks.

Durbin-Watson Procedure Used to Detect Autocorrelation –Residuals in One Time Period Are Related to Residuals in Another Period –Violation of Independence Assumption Durbin-Watson Test Statistic D (ee e i i i n i i n        )

H 0 : No positive autocorrelation exists (residuals are random) H 1 : Positive autocorrelation exists Accept Ho if d> d u Reject Ho if d < d L Inconclusive if d L < d < d u d =

Testing for Positive Autocorrelation There is positive autocorrelation The test is inconclusive There is no evidence of autocorrelation 0dLdL dudu 2 4

Rule of Thumb Positive autocorrelation - D will approach 0 No autocorrelation - D will be close to 2 Negative autocorrelation - D is greater than 2 and may approach a maximum of 4

Using SPSS with Autocorrelation Analyze/Regression/Linear Dependent; Independent Statistics/Durbin-Watson (use only time series data)

CustomersSales($000) Customers and sales for period of 15 consecutive weeks.

Durbin-Watson.883

Using SPSS with Autocorrelation Analyze/Regression/Linear Dependent; Independent Statistics/ Durbin-Watson (use only time series data) If DW indicates autocorrelation, then … –Analyze/Time Series/Autoregression –Cochrane-Orcutt –OK

Solutions for autocorrelation Use Final Parameters under Cochrane-Orcutt Changes in the dependent and independent variables - first differences Transform the variables Include an independent variable that measures the time of the observation Use lagged variables (once lagged value of dependent variable is introduced as independent variable, Durbon-Watson test is not valid