1. Chapter 13 Multiple Regression

2. Chapter Outline Multiple Regression Model Least Squares Method
Coefficient of Determination
Model Assumptions
Testing for Significance
Estimation and Prediction
Categorical Independent Variables

3. Multiple Regression Model
The equation that describes how the dependent variable y is related to the independent variables x1, x2, xp and an error term is:
y = b0 + b1x1 + b2x bpxp + e
where:
b0, b1, b2, , bp are the parameters, and
e is a random variable called the error term

4. Multiple Regression Equation
Multiple regression equation is:
The equation that describes how the mean value of y is related to x1, x2, xp is:
E(y) = 0 + 1x1 + 2x pxp

5. Estimated Multiple Regression Equation^
y = b0 + b1x1 + b2x bpxp
A simple random sample is used to compute sample statistics b0, b1, b2, , bp that are used as the point estimates of the parameters b0, b1, b2, , bp.

6. Estimation Process E(y) = 0 + 1x1 + 2x2 +. . .+ pxp + e
Multiple Regression Model
E(y) = 0 + 1x1 + 2x pxp + e
Multiple Regression Equation
E(y) = 0 + 1x1 + 2x pxp
Unknown parameters are
b0, b1, b2, , bp
Sample Data:
x1 x xp y
Estimated Multiple
Regression Equation
Sample statistics are
b0, b1, b2, , bp
b0, b1, b2, , bp
provide estimates of

7. Least Squares Method Least Squares Criterion
Computation of Coefficient Values:
The formulas for the regression coefficients
b0, b1, b2, bp involve the use of matrix algebra.
We will rely on computer software packages to
perform the calculations.

8. Example: Employee Salary Survey
Multiple Regression
Example: Employee Salary Survey
A local firm collected data for a sample of 20 employees. A suggestion was made that regression analysis could be used to determine if salary was related to the years of experience and the score on the firm’s aptitude test.
The gender of employees, years of experience, score on the aptitude test, and corresponding annual salary ($1000s) for a sample of 20 employees is shown on the next slide.

9. Multiple Regression
Example: Employee Salary Survey (data)

10. Multiple Regression Model
Suppose we believe that salary (y) is related to
the years of experience (x1) and the score on the
aptitude test (x2) by the following regression model:
y = 0 + 1x1 + 2x2 + 
where
y = annual salary ($1000)
x1 = years of experience
x2 = score on aptitude test

11. Solving for the Estimates of 0, 1, 2
Least Squares
Output
Input Data
x1 x2 y
Computer
Package
for Solving
Multiple
Regression
Problems
b0 =
b1 =
b2 =
R2 =
etc.

12. Solving for the Estimates of 0, 1, 2
Excel’s Regression Output – Parameter Estimates
Note: All the numbers are rounded to the 4th decimal point.

13. Estimated Regression Equation
SALARY = (YEARS) (SCORE)
Note: Predicted salary will be in thousands of dollars.

14. Interpreting the Coefficients
In multiple regression analysis, we interpret each
regression coefficient as follows:
bi represents an estimate of the change in y
corresponding to a 1-unit increase in xi when all
other independent variables are held constant.

15. Interpreting the Coefficients
b1 = 1.404
Salary is expected to increase by $1,404 for
each additional year of experience (when the variable
score on attitude test is held constant).

16. Interpreting the Coefficients
b2 = 0.251
Salary is expected to increase by $251 for each
additional point scored on the aptitude test (when the variable years of experience is held constant).

17. Multiple Coefficient of Determination
Relationship Among SST, SSR, SSE
SST = SSR SSE
where:
SST = total sum of squares (i.e. total variability
of y)
SSR = sum of squares due to regression (i.e. the
variability of y that is explained by regression)
SSE = sum of squares due to error (i.e. the variability
of y that cannot be explained by regression)

18. Multiple Coefficient of Determination
Excel’s ANOVA Output
SST
SSR

19. Multiple Coefficient of Determination
r2 = SSR/SST = / =
The regression relationship is strong. About
83.4% of the variability in the salary of employees can be
explained by the years of experience and the aptitude
score.

20. Adjusted Multiple Coefficient of Determination
Note: p is the number of slope coefficients.

21. Assumptions About the Error Term e
y = b0 + b1x1 + b2x2 + … + bpxp +e
1. The error  is a random variable with mean of zero.
2. The variance of  , denoted by  2, is the same for
all values of the independent variable.
3. The values of  are independent.
4. The error  is a normally distributed random variable
reflecting the deviation between the y value and the
expected value of y given by 0 + 1x1 + 2x pxp.

22. Testing for Significance
In simple linear regression, the F and t tests provide
the same conclusion.
In multiple regression, the F and t tests have different
purposes.

23. Testing for Significance
The F test is used to test the overall significance of a
regression model.
The t test is used to test the individual significance, i.e.
whether each of the individual independent variables
is significant.

24. Testing for Significance: F Test
H0: 1 = 2 = = p = 0
Ha: One or more of the parameters
is not equal to zero.
Hypotheses
Test Statistics
F = MSR/MSE
Rejection Rule
Reject H0 if p-value < a or if F > F ,
where F is based on an F distribution
with p d.f. in the numerator and
n - p - 1 d.f. in the denominator.

25. F Test: Employee Salary Survey
H0: 1 = 2 = 0
Ha: One or both of the parameters
is not equal to zero.
Hypotheses
Test Statistics
F = MSR/MSE
= /5.85 = 42.76
For  = .05 and d.f. = (2, 17); F.05 = 3.59
Reject H0 if p-value < .05 or F > 3.59
Rejection Rule

26. F Test: Employee Salary Survey
p-value
Conclusion
p-value < .05, so we can reject H0.
(Also, F = > 3.59)

27. Testing for Significance: t Test
Hypotheses
Test Statistics
Rejection Rule
Reject H0 if p-value < a or
if t < -tor t > t where t
is based on a t distribution
with n - p - 1 degrees of freedom.

28. t Test: Employee Salary Survey
Hypotheses
Test Statistics
Rejection Rule
For  = .05 and d.f. = 17, t.025 = 2.11
Reject H0 if p-value < .05, or
if t < or t > 2.11

29. t Test: Employee Salary Survey
Conclusions
Reject both H0: 1 = 0 and H0: 2 = 0.
Both independent variables are
significant.

30. Test for Significance: Multicollinearity
The term multicollinearity refers to the correlation
among the independent variables.
When the independent variables are highly correlated
(say, |r | > .7), it is not possible to determine the
separate effect of any particular independent variable
on the dependent variable.

31. Using the Estimated Regression Equation for Estimation and Prediction
The procedures for estimating the mean value of y
and predicting an individual value of y in multiple
regression are similar to those in simple regression.
We substitute the given values of x1, x2, , xp into
the estimated regression equation and use the
corresponding value of y as the point estimate.

32. Categorical Independent Variables
In many situations we must work with categorical
independent variables such as gender (male, female),
method of payment (cash, check, credit card), etc.
For example, xi might represent gender where xi = 0
indicates male and xi = 1 indicates female.
In this case, xi is called a dummy or indicator variable.

33. Categorical Independent Variables
Example: Employee Salary Survey
As an extension of the problem involving the employee salary survey, suppose that management wants to find out if the annual salary is related to employees’ gender.
The years of experience, the score on the aptitude test, employees’ gender, and the annual salary ($000) for each of the sampled 20 employees are shown on the next slide.

34. Estimated Regression Equation
y = b0 + b1x1 + b2x2 + b3x3 ^ ^
where:
y = annual salary ($1000)
x1 = years of experience
x2 = score on aptitude test
x3 = 0 if an employee is female;
1 if an employee is male.
x3 is a dummy variable

35. Categorical Independent Variables

36. Categorical Independent Variables
Excel’s Regression Statistics

37. Categorical Independent Variables
Excel’s ANOVA Output

38. Categorical Independent Variables
Excel’s Regression Equation Output
Not significant