1. Korelasi Parsial dan Pengontrolan Parsial Pertemuan 14
Matakuliah : I0174 – Analisis Regresi
Tahun : Ganjil 2007/2008
Korelasi Parsial dan Pengontrolan Parsial Pertemuan 14

2. Chapter Topics The Multiple Regression Model Residual Analysis
Testing for the Significance of the Regression Model
Inferences on the Population Regression Coefficients
Testing Portions of the Multiple Regression Model
Dummy-Variables and Interaction Terms
Bina Nusantara

3. The Multiple Regression Model
Relationship between 1 dependent & 2 or more independent variables is a linear function
Population Y-intercept
Population slopes
Random error
Dependent (Response) variable
Independent (Explanatory) variables
Bina Nusantara

4. Multiple Regression Model
Bivariate model
Bina Nusantara

5. Multiple Regression Equation
Bivariate model
Multiple Regression Equation
Bina Nusantara

6. Multiple Regression Equation
Too complicated by hand!
Ouch!
Bina Nusantara

7. Interpretation of Estimated Coefficients
Slope (bj )
Estimated that the average value of Y changes by bj for each 1 unit increase in Xj , holding all other variables constant (ceterus paribus)
Example: If b1 = -2, then fuel oil usage (Y) is expected to decrease by an estimated 2 gallons for each 1 degree increase in temperature (X1), given the inches of insulation (X2)
Y-Intercept (b0)
The estimated average value of Y when all Xj = 0
Bina Nusantara

8. Multiple Regression Model: Example
(0F)
Develop a model for estimating heating oil used for a single family home in the month of January, based on average temperature and amount of insulation in inches.
Bina Nusantara

9. Multiple Regression Equation: Example
Excel Output
For each degree increase in temperature, the estimated average amount of heating oil used is decreased by gallons, holding insulation constant.
For each increase in one inch of insulation, the estimated average use of heating oil is decreased by gallons, holding temperature constant.
Bina Nusantara

10. Multiple Regression in PHStat
PHStat | Regression | Multiple Regression …
Excel spreadsheet for the heating oil example
Bina Nusantara

11. Venn Diagrams and Explanatory Power of Regression
Variations in Oil explained by the error term
Variations in Temp not used in explaining variation in Oil
Oil
Variations in Oil explained by Temp or variations in Temp used in explaining variation in Oil
Temp
Bina Nusantara

12. Venn Diagrams and Explanatory Power of Regression
(continued)
Oil
Temp
Bina Nusantara

13. Venn Diagrams and Explanatory Power of Regression
Variation NOT explained by Temp nor Insulation
Overlapping variation in both Temp and Insulation are used in explaining the variation in Oil but NOT in the estimation of nor
Oil
Temp
Insulation
Bina Nusantara

14. Coefficient of Multiple Determination
Proportion of Total Variation in Y Explained by All X Variables Taken Together
Never Decreases When a New X Variable is Added to Model
Disadvantage when comparing among models
Bina Nusantara

15. Venn Diagrams and Explanatory Power of Regression
Oil
Temp
Insulation
Bina Nusantara

16. Adjusted Coefficient of Multiple Determination
Proportion of Variation in Y Explained by All the X Variables Adjusted for the Sample Size and the Number of X Variables Used
Penalizes excessive use of independent variables
Smaller than
Useful in comparing among models
Can decrease if an insignificant new X variable is added to the model
Bina Nusantara

17. Coefficient of Multiple Determination
Excel Output
Adjusted r2
reflects the number of explanatory variables and sample size
is smaller than r2
Bina Nusantara

18. Interpretation of Coefficient of Multiple Determination
96.56% of the total variation in heating oil can be explained by temperature and amount of insulation
95.99% of the total fluctuation in heating oil can be explained by temperature and amount of insulation after adjusting for the number of explanatory variables and sample size
Bina Nusantara

19. Simple and Multiple Regression Compared
The slope coefficient in a simple regression picks up the impact of the independent variable plus the impacts of other variables that are excluded from the model, but are correlated with the included independent variable and the dependent variable
Coefficients in a multiple regression net out the impacts of other variables in the equation
Hence, they are called the net regression coefficients
They still pick up the effects of other variables that are excluded from the model, but are correlated with the included independent variables and the dependent variable
Bina Nusantara

20. Simple and Multiple Regression Compared: Example
Two Simple Regressions:
Multiple Regression:
Bina Nusantara

21. Simple and Multiple Regression Compared: Slope Coefficients
Bina Nusantara

22. Simple and Multiple Regression Compared: r2=
Bina Nusantara

23. Example: Adjusted r2 Can Decrease
Adjusted r 2 decreases when
k increases from 2 to 3
Color is not useful in explaining the variation in oil consumption.
Bina Nusantara

24. Using the Regression Equation to Make Predictions
Predict the amount of heating oil used for a home if the average temperature is 300 and the insulation is 6 inches.
The predicted heating oil used is gallons.
Bina Nusantara

25. Predictions in PHStat PHStat | Regression | Multiple Regression …
Check the “Confidence and Prediction Interval Estimate” box
Excel spreadsheet for the heating oil example
Bina Nusantara

26. Residual Plots Residuals Vs Residuals Vs Time
May need to transform Y variable
May need to transform variable
May need to transform variable
Residuals Vs Time
May have autocorrelation
Bina Nusantara

27. Residual Plots: Example
Maybe some non-linear relationship
No Discernable Pattern
Bina Nusantara

28. Testing for Overall Significance
Shows if Y Depends Linearly on All of the X Variables Together as a Group
Use F Test Statistic
Hypotheses:
H0: 1 = 2 = … = k = 0 (No linear relationship)
H1: At least one i  0 ( At least one independent variable affects Y )
The Null Hypothesis is a Very Strong Statement
The Null Hypothesis is Almost Always Rejected
Bina Nusantara

29. Testing for Overall Significance
(continued)
Test Statistic:
Where F has k numerator and (n-k-1) denominator degrees of freedom
Bina Nusantara

30. Test for Overall Significance Excel Output: Example
p-value
k = 2, the number of explanatory variables
n - 1
Bina Nusantara

31. Test for Overall Significance: Example Solution
H0: 1 = 2 = … = k = 0
H1: At least one j  0
 = .05
df = 2 and 12
Critical Value:
Test Statistic:
Decision:
Conclusion:  F
168.47
(Excel Output)
Reject at  = 0.05.
 = 0.05
There is evidence that at least one independent variable affects Y. F
3.89
Bina Nusantara

32. Test for Significance: Individual Variables
Show If Y Depends Linearly on a Single Xj Individually While Holding the Effects of Other X’s Fixed
Use t Test Statistic
Hypotheses:
H0: j = 0 (No linear relationship)
H1: j  0 (Linear relationship between Xj and Y)
Bina Nusantara

33. t Test Statistic Excel Output: Example
t Test Statistic for X1 (Temperature)
t Test Statistic for X2 (Insulation)
Bina Nusantara

34. t Test : Example Solution
Does temperature have a significant effect on monthly consumption of heating oil? Test at  = 0.05.
Test Statistic:
Decision:
Conclusion:
H0: 1 = 0
H1: 1  0
df = 12
Critical Values:
t Test Statistic =
Reject H0 at  = 0.05.
Reject H
Reject H
There is evidence of a significant effect of temperature on oil consumption holding constant the effect of insulation.
.025
.025 t
2.1788
Bina Nusantara

35. Venn Diagrams and Estimation of Regression Model
Only this information is used in the estimation of
Only this information is used in the estimation of
Oil
This information is NOT used in the estimation of nor
Temp
Insulation
Bina Nusantara

36. Confidence Interval Estimate for the Slope
Provide the 95% confidence interval for the population slope 1 (the effect of temperature on oil consumption).
 1 
We are 95% confident that the estimated average consumption of oil is reduced by between 4.7 gallons to 6.17 gallons per each increase of 10 F holding insulation constant.
We can also perform the test for the significance of individual variables, H0: 1 = 0 vs. H1: 1  0, using this confidence interval.
Bina Nusantara

37. Contribution of a Single Independent Variable
Let Xj Be the Independent Variable of Interest
Measures the additional contribution of Xj in explaining the total variation in Y with the inclusion of all the remaining independent variables
Bina Nusantara

38. Contribution of a Single Independent Variable
From ANOVA section of regression for
From ANOVA section of regression for
Measures the additional contribution of X1 in explaining Y with the inclusion of X2 and X3.
Bina Nusantara

39. Coefficient of Partial Determination of
Measures the proportion of variation in the dependent variable that is explained by Xj while controlling for (holding constant) the other independent variables
Bina Nusantara

40. Coefficient of Partial Determination for
(continued)
Example: Model with two independent variables
Bina Nusantara

41. Venn Diagrams and Coefficient of Partial Determination for
Oil =
Temp
Insulation
Bina Nusantara

42. Coefficient of Partial Determination in PHStat
PHStat | Regression | Multiple Regression …
Check the “Coefficient of Partial Determination” box
Excel spreadsheet for the heating oil example
Bina Nusantara

43. Contribution of a Subset of Independent Variables
Let Xs Be the Subset of Independent Variables of Interest
Measures the contribution of the subset Xs in explaining SST with the inclusion of the remaining independent variables
Bina Nusantara

44. Contribution of a Subset of Independent Variables: Example
Let Xs be X1 and X3
From ANOVA section of regression for
From ANOVA section of regression for
Bina Nusantara

45. Testing Portions of Model
Examines the Contribution of a Subset Xs of Explanatory Variables to the Relationship with Y
Null Hypothesis:
Variables in the subset do not improve the model significantly when all other variables are included
Alternative Hypothesis:
At least one variable in the subset is significant when all other variables are included
Bina Nusantara

46. Testing Portions of Model
(continued)
One-Tailed Rejection Region
Requires Comparison of Two Regressions
One regression includes everything
Another regression includes everything except the portion to be tested
Bina Nusantara

47. Partial F Test for the Contribution of a Subset of X Variables
Hypotheses:
H0 : Variables Xs do not significantly improve the model given all other variables included
H1 : Variables Xs significantly improve the model given all others included
Test Statistic:
with df = m and (n-k-1)
m = # of variables in the subset Xs
Bina Nusantara

48. Partial F Test for the Contribution of a Single
Hypotheses:
H0 : Variable Xj does not significantly improve the model given all others included
H1 : Variable Xj significantly improves the model given all others included
Test Statistic:
with df = 1 and (n-k-1 )
m = 1 here
Bina Nusantara

49. Testing Portions of Model: Example
Test at the  = .05 level to determine if the variable of average temperature significantly improves the model, given that insulation is included.
Bina Nusantara

50. Testing Portions of Model: Example
H0: X1 (temperature) does not improve model with X2 (insulation) included
H1: X1 does improve model
 = .05, df = 1 and 12
Critical Value = 4.75
(For X1 and X2)
(For X2)
Conclusion: Reject H0; X1 does improve model.
Bina Nusantara

51. Testing Portions of Model in PHStat
PHStat | Regression | Multiple Regression …
Check the “Coefficient of Partial Determination” box
Excel spreadsheet for the heating oil example
Bina Nusantara

52. Do We Need to Do This for One Variable?
The F Test for the Contribution of a Single Variable After All Other Variables are Included in the Model is IDENTICAL to the t Test of the Slope for that Variable
The Only Reason to Perform an F Test is to Test Several Variables Together
Bina Nusantara

53. Dummy-Variable Models
Categorical Explanatory Variable with 2 or More Levels
Yes or No, On or Off, Male or Female,
Use Dummy-Variables (Coded as 0 or 1)
Only Intercepts are Different
Assumes Equal Slopes Across Categories
The Number of Dummy-Variables Needed is (# of Levels - 1)
Regression Model Has Same Form:
Bina Nusantara

54. Dummy-Variable Models (with 2 Levels)
Given:
Y = Assessed Value of House
X1 = Square Footage of House
X2 = Desirability of Neighborhood =
Desirable (X2 = 1)
Undesirable (X2 = 0)
0 if undesirable if desirable
Same slopes
Bina Nusantara

55. Dummy-Variable Models (with 2 Levels)
(continued)
Y (Assessed Value)
Same slopes
Desirable Location
b0 + b2
Undesirable
Intercepts different b0
X1 (Square footage)
Bina Nusantara

56. Interpretation of the Dummy-Variable Coefficient (with 2 Levels)
Example:
: Annual salary of college graduate in thousand $
0 non-business degree
: GPA :
1 business degree
With the same GPA, college graduates with a business degree are making an estimated 6 thousand dollars more than graduates with a non-business degree, on average.
Bina Nusantara

57. Dummy-Variable Models (with 3 Levels)
Bina Nusantara

58. Interpretation of the Dummy-Variable Coefficients (with 3 Levels)
With the same footage, a Split-level will have an estimated average assessed value of thousand dollars more than a Condo.
With the same footage, a Ranch will have an estimated average assessed value of thousand dollars more than a Condo.
Bina Nusantara

59. Regression Model Containing an Interaction Term
Hypothesizes Interaction between a Pair of X Variables
Response to one X variable varies at different levels of another X variable
Contains a Cross-Product Term
Can Be Combined with Other Models
E.g., Dummy-Variable Model
Bina Nusantara

60. Effect of Interaction Given:
Without Interaction Term, Effect of X1 on Y is Measured by 1
With Interaction Term, Effect of X1 on Y is Measured by 1 + 3 X2
Effect Changes as X2 Changes
Bina Nusantara

61. Effect (slope) of X1 on Y depends on X2 value
Interaction Example
Y = 1 + 2X1 + 3X2 + 4X1X2 Y
Y = 1 + 2X1 + 3(1) + 4X1(1) = 4 + 6X1 12 8
Y = 1 + 2X1 + 3(0) + 4X1(0) = 1 + 2X1 4 X1
0.5 1
1.5
Effect (slope) of X1 on Y depends on X2 value
Bina Nusantara

62. Interaction Regression Model Worksheet
Case, i Yi
X1i
X2i
X1i X2i 1 3 2 4 8 5 40 6 30 :
Multiply X1 by X2 to get X1X2 Run regression with Y, X1, X2 , X1X2
Bina Nusantara

63. Interpretation When There Are 3+ Levels
MALE = 0 if female and 1 if male
MARRIED = 1 if married; 0 if not
DIVORCED = 1 if divorced; 0 if not
MALE•MARRIED = 1 if male married; 0 otherwise = (MALE times MARRIED)
MALE•DIVORCED = 1 if male divorced; 0 otherwise = (MALE times DIVORCED)
Bina Nusantara

64. Interpretation When There Are 3+ Levels
(continued)
Bina Nusantara

65. Interpreting Results MALE Single: Married: Divorced: Difference
FEMALE
Single:
Married:
Divorced:
MALE
Single:
Married:
Divorced:
Difference
Main Effects : MALE, MARRIED and DIVORCED
Interaction Effects : MALE•MARRIED and MALE•DIVORCED
Bina Nusantara

66. Evaluating the Presence of Interaction with Dummy-Variable
Suppose X1 and X2 are Numerical Variables and X3 is a Dummy-Variable
To Test if the Slope of Y with X1 and/or X2 are the Same for the Two Levels of X3
Model:
Hypotheses:
H0: 4 = 5 = 0 (No Interaction between X1 and X3 or X2 and X3 )
H1: 4 and/or 5  0 (X1 and/or X2 Interacts with X3)
Perform a Partial F Test
Bina Nusantara

67. Evaluating the Presence of Interaction with Numerical Variables
Suppose X1, X2 and X3 are Numerical Variables
To Test If the Independent Variables Interact with Each Other
Model:
Hypotheses:
H0: 4 = 5 = 6 = 0 (no interaction among X1, X2 and X3 )
H1: at least one of 4, 5, 6  0 (at least one pair of X1, X2, X3 interact with each other)
Perform a Partial F Test
Bina Nusantara

68. Chapter Summary Developed the Multiple Regression Model
Discussed Residual Plots
Addressed Testing the Significance of the Multiple Regression Model
Discussed Inferences on Population Regression Coefficients
Addressed Testing Portions of the Multiple Regression Model
Discussed Dummy-Variables and Interaction Terms
Bina Nusantara