Presentation is loading. Please wait.

Presentation is loading. Please wait.

12-1 Multiple Linear Regression Models Many applications of regression analysis involve situations in which there are more than one regressor variable.

Similar presentations


Presentation on theme: "12-1 Multiple Linear Regression Models Many applications of regression analysis involve situations in which there are more than one regressor variable."— Presentation transcript:

1

2

3

4 12-1 Multiple Linear Regression Models Many applications of regression analysis involve situations in which there are more than one regressor variable. A regression model that contains more than one regressor variable is called a multiple regression model. 12-1.1 Introduction

5 12-1 Multiple Linear Regression Models For example, suppose that the effective life of a cutting tool depends on the cutting speed and the tool angle. A possible multiple regression model could be where Y – tool life x 1 – cutting speed x 2 – tool angle 12-1.1 Introduction

6 12-1 Multiple Linear Regression Models Figure 12-1 (a) The regression plane for the model E(Y) = 50 + 10x 1 + 7x 2. (b) The contour plot 12-1.1 Introduction

7 12-1 Multiple Linear Regression Models 12-1.1 Introduction

8 12-1 Multiple Linear Regression Models Figure 12-2 (a) Three-dimensional plot of the regression model E(Y) = 50 + 10x 1 + 7x 2 + 5x 1 x 2. (b) The contour plot 12-1.1 Introduction

9 12-1 Multiple Linear Regression Models Figure 12-3 (a) Three-dimensional plot of the regression model E(Y) = 800 + 10x 1 + 7x 2 – 8.5x 1 2 – 5x 2 2 + 4x 1 x 2. (b) The contour plot 12-1.1 Introduction

10 12-1 Multiple Linear Regression Models 12-1.2 Least Squares Estimation of the Parameters

11 12-1 Multiple Linear Regression Models 12-1.2 Least Squares Estimation of the Parameters The least squares function is given by The least squares estimates must satisfy

12 12-1 Multiple Linear Regression Models 12-1.2 Least Squares Estimation of the Parameters The solution to the normal Equations are the least squares estimators of the regression coefficients. The least squares normal Equations are

13 12-1 Multiple Linear Regression Models Example 12-1

14 12-1 Multiple Linear Regression Models Example 12-1

15 12-1 Multiple Linear Regression Models Figure 12-4 Matrix of scatter plots (from Minitab) for the wire bond pull strength data in Table 12-2.

16 12-1 Multiple Linear Regression Models Example 12-1

17 12-1 Multiple Linear Regression Models Example 12-1

18 12-1 Multiple Linear Regression Models Example 12-1

19 12-1 Multiple Linear Regression Models 12-1.3 Matrix Approach to Multiple Linear Regression Suppose the model relating the regressors to the response is In matrix notation this model can be written as

20 12-1 Multiple Linear Regression Models 12-1.3 Matrix Approach to Multiple Linear Regression where

21 12-1 Multiple Linear Regression Models 12-1.3 Matrix Approach to Multiple Linear Regression We wish to find the vector of least squares estimators that minimizes: The resulting least squares estimate is

22 12-1 Multiple Linear Regression Models 12-1.3 Matrix Approach to Multiple Linear Regression

23 12-1 Multiple Linear Regression Models Example 12-2

24

25 12-1 Multiple Linear Regression Models Example 12-2

26 12-1 Multiple Linear Regression Models Example 12-2

27 12-1 Multiple Linear Regression Models Example 12-2

28 12-1 Multiple Linear Regression Models Example 12-2

29

30 12-1 Multiple Linear Regression Models Estimating  2 An unbiased estimator of  2 is

31 12-1 Multiple Linear Regression Models 12-1.4 Properties of the Least Squares Estimators Unbiased estimators: Covariance Matrix:

32 12-1 Multiple Linear Regression Models 12-1.4 Properties of the Least Squares Estimators Individual variances and covariances: In general,

33 12-2 Hypothesis Tests in Multiple Linear Regression 12-2.1 Test for Significance of Regression The appropriate hypotheses are The test statistic is

34 12-2 Hypothesis Tests in Multiple Linear Regression 12-2.1 Test for Significance of Regression

35 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-3

36 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-3

37 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-3

38 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-3

39 12-2 Hypothesis Tests in Multiple Linear Regression R 2 and Adjusted R 2 The coefficient of multiple determination For the wire bond pull strength data, we find that R 2 = SS R /SS T = 5990.7712/6105.9447 = 0.9811. Thus, the model accounts for about 98% of the variability in the pull strength response.

40 12-2 Hypothesis Tests in Multiple Linear Regression R 2 and Adjusted R 2 The adjusted R 2 is The adjusted R 2 statistic penalizes the analyst for adding terms to the model. It can help guard against overfitting (including regressors that are not really useful)

41 12-2 Hypothesis Tests in Multiple Linear Regression 12-2.2 Tests on Individual Regression Coefficients and Subsets of Coefficients The hypotheses for testing the significance of any individual regression coefficient:

42 12-2 Hypothesis Tests in Multiple Linear Regression 12-2.2 Tests on Individual Regression Coefficients and Subsets of Coefficients The test statistic is Reject H 0 if |t 0 | > t  /2,n-p. This is called a partial or marginal test

43 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-4

44 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-4

45 12-2 Hypothesis Tests in Multiple Linear Regression The general regression significance test or the extra sum of squares method: We wish to test the hypotheses:

46 12-2 Hypothesis Tests in Multiple Linear Regression A general form of the model can be written: where X 1 represents the columns of X associated with  1 and X 2 represents the columns of X associated with  2

47 12-2 Hypothesis Tests in Multiple Linear Regression For the full model: If H 0 is true, the reduced model is

48 12-2 Hypothesis Tests in Multiple Linear Regression The test statistic is: Reject H 0 if f 0 > f ,r,n-p The test in Equation (12-32) is often referred to as a partial F-test

49 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-5

50 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-5

51 12-2 Hypothesis Tests in Multiple Linear Regression Example 12-5

52 12-3 Confidence Intervals in Multiple Linear Regression 12-3.1 Confidence Intervals on Individual Regression Coefficients Definition

53 12-3 Confidence Intervals in Multiple Linear Regression Example 12-6

54 12-3 Confidence Intervals in Multiple Linear Regression 12-3.2 Confidence Interval on the Mean Response The mean response at a point x 0 is estimated by The variance of the estimated mean response is

55 12-3 Confidence Intervals in Multiple Linear Regression 12-3.2 Confidence Interval on the Mean Response Definition

56 12-3 Confidence Intervals in Multiple Linear Regression Example 12-7

57 12-3 Confidence Intervals in Multiple Linear Regression Example 12-7

58 12-4 Prediction of New Observations A point estimate of the future observation Y 0 is A 100(1-  )% prediction interval for this future observation is

59 12-4 Prediction of New Observations Figure 12-5 An example of extrapolation in multiple regression

60 12-4 Prediction of New Observations Example 12-8

61 12-5 Model Adequacy Checking 12-5.1 Residual Analysis Example 12-9 Figure 12-6 Normal probability plot of residuals

62 12-5 Model Adequacy Checking 12-5.1 Residual Analysis Example 12-9

63 12-5 Model Adequacy Checking 12-5.1 Residual Analysis Example 12-9 Figure 12-7 Plot of residuals against ŷ i.

64 12-5 Model Adequacy Checking 12-5.1 Residual Analysis Example 12-9 Figure 12-8 Plot of residuals against x 1.

65 12-5 Model Adequacy Checking 12-5.1 Residual Analysis Example 12-9 Figure 12-9 Plot of residuals against x 2.

66 12-5 Model Adequacy Checking 12-5.1 Residual Analysis

67 12-5 Model Adequacy Checking 12-5.1 Residual Analysis The variance of the ith residual is

68 12-5 Model Adequacy Checking 12-5.1 Residual Analysis

69 12-5 Model Adequacy Checking 12-5.2 Influential Observations Figure 12-10 A point that is remote in x- space.

70 12-5 Model Adequacy Checking 12-5.2 Influential Observations Cook’s distance measure

71 12-5 Model Adequacy Checking Example 12-10

72 12-5 Model Adequacy Checking Example 12-10

73 12-6 Aspects of Multiple Regression Modeling 12-6.1 Polynomial Regression Models

74 12-6 Aspects of Multiple Regression Modeling Example 12-11

75 12-6 Aspects of Multiple Regression Modeling Example 12-11 Figure 12-11 Data for Example 12-11.

76 Example 12-11

77 12-6 Aspects of Multiple Regression Modeling Example 12-11

78 12-6 Aspects of Multiple Regression Modeling 12-6.2 Categorical Regressors and Indicator Variables Many problems may involve qualitative or categorical variables. The usual method for the different levels of a qualitative variable is to use indicator variables. For example, to introduce the effect of two different operators into a regression model, we could define an indicator variable as follows:

79 12-6 Aspects of Multiple Regression Modeling Example 12-12

80 12-6 Aspects of Multiple Regression Modeling Example 12-12

81 12-6 Aspects of Multiple Regression Modeling Example 12-12

82

83 12-6 Aspects of Multiple Regression Modeling Example 12-12

84 12-6 Aspects of Multiple Regression Modeling Example 12-12

85 12-6 Aspects of Multiple Regression Modeling 12-6.3 Selection of Variables and Model Building

86 12-6 Aspects of Multiple Regression Modeling 12-6.3 Selection of Variables and Model Building All Possible Regressions – Example 12-13

87 12-6 Aspects of Multiple Regression Modeling 12-6.3 Selection of Variables and Model Building All Possible Regressions – Example 12-13

88 12-6 Aspects of Multiple Regression Modeling 12-6.3 Selection of Variables and Model Building All Possible Regressions – Example 12-13 Figure 12-12 A matrix of Scatter plots from Minitab for the Wine Quality Data.

89

90 12-6.3 Selection of Variables and Model Building Stepwise Regression Example 12-13

91 12-6.3 Selection of Variables and Model Building Backward Regression Example 12-13

92 12-6 Aspects of Multiple Regression Modeling 12-6.4 Multicollinearity Variance Inflation Factor (VIF)

93 12-6 Aspects of Multiple Regression Modeling 12-6.4 Multicollinearity The presence of multicollinearity can be detected in several ways. Two of the more easily understood of these are:

94


Download ppt "12-1 Multiple Linear Regression Models Many applications of regression analysis involve situations in which there are more than one regressor variable."

Similar presentations


Ads by Google