Multiple Regression Analysis Multivariate Analysis

Multiple Regression Analysis Multivariate Analysis Statistical methods that allow the simultaneous investigation of more than two variables. Statistical methods that allow the simultaneous investigation of more than two variables. The techniques can be used to summarize data and reduce the number of variables necessary to describe the data. Several of the more common multivariate techniques: Multiple Regression Analysis Multiple Discriminate Analysis Factor Analysis Cluster Analysis Multidimensional Scaling

Regression Analysis Regression Analysis Regression Analysis is concerned with the dependence of one variable, the dependent variable, on one or more of other variables, the explanatory variables, with a view to estimating and/or predicting the mean value of the former in terms of known or fixed values of the later.

Multiple Regression Analysis  To determine the association or relationship between dependent and independent variables.  In multiple regression analysis, two or more independent variables are included in examinations.  The general form of the multiple regression model is as follows: Multiple Regression Analysis Where = Y intercept of regression model = Y intercept of regression model = Slope of the regression model = Slope of the regression model ε i = Random error ε i = Random error

Multiple Regression Analysis Partial Regression Coefficient Denotes the change in the computed value,, per one unit change in when all other independent variables are held constant.

Multiple Regression Analysis The % of the variance in the dependent variable that is explained by the variation in the independent variables. The % of the variance in the dependent variable that is explained by the variation in the independent variables. where TSS = total sum of squares = RSS = regression sum of squares = ESS = error sum of squares = Coefficient of Multiple Determination

Multiple Regression Analysis Adjusted Coefficient of Determination  The number of independent variables in a multiple regression equation makes the coefficient of determination larger. Each new independent variable causes the predictions to be more accurate.  To balance the effect that the number of independent variables has on the coefficient of multiple determination, we use an adjusted coefficient of multiple determination.

Multiple Regression Analysis Salsberry Realty sells homes along the east coast of the United States. One of the questions most frequently asked by prospective buyers is: If we purchase this home, how much can we expect to pay to heat it during the winter? The research department at Salsberry has been asked to develop some guidelines regarding heating costs for single-family homes. Three variables are thought to relate to the heating costs: (1) the mean daily outside temperature, (2) the number of inches of insulation in the attic, and (3) the age in years of the furnace. To investigate, Salsberry’s research department selected a random sample of 20 recently sold homes. It determined the cost to heat each home last January, as well Multiple Linear Regression - Example

Multiple Regression Analysis

Testing the Multiple Regression Model The test is used to investigate whether any of the independent variables have significant coefficients. The hypotheses are:

Multiple Regression Analysis  The test statistic is the F distribution with k (number of independent variables) and n-(k+1) degrees of freedom, where n is the sample size.  Decision Rule: Reject H 0 if F > F ,k,n-k-1 Testing the Multiple Regression Model

Multiple Regression Analysis The ANOVA Table The ANOVA (Analysis Of Variation) table reports the variation in the dependent variable. The variation is divided into two components.  The Explained Variation is that accounted for by the set of independent variable.  The Unexplained or Random Variation is not accounted for by the independent variables.

Multiple Regression Analysis ANOVA Table

Multiple Regression Analysis Interpretation  The computed value of F is 21.90, which is in the rejection region.  The null hypothesis that all the multiple regression coefficients are zero is therefore rejected.  Interpretation: some of the independent variables (amount of insulation, etc.) do have the ability to explain the variation in the dependent variable (heating cost).  Logical question – which ones?

Multiple Regression Analysis Evaluating Individual Regression Coefficients (β i = 0)  This test is used to determine which independent variables have nonzero regression coefficients.  The variables that have zero regression coefficients are usually dropped from the analysis.  The test statistic is the t distribution with n-(k+1) degrees of freedom.  The hypothesis test is as follows: H 0 : β i = 0 H 1 : β i ≠ 0 Reject H 0 if t > t  /2,n-k-1 or t t  /2,n-k-1 or t < -t  /2,n-k-1

Multiple Regression Analysis Critical t-stat for the Slopes

Multiple Regression Analysis The t-test shows that mean outside temperature and attic insulation are significantly (p-value 0.05) relationship with cost of heating. Interpretation

Multiple Regression Analysis Qualitative Independent Variables  Frequently we wish to use nominal-scale variables—such as gender, whether the home has a swimming pool, or whether the sports team was the home or the visiting team—in our analysis. These are called qualitative variables.  To use a qualitative variable in regression analysis, we use a scheme of dummy variables in which one of the two possible conditions is coded 0 and the other 1.

Multiple Regression Analysis  Use of dummy variables Dummy variables are used when a nominal scale variable is to be included in the regression Dummy variables are used when a nominal scale variable is to be included in the regression When there are two categories of the variable, then one dummy variable is used. When there are two categories of the variable, then one dummy variable is used. When there are n categories, then n-1 dummy variables are used. When there are n categories, then n-1 dummy variables are used. Qualitative Independent Variables

Multiple Regression Analysis Suppose in the Salsberry Realty example that the independent variable “garage” is added. For those homes without an attached garage, 0 is used; for homes with an attached garage, a 1 is used.