Intro to Statistics for the Behavioral Sciences PSYC 1900 Lecture 7: Multiple Regression

Bivariate Regression Review Predicts values of Y as a linear function of X The intercept: a The predicted value of Y when X=0 The slope: b The change in Y associated with a one unit change in X Regression line is one that minimizes errors

Partitioning Variance Total variance of Y is partitioned into portion explained by X and error R2 is portion that is explained Standard error of estimate is average deviation around prediction line

r r2 1 - r2 .10 .01 .99 .20 .04 .96 .30 .09 .91 .40 .16 .84 .50 .25 .75 .60 .36 .64 .70 .49 .51 .80 .90 .81 .19 .95

Hypothesis Testing in Regression The null hypothesis is simply that the slope equals zero. This is equivalent to testing r=0 in correlation. If the correlation is significant, so must the slope be. The actual significance of the slope is tested using a t-distribution. The logic is similar to all hypothesis testing. We compare the magnitude of the slope (b) to its standard error (i.e., the variability of slopes drawn from a population where the null is true).

Hypothesis Testing in Regression The formula to calculate the t value is: Note that standard error of b increases as standard error of the estimate increases We then determine how likely it would be that we found a slope as large as we did using a t distribution (similar to the normal distribution).

Multiple Regression Allows analysis of more than one independent variable Explains variance in Y as a function of a linear composite of iv’s Each iv has a regression coefficient which provides an estimate of its independent effects.

An Example Suppose we want to examine two factors related to the percentage of low birth weights in an area. Percentage of mothers with low education levels Percentage of mothers not seeking prenatal care

Looking at Correlations Here it appears that both low education and lack of care predict increased lower birth weights. Is it fair to say that both variables might be causes?

Example Continued Given that low education and lack of prenatal care are associated, we need to control for one while examining the relation of the other to low birth weights. Multiple regression accomplishes this task.

Another Example Let’s examine applicant attractiveness as a function of GREV, Letters of Rec, & Personal Statements Y, letters, & statements are rated on 1-7 scales If all are significant, which variables are “stronger” predictors? Use standardized regression coefficients

Standardized Regressions The use of standardized coefficients allows easier comparisons of the magnitude of effects Coefficients refer to changes in predicted z-scores of Y as a function of z-scores of x What is the relation of b to r here? In multiple regression, b only equals r if all iv’s are uncorrelated.

Squared Multiple Correlation Coefficient In bivariate regression, we saw that r2 provided an estimate of the variance in Y explained by variance in X. In multiple regression, R2 serves this value. Note that capital R refers to the correlation of the linear composite (i.e., the weighted combination of all iv’s) with Y.

Squared Multiple Correlation Coefficient Returning to the low birth weight example: Here we see that education level and prenatal care explain 67% of the variance in the percentage of low birth weights.

Hypothesis Testing There are two types of null hypotheses in multiple regression. Null for the entire model The linear function (i.e., all variables) does not explain any variance in Y. R2 = 0 If you cannot reject this null, the analysis is over. Null for each predictor Any given iv does not explain variance in Y. b = 0

Testing Hypotheses for Individual Predictors Hypothesis testing here is quite similar to that for the single iv in bivariate regression. But note that in multiple regression, the standard error is sensitive to the overlap (i.e., correlation) among coefficients. As the intercorrelation increases, so does the standard error.

Testing Hypotheses for the Model Here, we must determine if R2 differs form zero. This is accomplished using the F-distribution (we’ll see this again in ANOVA). Logic is the same, however. Is R2 in the sample large enough to make the null unlikely in the population? p=.02

Refining a Model When you are building a model, one way to determine if the addition of new variables improves fit is to see if adding them results in a significant change in R2. If so, this means that adding the variable explains a significant amount of previously unexplained variability. Why not just add variables that significantly correlate with Y?

Time for Examples