Regression

Outline of Today’s Discussion 1.Coefficient of Determination 2.Regression Analysis: Introduction 3.Regression Analysis: SPSS 4.Regression Analysis: Excel 5.Independent Predictors

Part 1 Coefficient of Determination

In correlational research Researchers often use the “r-squared” statistic, also called the “coefficient of determination”, to describe the proportion of Y variability explained by X.

Coefficient of Determination What range of values is possible for the coefficient of determination (the r-squared statistic)?

Coefficient of Determination Example: What is the evidence that IQ is heritable?

Coefficient of Determination R-value for the IQ of identical twins reared apart = 0.6. What is the value of r-squared in this case?

Coefficient of Determination So what proportion of the IQ is unexplained (unaccounted for) by genetics?

Coefficient of Determination Different sciences are characterized by the r-squared values that are deemed impressive. (Chemists might r-squared to be > 0.99).

Coefficient of Determination As we have already seen r-squared is the same as “eta-squared”.

Part 2 Regression Analysis Introduction

Correlation is the process of finding a relationship between variables. Regression is the process of finding the best-fitting trend (line) that describes the relationship between variables. So, correlation and regression are very similar!

Regression Analysis Introduction The ‘r’ statistic can be tested for statistical significance! Potential Pop Quiz Question: What two factors determine the critical value (i.e., the number to beat) when we engage in hypothesis testing?

Regression Analysis Introduction DF for Correlation & Regression Here n stands for the number of pairs of scores. Why would this be n-2, rather than the usual n-1?

Regression Analysis Introduction In general, the formula for the degrees of freedom is the number of observations minus the number of parameters estimated. For correlation, we have one estimate for the mean of X, and another estimate for the mean of Y. For regression, we have one estimate for the slope, and another estimate for y intercept.

Regression Analysis Introduction Slope can also be though of as “rise over run”.

Regression Analysis Introduction The “rise” on the ordinate = Y 2 - Y 1. The “run” on the abscissa = X 2 - X 1.

Regression Analysis Introduction “Rise over run” in pictures.

Regression Analysis Introduction Here, the regression is “linear”…

Regression Analysis Introduction Here, the regression is non-linear! What would the equation look like for this trend?

Regression Analysis Introduction Let’s now return to linear regression, and learn how to manually compute the slope and y-intercept. To compute the slope, we need two quantities that we have already learned. These are SP xy (sums of products) and SS x (sums of squares for X)…

Regression Analysis Introduction

Once we have the slope, it’s easy to get the y-intercept!

Part 3 Regression Analysis: SPSS

Regression Analysis: SPSS Later we’ll go to SPSS and get some practice with regression. The steps in SPSS will be Analyze ---> Regression --> Linear. We will place the criterion (i.e., the Y-axis variable) in the “Dependent” box, and the predictor (i.e., the X-axis variable) in the “Independent(s)” box. Click the “Statistics” box, and check “estimates”, “model fit”, and “descriptives”.

Regression Analysis: SPSS The “Coefficients” Section in the SPSS output contains all the info needed for the regression equation, the r statistic, and the evaluation of Ho (retain or reject). The “Coefficients” Section In SPSS Output

Regression Analysis: SPSS The constant is the “b” in, Y = mX + b. Here, b = The “Coefficients” Section In SPSS Output

Regression Analysis: SPSS The slope is the “m” in, Y = mX + b. Here, m = The “Coefficients” Section In SPSS Output

Regression Analysis: SPSS So, our regression equation is, Y = mX + b. or Y = X The “Coefficients” Section In SPSS Output

Regression Analysis: SPSS The r statistic is the standardized coefficient, Beta. r =.705 The “Coefficients” Section In SPSS Output

Regression Analysis: SPSS Lastly, we look at the ‘sig’ value for the predictor, (which is “EDU” in this case) to determine whether predictor (x-axis variable) is significantly correlated with the criterion (y-axis variable). Evaluate Ho: …do we retain or reject? The “Coefficients” Section In SPSS Output

Part 4 Regression Analysis: Excel

Regression Analysis: Excel Correlation and regression are very similar. If we have a significant correlation, the best-fitting regression line is said to have a slope significantly different from zero. Sometimes it is stated that “the slope departs significantly from zero”.

Regression Analysis: Excel Note: A slope can be very modestly different from zero, and still be “statistically significant” if all data points fall very close to the line. In correlation and regression, statistical significance is determined by the strength of the correlation between two variables (the r-value), and NOT by the slope of the regression line. The significance of the r-value, as always, depends on the alpha level, and the df (which is n-2). Take a peak at the r-value table.

Regression Analysis: Excel

Remember: The regression line (equation) can help us predict one score, given another score, but only if there is a significant r-value. The terminology w/b… “the regression line explains (or accounts for)” 42% of the variability in the scores (if r- squared =.42). To “explain” or “account for” does NOT mean “to cause”. Correlation does not imply causation!

Regression Analysis Continued A synonym for regression is prediction! Recall that prediction is one of the four goals of the scientific method. What were the others? A significant correlation implies a significant capacity for prediction, i.e., a prediction that is reliably better than chance!

Regression Analysis Continued The equation for a straight line, again, is: y = mx + B or Criterion = ( slope * Predictor) + Intercept How many “parameters” in a linear equation? How about a quadratic equation?

Part 5 Independent Predictors

So far, we’ve attempted to use regression for prediction. Specifically, we’ve tried to predict one variable Y (called the criterion), using one other variable (called the predictor). Multiple Regression - the process by which one variable Y (called the criterion) is predicted on the basis of more than one variable (say, X1, X2, X3…).

Independent Predictors Here’s the simple case of one predictor variable. The overlap (in gray) indicates the predictive strength.

Independent Predictors If the overlap in the Venn diagram were to grow, the r-value would grow, too!

Independent Prediction Here’s the same thing again… but we’ll call the the predictor variable X 1. Criterion (Y) Variable X 1

Independent Prediction By adding another predictor variable X 2, we could sharpen our predictions. Why? Criterion (Y) Variable X 1 Variable X 2

Independent Prediction Unfortunately, X 1 and X 2 provide some redundant information about Y, so the predictive increase is small. Criterion (Y) Variable X 1 Variable X 2

Independent Prediction Unfortunately, X 1 and X 2 provide some redundant information about Y, so the predictive increase is small. Criterion (Y) Variable X 1 Variable X 2

Independent Prediction By contrast, variable X 3 has no overlap with either X 1 or X 2, so it would add the most new information. Criterion (Y) Variable X 1 Variable X 2 Variable X 3

Independent Prediction In short, since all three predictors provide some unique information, predictions w/b best when using all three. Criterion (Y) Variable X 1 Variable X 2 Variable X 3

Independent Prediction If you wanted to be more parsimonious and use only two of the three, which two would you pick, and why? Criterion (Y) Variable X 1 Variable X 2 Variable X 3

Independent Predictors That was a conceptual introduction to Multiple Regression (predicting Y scores from more than one variable). We will not learn about the computations for multiple regression in this course (but you will if you take the PSYCH 370 course). For our purposes, simply know that predictions improve to the extent that the various predictors are independent of each other.