Simple Linear Regression 1

Correlation indicates the magnitude and direction of the linear relationship between two variables. Linear Regression: variable Y (criterion) is predicted by variable X (predictor) using a linear equation. Advantages: Scores on X allow prediction of scores on Y. Allows for multiple predictors (continuous and categorical) so you can control for variables. 2

Linear Regression Equation Geometry equation for a line: y = mx + b Regression equation for a line (population): y = β 0 + β 1 x β 0 : point where the line intercepts y-axis β 1 : slope of the line

Regression: Finding the Best-Fitting Line Course Evaluations Grade in Class

Best-Fitting Line Course Evaluations Grade in Class Minimize this squared distance across all data points

Slope and Intercept in Scatterplots y = b 0 + b 1 x + e y = x + e y = b 0 + b 1 x + e y = 3 - 2x + e slope is: rise/run = -2/1

Estimating Equation from Scatterplot y = b 0 + b 1 x + e y = 5 +.3x + e run = 50 rise = 15 slope = 15/50 =.3 Predict price at quality = 90 y = 5 +.3x + e y = 5 +.3*90 = 35

Example Van Camp, Barden & Sloan (2010) Contact with Blacks Scale: Ex: “What percentage of your neighborhood growing up was Black?” 0%-100% Race Related Reasons for College Choice: Ex: “To what extent did you come to Howard specifically because the student body is predominantly Black?” 1(not very much) – 10 (very much) Your predictions, how would prior contact predicts race related reasons?

Results Van Camp, Barden & Sloan (2010) Regression equation (sample): y = b 0 + b 1 x + e Contact(x) predict Reasons: y = x + e b 0 : t(107) = 14.17, p <.01 b 1 : t(107) = -2.93, p <.01 df = N – k – 1 = 109 – 1 – 1 k: predictors entered

Unstandardized and Standardized b unstandardized b: in the original units of X and Y tells us how much a change in X will produce a change in Y in the original units (meters, scale points…) not possible to compare relative impact of multiple predictors standardized b: scores 1 st standardized to SD units +1 SD change in X produces b*SD change in Y indicates relative importance of multiple predictors of Y

Results Van Camp, Barden & Sloan (2010) Contact predicts Reasons: Unstandardized: y = x + e Standardized: y = x + e (M x = 5.89, SD x = 2.53; M y = 5.61, SD y = 2.08) (M x = 0, SD x = 1.00; M y = 0, SD y = 1.00)

save new variables that are standardized versions of current variables

add fit lines add reference lines (may need to adjust to mean) select fit line

Predicting Y from X Once we have a straight line we can know what the change in Y is with each change in X Y prime (Y’) is the prediction of Y at a given X, and it is the average Y score at that X score. Warning: Predictions can only be made: (1) within the range of the sample (2) for individuals taken from a similar population under similar circumstances.

Errors around the regression line Regression equation give us the straight line that minimizes the error involved in making predictions (least squares regression line). Residual: difference between an actual Y value and predicted (Y’) value: Y – Y’ – It is the amount of the original value that is left over after the prediction is subtracted out – The amount of error above and below the line is the same

Y’ Y residual Y

Dividing up Variance Total: deviation of individual data points from the sample mean Explained: deviation of the regression line from the mean Unexplained: deviation of individual data points from the regression line (error in prediction) unexplained explained total variance variance variance (residual)

Y Y’ Y residual explained total variance unexplained explained total variancevariance (residual)

Coefficient of determination: proportion of the total variance that is explained by the predictor variable R 2 = explained variance total variance

SPSS - regression Analyze → regression → linear Select criterion variable (Y) [Racereas] [SPSS calls DV] Select predictor variable (X) [ContactBlacks] [SPSS calls IV] OK

Coefficients a Model Unstandardized Coefficients Standardized Coefficients tSig. BStd. ErrorBeta 1(Constant) ContactBlacksperc a. Dependent Variable: RaceReasons ANOVA b Model Sum of Squaresdf Mean SquareFSig. 1Regression a Residual Total a. Predictors: (Constant), ContactBlacksperc124 b. Dependent Variable: RaceReasons Model Summary b ModelRR Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), ContactBlacksperc124 b. Dependent Variable: RaceReasons Unstandardized: Standardized: y = x + e y = x + e coefficient of determination Reporting in Results: b = -.27, t(107) = -2.93, p <.01. (pp. 240 in Van Camp 2010) SS error : minimized in OLS

Assumptions Underlying Linear Regression 1.Independent random sampling 2.Normal distribution 3.Linear relationships (not curvilinear) 4.Homoscedasticity of errors (homogeneity) Best way to check 2-4? Diagnostic Plots.

Test for Normality Right (positive) Skew Narrow Distribution Positive Outliers o o o o Normal Distribution: Solution? transform data not serious investigate further

Homoscedastic? Linear Appropriate? Heteroscedasticity (of residual errors) Curvilinear relationship Homoscedastic residual errors & Linear relationship Solution: add x 2 as predictor (linear regression not appropriate) Solution: transform data or weighted least squares (WLS)

SPSS—Diagnostic Graphs

END