1. Sociology 601 Class 17: October 28, 2009 Review (linear regression) –new terms and concepts –assumptions –reading regression computer outputs Correlation (Agresti and Finlay 9.4) –the correlation coefficient r –relationship to regression coefficient b –r-squared: the reduction in error 1

2. Review: Linear Regression N ew terms and concepts –slope –intercept –– –– –negative and positive slopes –zero slope –least squares regression –predicted value –residuals –sums of squares error 2

3. Review: Linear Regression Assumptions –random sample (errors are independent) –linear –no heteroscedasticity –no outliers Linearity, heteroscedasticity, and outliers can be checked with scattergrams and crosstabs –before computing regressions –on residuals 3

4. A Problem with Regression Coefficients Regression coefficients don’t measure the strength of an association in a way that is easily compared across different models with different variables or different scales. Rescaling one or both axes changes the slope b. Example: murder rate and poverty rate for 50 US States. Y hat = -.86 +.58X, o where Y = murder rate per 100,000 per year o and X = poverty rate per 100 If we rescale y, the murder rate, to murders per 100 persons per year, then Y hat = -.00086 +.00058X o (does this mean the association is now weaker?) If we rescale x, the poverty rate, to proportion in poverty (0.00 -> 1.00), then Y hat = -.86 + 58X o (does this mean the association is now stronger?) 4

5. the correlation – a standardized slope An accepted solution for the problem of scale is to standardize both axes (e.g., change them into z-scores with mean zero and a standard deviation of 1), then calculate the slope. b =  Y /  X r = (  Y /s Y )/(  X /s X ) = (  Y /  X )*(s X /s Y )= b*(s X /s Y ) where 5

6. The Correlation Coefficient, r r is called … –the Pearson correlation (or simply the correlation) –the standardized regression coefficient (or the standardized slope) r = b*(s X /s Y ) r is a sample statistic we use to estimate a population parameter  6

7. Calculating r: an example Calculating r for the murder and poverty example b =.58, s X = 4.29, s Y = 3.98 r = b*(s X /s Y ) =.58*(4.29/3.98) =.629=.63 alternatively (if the murder rate is per 100 persons), b =.00058, s X = 4.29, s Y =.00398 r = b*(s X /s Y ) =.00058*(4.29/.00398) =.629 =.63 7

8. Properties of the correlation coefficient r: –1  r  1 r can be positive or negative, and has the same sign as b. r = ± 1 when all the points fall exactly on the prediction line. The larger the absolute value of r, the stronger the linear association. r = 0 when there is no linear trend in the relationship between X and Y. 8

9. Properties of the Correlation Coefficient r: The value of r does not depend on the units of X and Y. The correlation treats X and Y symmetrically –(unlike the slope β) –this means that a correlation implies nothing about causal direction! The correlation is valid only when a straight line is a reasonable model for the relationship between X and Y. 9

10. Examples of the correlation coefficient r: b = 1, r = 1 b = 5, r = 1 b =.2, r = 1 b = -1, r = -1 b =.5, r =.8 b =.5, r =.3 b = 0, linear assumption holds b = 0, linear assumption does not hold 10

11. Calculating a correlation coefficient using STATA Recall the religion and state control study, where high levels of state regulation were associated with low levels of weekly church attendance.. correlate attend regul (obs=18) | attend regul -------------+------------------ attend | 1.0000 regul | -0.6133 1.0000 11

12. An alternative interpretation of r: proportional reduction in error Old interpretation for murder and poverty example: r =.63, the murder rate for a state is expected to be higher by 0.63 standard deviations for each 1.0 standard deviation increase in the poverty rate. New interpretation: by using poverty rates to predict murder rates, we explain ?? percent of the variation in states’ murder rates. 12

13. Proportional reduction in error: Predicting Y without using X: Y = Y bar + e 1 ; E 1 =  e 1 2 =  (observed Y – predicted Y) 2 = Total Sums of Squares = TSS Predicting Y using X: Y = Y hat + e 2 = a + bX + e 2 ; E 2 =  e 2 2 =  (observed Y – predicted Y) 2 = Sum of Squared Error = SSE Proportional reduction in error: r 2 = PRE = (E 1 – E 2 ) / E 1 = (TSS – SSE) / TSS 13

14. Proportional reduction in error. calculating r 2 for the murder and poverty example: r 2 =.629 2 =.395 alternatively (using computer output), r 2 = (TSS – SSE) / TSS = (777.7 – 470.4)/777.7 =.395 interpretation: 39.5% of the variation in states’ murder rates is explained by its linear relationship with states’ poverty rates. 14

15. R-square r 2 is also called the coefficient of determination. Properties of r 2 : 0  r 2  1 r 2 = 1 (its maximum value) when SSE = 0. r 2 = 0 when SSE = TSS. (furthermore, b = 0) the higher r 2 is, the stronger the linear association between X and Y. r 2 does not depend on the units of measurement. r 2 takes the same value when X predicts Y as when Y predicts X. 15

16. Next Class drawing inference to populations from sample b’s & r’s. x 16