1. 1 Regression & Correlation (1) 1.A relationship between 2 variables X and Y 2.The relationship seen as a straight line 3.Two problems 4.How can we tell if our regression line is useful? 5.Test of hypothesis about the slope, β 1 6.Correlation 7.Useful features of r 8.Test of hypothesis about ρ 9.Examples

2. 2 A relationship between two variables X & Y We often have pairs of scores for a given set of cases. For example, we might have: * # of years of education and annual income, or * IQ and GPA * income and # of books in the household More generally, we have any X and Y, and our question is, does knowing something about X tell us anything about Y?

3. 3 A relationship between two variables X & Y Does knowing something about X tell us anything about Y? For example, knowing how many years of education a person has, could you usefully estimate their annual income, or the number of cigarettes they smoke in a year?

4. 4 A relationship between two variables X & Y Often, the answer to that question is, Yes – there is a relationship between the X and Y scores you have measured. * On average, as number of years of education goes up (across a set of people), number of cigarettes smoked per year goes down.

5. 5 A relationship between two variables X & Y In the graph on the next slide, we see two things: 1.X goes down as Y goes up. 2.At each value of X, there is some variability in Y – but substantially less than there is in Y overall.

6. 6 X = Years of education Y = Cigarettes per year Note that the range of the Y values for this value of X is small, compared to the whole range of Y in the data set.

7. 7 The relationship seen as a straight line The relationship between an X and a Y can be described using the equation for a straight line. Y = β 0 + β 1 X + ε Y-interceptSlopeError Note: this is the (theoretical) population equation relating Y to X

8. 8 Two problems Y = β 0 + β 1 X + ε In principle, this equation would let us predict the value of Y for a given X without error IF A. X were the only variable that influenced Y * Usually, it isn’t B. We knew the population values of β 0 + β 1 * Usually, we don’t

9. 9 Two problems Be sure to distinguish between A.Actual values of Y in the population. B.Values of Y we would predict using Y = β 0 + β 1 X + ε if we had the population values for β 0 + β 1. C. Values of Y we predict on the basis of the X-Y relationship in our sample data: Y = β 0 + β 1 X ^^ Why no ε here?

10. 10 Two problems When we predict Y on the basis of X for a given case, two things can cause the predicted values to be different from the values we would find if we actually measured Y for that case: 1. We don’t know the population values of β 0 and β 1 – only the sample values β 0 and β 1. Note that if we did know β 0 & β 1, this source of error would disappear. ^^

11. 11 Two problems 2. In the population, Y is not uniquely determined by X. As a result, for each value of X, there is a distribution of Y values. * relative to our predicted Y for a given value of X, the observed values of Y will sometimes be higher and sometimes be lower. * these “errors” are random – over the long term, they will cancel each other out * but even if we knew β 0 and β 1, this source of error would still exist.

12. 12 Two problems In other words 1.We don’t have population values for the slope and the intercept of the line relating X to Y. That’s one problem. 2.Even if we had population values for the slope and the intercept, the equation relating X to Y would still not perfectly predict Y. That’s the other problem.

13. 13 How can we tell if our regression line is useful? The line is useful if the predicted values of Y are close to the observed values of Y (in the sample). We use our sample X and Y values to compute the regression line, Y = β 0 + β 1 X. We then use this line to predict the same Y values, and compare our predicted values with the observed values in the sample data. If the prediction is good, we can then use the regression line to predict Y for values of X not in our sample. ^

14. 14 How can we tell if our regression line is useful? (Y i – Y i ) = Y i – (β 0 + β 1 X i )(since Y i = β 0 + β 1 X i ) Therefore, the sum of the squared deviations of predicted Y values from actual Y values is: SSE = Σ[Yi – (β 0 + β 1 X i )] 2 Now β 0 and β 1 are the “least squares estimators” of β 0 + β 1 – giving smaller SSE than any other values of β 0 and β 1 would. ^^^^^^ ^^ ^^ ^ ^

15. 15 X Y When there is no relation between X and Y, the best estimator of the Y value for any case is the mean, Y. Notice that the slope of this line is zero!

16. 16 How can we tell if our regression line is useful? If X is completely unrelated to Y, the best estimate we could make of Y would be the mean, Y, for any value of X. We find out whether our regression line is useful by asking whether its slope is different from 0. H 0 : β 1 = 0 [Why not β 1 ?] ^

17. 17 How can we tell if our regression line is useful? To test that null hypothesis, we use the fact that β1 is one slope taken from the sampling distribution of β 1. β 1 = SS XY β 0 = Y - β 1 X SS XX Where SS XY = Σ(X i – X) (Y i –Y) = ΣX i Y i – ΣX i ΣY i n ^ ^ ^^^

18. 18 How can we tell if our regression line is useful? SS XX = Σ(X i – X) 2 = ΣX 2 – (ΣX) 2 n (n = sample size) For the sampling distribution of β 1 : The mean = β 1  β 1 =  √SS XX ^ ^

19. 19 How can we tell if our regression line is useful? We estimate  β 1 by s β 1 = s √SS XX Where s = SSE n-2 ^ ^ √

20. 20 Test of hypothesis about the slope, β 1 Since  is unknown, we use t to test H 0 : H 0 : β 1 = 0H 0 : β 1 = 0 H A : β 1 < 0H A : β 1 ≠ 0 or β 1 > 0 Test statistic:t = β 1 – 0 S β 1 ^ ^

21. 21 Test of hypothesis about the slope, β 1 Rejection region: t obt t  /2 t obt > t  t crit is based on n-2 degrees of freedom.

22. 22 Correlation The Pearson Correlation coefficient r is a numerical, descriptive measure of the strength and direction of relationship between two variables X and Y. r = SS XY SS XX SS YY r gives much the same information as β 1. However r is “scale-less” and (-1 ≤ r ≤1) √ ^

23. 23 Useful features of r r indexes the X-Y relationship: r > 0 means Y increases as X increases r < 0 means Y decreases as X increases r = 0 means there is no relationship between X & Y r is the sample correlation coefficient. We can use it to estimate rho (ρ), the population correlation coefficient, and use r to test H 0 : ρ = 0

24. 24 Test of hypothesis about ρ H 0 : ρ = 0H 0 : ρ = 0 H A : ρ < 0H A : ρ ≠ 0 or ρ > 0 Test statistic:t = r – ρ 1 – r 2 n – 2 t crit has n-2 degrees of freedom. √

25. 25 Example 1 H 0 : ρ = 0 H A : ρ ≠ 0 Test statistic:t = r – ρ 1 – r 2 n – 2 t crit = t (5, α/2 =.025) = 2.571. √

26. 26 Example 1 – Sum formulas First, calculations involving X: ΣX = 74(ΣX) 2 = 5476ΣX 2 = 922 Then, analogous calculations involving Y: ΣY = 82(ΣY) 2 = 6724ΣY 2 = 1076 Then, calculations involving X and Y: ΣXY = 976

27. 27 Example 1 – Sums of squares formulas SS XY = Σ(X i – X) (Y i –Y) = ΣX i Y i – ΣX i ΣY i n SS XX = Σ(X i – X) 2 = ΣX 2 – (ΣX) 2 n SS YY = Σ(Y i – Y) 2 = ΣY 2 – (ΣY) 2 n

28. 28 Example 1 – calculate r SS XY = 109.143 SS XX = 139.71 SS YY = 115.429 r = SS XY r =.859 SS XX SS YY √

29. 29 Example 1 – do t-test t = r – ρ 1 – r 2 n – 2 t =.859 - 0=.859= 3.751 1 -.738.229 5 Reject H 0 : A significant correlation exists. √ √

30. 30 Example 2 H 0 : ρ = 0 H A : ρ > 0 Test statistic:t = r – ρ 1 – r 2 n – 2 t crit = t (7-2 = 5, α =.05) = 2.015 √ Note – these are the Greek letter rho, NOT the English letter P

31. 31 Example 2 – Sum formulas First, calculations involving X: ΣX = 4.2(ΣX) 2 = 17.64ΣX 2 = 2.86 Then, analogous calculations involving Y: ΣY = 32(ΣY) 2 = 1024ΣY 2 = 161.5 Then, calculations involving X and Y: ΣXY = 21.35

32. 32 Example 2 – calculate r SS XY = 21.35 – (4.2)(32) = 2.15 7 SS XX = 2.86 – 17.64=.34 7

33. 33 Example 2 – calculate r SS YY = 161.5 – 1024= 15.2143 7 r = SS XY SS XX SS YY r =.945 √

34. 34 Example 2 – do t-test t = r – ρ 1 – r 2 n – 2 t =.945 - 0=.945= 6.48 1 -.893.146 5 Reject H 0 : A significant correlation exists. √ √