3. Relationships If we are doing a study which involves more than one variable, how can we tell if there is a relationship between two (or more) of the variables ? Association Between Variables : Two variables measured on the same individuals are associated if some values of one variable tend to occur more often with some values of the second variable than with other values of that variable. Response Variable : A response variable measures an outcome of a study. Explanatory Variable : An explanatory variable explains or causes changes in the response variable.

4. 2.1: Scatterplots A scatterplot shows the relationship between two variables. The values of one variable appear on the horizontal axis, and the values of the other variable appear on the vertical axis. Always plot the explanatory variable on the horizontal axis, and the response variable as the vertical axis. Example: If we are going to try to predict someone’s weight from their height, then the height is the explanatory variable, and the weight is the response variable. The explanatory variable is often denoted by the variable x, and is sometimes called the independent variable. The response variable is often denoted by the variable y, and is sometimes called the dependent variable.

5. Scatterplots Example: Do you think that a father’s height would affect a son’s height? We are saying that given a father’s height, can we make any determinations about the son’s height ? The explanatory variable is :The father’s height The response variable is :The son’s height Data Set : Father’s HeightSon’s Height 6465 6867 6870 72 75 7470 7573 7576 77 76

6. Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76 64687276 64 68 72 76 Explanatory Variable (Father’s Height) Response Variable (Son’s Height)

7. Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76 64687276 64 68 72 76 Father Son

8. Examining A Scatterplot In any graph of data, look for the overall pattern and for striking deviations from that pattern. You can describe the overall pattern of a scatterplot by the form, direction, and strength of the relationship. An important kind of deviation is an outlier, an individual that falls outside the overall pattern of the relationship. Two variables are positively associated when above-average values of one tend to accompany above average values of the other and below average values also tend to occur together. Two variables are negatively associated when above-average values of one accompany below-average values of the other; and vice versa. Strength : How closely the points follow a clear form.

9. Direction Type of associations between X and Y. 1. Two variables are positively associated if small values of X are associated with small values of the Y, and if large values of X are associated with large values of Y. There is an upward trend from left to right.

10. Positive Association Y....... X

11. Direction Type of associations between X and Y. 2. Two variables are negatively associated if small values of one variable are associated with large values of the other variable, and vice versa. There is a downward trend from left to right.

12. Negative Association Y........... X

13. Form Describe the type of trend between X and Y. 1. Linear - points fall close to a straight line.

14. Linear Association Y....... X

15. Form Describe the type of trend between X and Y. 1. Linear - points fall close to a straight line. 2. Quadratic - points follow a parabolic pattern.

16. Quadratic Association Y................ X

17. Form Describe the type of trend between X and Y. 1. Linear - points fall close to a straight line. 2. Quadratic - points follow a parabolic pattern. 3. Exponential - points follow a curved pattern.

18. Exponential Growth Y............................. X

19. Strength Measures the amount of scatter around the general trend. Linear- The closer the points fall to a straight line, the stronger the relationship between the two variables.

20. Strong Association Y....... X

21. Moderate Association Y................. X

22. Weak Association Y................ X

23. Examining A Scatterplot Consider the previous scatterplot : 6468 72 76 64 68 72 76 Father Son Direction :Going up Form :Linear Association :Positive Strength :Strong Outliers :None

24. Example : The following is a scatterplot of data collected from states about students taking the SAT. The question is whether the percentage of students from a state that takes the test will influence the state’s average scores. For instance, in California, 45 % of high school graduates took the SAT and the mean verbal score was 495. Direction :Downward Form :Curved Association :Negative Strength :Strong Outliers : Maybe

25. §2.2: Correlation Recall that a scatterplot displays the form, direction, and strength of the relationship between two quantitative variables. Linear relationships are important because they are the easiest to model, and are fairly common. We say a linear relationship is strong if the points lie close to a straight line, and weak if the points are scattered around the line. Correlation (r) measures the direction and the strength of the linear relationship between two quantitative variables. The + / - sign denotes a positive or negative association. The numeric value shows the strength. If the strength is strong, then r will be close to 1 or -1. If the strength is weak, then r will be close to 0.

26. Correlation Examples Correlation = - 0.99 Correlation = 0.9 Correlation = - 0.7 Correlation = 0.5 Correlation = - 0.3 Correlation = 0

27. Which has the better correlation ?

28. Correlation So, how do we find the correlation ? Suppose we have data on variables x and y for n individuals. The means and standard deviations of the two variables are and for the x-values, and and for the y-values. x ys x s y r = n - 1 1  x i - xy i - y s y s x ( ) () Question : Will outliers effect the correlation ?

29. Example: Recall the scatterplot data for the heights of fathers and their sons. Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76 We decided that the father’s heights was the explanatory variable and the son’s heights was the response variable. The average of the x terms is 71.9 and the standard deviation is 4.25 The average of the y terms is 72.1 and the standard deviation is 4.07

30. r = n - 1 1  x i - xy i - y s y s x ( ) () x x - x x i x i - x i s 64-7.9-1.86 68-3.9-0.92 68-3.9-0.92 70-1.9-0.45 720.10.02 742.10.49 753.10.73 753.10.73 764.10.96 775.11.20 65-7.1-1.75 67-5.1-1.25 70-2.1-0.52 72-0.1-0.02 752.90.71 70-2.1-0.52 730.90.22 763.90.95 774.91.20 763.90.95 y- y y i y i - y i s y

31. x x - x x i x i - x i s 64-7.9-1.86 68-3.9-0.92 68-3.9-0.92 70-1.9-0.45 720.10.02 742.10.49 753.10.73 753.10.73 764.10.96 775.11.20 65-7.1-1.75 67-5.1-1.25 70-2.1-0.52 72-0.1-0.02 752.90.71 70-2.1-0.52 730.90.22 763.90.95 774.91.20 763.90.95 y- y y i y i - y i s y r = 10 - 1 1 [ (-1.86)(-1.75) + (-0.92)(-1.25) + ….. + (1.20)(0.95) ] = 9 1 [ (3.24) + (1.14) + ….. + (1.14) ] = 0.87

32. Shortcut Calculations

33. Facts about Correlation Correlation makes no distinction between explanatory and response variables. The correlation between x and y is the same as the correlation between y and x. Correlation requires that both variables be quantitative. We cannot compute a correlation between a categorical variable and a quantitative variable or between two quantitative variables. r does not change when we do transformations. The correlation between height and weight is the same whether height was measured in feet or centimeters or weight was measured in kilograms or pounds. This happens because all the observations are standardized in the Calculation of correlation. The correlation r itself has no unit of measurement, it is just a number.

34. Exercise What’s wrong with these statements? 1. At AU there is no correlation between the ethnicity of students and their GPA. 2. The correlation between height and weight of stat202 students (b) is 0.61 inches per pound (a) is 2.61 (d) is 0.61 using inches and pounds, but converting inches to centimeters would make r > 0.61 (since an inch equals about 2.54 centimeters). (c) is 0.61, so the corr. between weight and height is -0.61

35. §2.3: Least-Squares Regression Correlation measures the direction and strength of a straight-line (linear) relationship between two quantitative variables. We have tried to summarize the data by drawing a straight-line the through the data. A regression line summarizes the relationship between two variables. These can only be used in one setting : when one variable helps explain or predict the other variable.

36. Regression Line A regression line is a straight line that describes how a response variable y changes as an explanatory variable x changes. We often use a regression line to predict the value of y for a given value of x. Regression, unlike correlation, requires that we have an explanatory variable and a response variable. If a scatterplot displays a linear pattern, we can describe the overall pattern by drawing a straight line through the points. This is called fitting a line to the data. This is a mathematical model which we can use to make predictions based on the given data.

37. Example: Recall the data we were using before where we compared the heights of fathers and sons. Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76 The first thing we did was to plot the points.

38. 64687276 64 68 72 76 Father Son Example: Recall the data we were using before where we compared the heights of fathers and sons. The line which is closest to all the points is the regression line.

39. Least-Squares Regression Line The least squares regression line of y on x is the line that makes the sum of the squares of the vertical distances of the data points from the line as small as possible.

40. Equation of the Least-Squares Regression Line Imagine we have data on an explanatory variable x and a response y for n individuals. Assume the mean for the explanatory variable is and the standard deviation is s x x Assume the mean for the response variable is and the standard deviation is s y y Assume the correlation between x and y is r.

41. Equation of the Least-Squares Regression Line The equation of the least-squares regression line of y on x is : y = a + bx slope intercept The slope is b : b = r s x s y () The intercept is a : a = y- bx

42. Interpretation of Regression Coefficients  The Y-Intercept a is the value of the response variable, y, when the explanatory variable, x, is zero.  The Slope, b is the change in the response variable, y, for a unit increase in the explanatory variable, x.

43. Example: What if we want to find the least-squares regression line where we will predict the son’s height from the father’s height ? We need the means and the standard deviations : Note: The father’s heights are the explanatory variable, and the son’s height is the response variable. The average of the x terms is 71.9 and the standard deviation is 4.25 The average of the y terms is 72.1 and the standard deviation is 4.07 The correlation between the two variables is 0.87

44. We need to find the slope : y = a + bx b = r s x s y () = 71.9x s x = 4.25y= 72.1 s y = 4.07 r = 0.87 The equation for the regression line is : b = 0.87 4.07 4.25 () = 0.8331529 Next, find the intercept : a = y- bx a = 72.1 - (0.8331529)(71.9) So, the equation for the regression line is : = 12.196307 y = a + bx y = 12.2 +.83x

45. Making Predictions We can use the regression line to make some predicts. Example : Based on the previous data, we can predict the son’s height from the father’s height. y = 12.2 +.83x Q: If the father’s height is 70 inches, what is our prediction for the son’s height? A: = 12.2 +.83(70) = y70.1 Note: These predictions are only good on relevant data!!

46. 64687276 64 68 72 76 Father Son y = 12.2 +.83x

47. Notes On Regression b = r s x s y () This equation says that a change of one standard deviation in x corresponds to a change of r standard deviations in y. The point is always on the regression line. yx, () The square of the correlation,, is the fraction of the variation in the values of y that is explained by the least-squares regression of y on x. r 2 Example: The straight line relationship between father’s heights and son’s height is = (0.87) = 0.7569 explains the variation in heights. r 22

48. Residuals Analysis A regression line is a mathematical model for the overall pattern of a linear relationship between an explanatory variable, and a response variable. The regression line is chosen so that the vertical distances to the line from all the points is as small as possible. A residual is the difference between an observed value of the response variable, and the value predicted by the regression line. Residual = observed y - predicted y Residual = y -y

49. Example of Residuals Go back to our favorite example : Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76

50. Example of Residuals Go back to our favorite example : Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76 y = 12.47 +.8293xWe found the regression line to be: So, when the father’s height is 64 inches, we expect the son to be how tall? y= 12.47 + (.8293)(64) =65.5452 However, the actual height of the son is 65 inches, so the residual is : 65 - 65.5452 = -0.5452 This tells us the point is.5452 units below the regression line.

51. Son’s Height Predicted Height Residual 65 67 70 72 75 70 73 76 77 76 y = 12.47 +.8293x 65.5452 68.8624 70.521 72.1796 73.8382 74.6675 75.4968 76.3261 Residual = y -y -0.5452 -1.8624 1.1376 1.479 2.8204 -3.8382 -1.6675 1.3325 1.5032 -0.3261 Average = 0.0037678 Again, we could have drawn our line anywhere on the graph The least squares regression line has the property that the mean of the least-squares residuals is always zero!

52. Residual Plot A residual plot is a scatterplot of the regression residuals against the explanatory variable. Residual plots help us assess the fit of the regression line. The regression line shows the overall pattern of the data. So, the residual plot should *not* have a pattern.

53. Residual Plot Example : What does this residual plot show us : This indicates the relationship between the explanatory variable and the response variable is curved, and not linear. Regression should not be used in this example.

54. Residual Plot Example : What does this residual plot show us : This shows that the variation of the response variable about the line increases as the explanatory variable increases. The predictions for y will be better on the less variable part, than the more variable part.

55. Residual Plot Here is what our residual plot would look like : 64666870727476 0 -2 2 4 -4

56. Outliers and Influential Observations Consider our favorite example : Father’s HeightSon’s Height 6465 6867 6870 72 75 Father’s HeightSon’s Height 7470 7573 7576 77 76 What happens if we add in an outlier ? 6482 How does this change our results?

57. Correlation & Scatterplot The correlation drops from 0.87 to 0.28

58. Outliers and Influential Observations Consider the following scatterplot : There is one outlier :

59. Outliers and Influential Observations An outlier has a large residual from the regression line. This could be called an outlier in the y direction. If you look at the previous picture, there is an “outlier” in the x direction This is called an influential observation.

60. Outliers and Influential Observations An influential observation is a score which is far from the other data points in the x direction. Note that it should still be close to the regression line, otherwise we would label it an outlier. An influential observation is a score that is extreme in the x direction with no other points around it. These values will pull the regression line towards itself.

61. Outliers and Influential Observations

62. An influential observation is a score which is far from the other data points in the x direction. Note that it should still be close to the regression line, otherwise we would label it an outlier. An influential observation is a score that is extreme in the x direction with no other points around it. These values will pull the regression line towards itself. So, an observation is influential if removing it would markedly change the result of the calculation.

63. Influential Observations Q: How can we check data for influential observations ? A1 : residuals ? A2 : Scatterplot ?Sort of. A3 : Remove the point and see what happens ?