1. Describing the Relation between Two Variables
Chapter 4
Describing the Relation between Two Variables

2. Scatter Diagrams and Correlation
Section
Scatter Diagrams and Correlation
4.1

3. The response variable is the variable whose value can be explained by the value of the explanatory or predictor variable.
A scatter diagram is a graph that shows the relationship between two quantitative variables measured on the same individual. Each individual in the data set is represented by a point in the scatter diagram. The explanatory variable is plotted on the horizontal axis, and the response variable is plotted on the vertical axis. 3

4. EXAMPLE Drawing and Interpreting a Scatter Diagram
The data shown to the right are based on a study for drilling rock. The researchers wanted to determine whether the time it takes to dry drill a distance of 5 feet in rock increases with the depth at which the drilling begins. So, depth at which drilling begins is the explanatory variable, x, and time (in minutes) to drill five feet is the response variable, y. Draw a scatter diagram of the data.
Source: Penner, R., and Watts, D.G. “Mining Information.” The American Statistician, Vol. 45, No. 1, Feb. 1991, p. 6. 4

5. Time 7 5

6. Various Types of Relations in a Scatter Diagram6

7. Two variables that are linearly related are positively associated when above-average values of one variable are associated with above-average values of the other variable and below-average values of one variable are associated with below-average values of the other variable. That is, two variables are positively associated if, whenever the value of one variable increases, the value of the other variable also increases. 7

8. Two variables that are linearly related are negatively associated when above-average values of one variable are associated with below-average values of the other variable. That is, two variables are negatively associated if, whenever the value of one variable increases, the value of the other variable decreases. 8

9. The linear correlation coefficient or Pearson product moment correlation coefficient is a measure of the strength and direction of the linear relation between two quantitative variables. The Greek letter ρ (rho) represents the population correlation coefficient, and r represents the sample correlation coefficient. We present only the formula for the sample correlation coefficient. 9

10. Sample Linear Correlation Coefficient
where is the sample mean of the explanatory variable sx is the sample standard deviation of the explanatory variable is the sample mean of the response variable sy is the sample standard deviation of the response variable n is the number of individuals in the sample 10

11. Properties of the Linear Correlation Coefficient
The linear correlation coefficient is always between –1 and 1, inclusive. That is, –1 ≤ r ≤ 1.
If r = + 1, then a perfect positive linear relation exists between the two variables.
If r = –1, then a perfect negative linear relation exists between the two variables.
The closer r is to +1, the stronger the evidence is of a positive association between the two variables.
The closer r is to –1, the stronger the evidence is of a negative association between the two variables. 11

12. If r is close to 0, then little or no evidence exists of a linear relation between the two variables. So r close to 0 does not imply no relation, just no linear relation.
The linear correlation coefficient is a unitless measure of association. So the unit of measure for x and y plays no role in the interpretation of r.
The correlation coefficient is not resistant. Therefore, an observation that does not follow the overall pattern of the data could affect the value of the linear correlation coefficient. 12

13. 13

14. 14

15. EXAMPLE
Draw a scatter plot of data. Determine the linear correlation coefficient of the following data by hand. X Y 1 18 3 13 7 4 15

16. EXAMPLE
Determine the linear correlation coefficient between club head speed and distance of the following data using technology.
Club Head Speed (mph)
Distance (yards)
100
257
102
264
103
274
101
266
105
277
263 99
258
275 16

17. Determine the linear correlation coefficient of the drilling data.
EXAMPLE Determining the Linear Correlation Coefficient
Determine the linear correlation coefficient of the drilling data.
0.773 17

18. Testing for a Linear Relation
Step 1 Determine the absolute value of the correlation coefficient.
Step 2 Find the critical value in Table II from Appendix A for the given sample size.
Step 3 If the absolute value of the correlation coefficient is greater than the critical value, we say a linear relation exists between the two variables. Otherwise, no linear relation exists. 18

19. EXAMPLE Does a Linear Relation Exist?
Determine whether a linear relation exists between time to drill five feet and depth at which drilling begins. Comment on the type of relation that appears to exist between time to drill five feet and depth at which drilling begins.
The correlation between drilling depth and time to drill is The critical value for n = 12 observations is Since > 0.576, there is a positive linear relation between time to drill five feet and depth at which drilling begins. 19

20. EXAMPLE Does a Linear Relation Exist?
Determine whether a linear relation exists between club head speed and distance traveled.
The correlation between club head speed and distance is The critical value for n = 8 observations is Since > 0.707, there is a positive linear relation between time to club head speed and distance. 20

21. According to data obtained from the Statistical Abstract of the United States, the correlation between the percentage of the female population with a bachelor’s degree and the percentage of births to unmarried mothers since 1990 is
Does this mean that a higher percentage of females with bachelor’s degrees causes a higher percentage of births to unmarried mothers? 21

22. Certainly not! The correlation exists only because both percentages have been increasing since It is this relation that causes the high correlation. In general, time series data (data collected over time) may have high correlations because each variable is moving in a specific direction over time (both going up or down over time; one increasing, while the other is decreasing over time).
When data are observational, we cannot claim a causal relation exists between two variables. We can only claim causality when the data are collected through a designed experiment. 22

23. Another way that two variables can be related even though there is not a causal relation is through a lurking variable.
A lurking variable is related to both the explanatory and response variable.
For example, ice cream sales and crime rates have a very high correlation. Does this mean that local governments should shut down all ice cream shops? No! The lurking variable is temperature. As air temperatures rise, both ice cream sales and crime rates rise. 23

24. EXAMPLE Lurking Variables in a Bone Mineral Density Study
Because colas tend to replace healthier beverages and colas contain caffeine and phosphoric acid, researchers Katherine L. Tucker and associates wanted to know whether cola consumption is associated with lower bone mineral density in women. The table lists the typical number of cans of cola consumed in a week and the femoral neck bone mineral density for a sample of 15 women. The data were collected through a prospective cohort study. 24

25. EXAMPLE Lurking Variables in a Bone Mineral Density Study
The figure on the next slide shows the scatter diagram of the data. The correlation between number of colas per week and bone mineral density is –0.806.The critical value for correlation with n = 15 from Table II in Appendix A is Because |–0.806| > 0.514, we conclude a negative linear relation exists between number of colas consumed and bone mineral density. Can the authors conclude that an increase in the number of colas consumed causes a decrease in bone mineral density? Identify some lurking variables in the study. 25

26. EXAMPLE Lurking Variables in a Bone Mineral Density Study26

27. EXAMPLE Lurking Variables in a Bone Mineral Density Study
In prospective cohort studies, data are collected on a group of subjects through questionnaires and surveys over time. Therefore, the data are observational. So the researchers cannot claim that increased cola consumption causes a decrease in bone mineral density.
Some lurking variables in the study that could confound the results are:
body mass index
height
smoking
alcohol consumption
calcium intake
physical activity 27

28. EXAMPLE Lurking Variables in a Bone Mineral Density Study
The authors were careful to say that increased cola consumption is associated with lower bone mineral density because of potential lurking variables. They never stated that increased cola consumption causes lower bone mineral density. 28

29. Least-squares Regression
Section
Least-squares Regression
4.2

30. Let’s draw a line through a residual plot through a scatter diagram of a data set to try to predict data points. 30

31. residual = observed y – predicted y = 5.2 – 4.75 = 0.45
The difference between the observed value of y and the predicted value of y is the error, or residual.
residual = observed y – predicted y
= 5.2 – 4.75
= 0.45
(3, 5.2) }
residual = observed y – predicted y
= 5.2 – 4.75
= 0.45 31

32. Least-Squares Regression Criterion
The least-squares regression line is the line that minimizes the sum of the squared errors (or residuals). This line minimizes the sum of the squared vertical distance between the observed values of y and those predicted by the line , (“y-hat”). We represent this as
“ minimize Σ residuals2 ”. 32

33. The Least-Squares Regression Line
The equation of the least-squares regression line is given by
where
is the slope of the least-squares regression line
where
is the y-intercept of the least-squares regression line 33

34. The Least-Squares Regression Line
Note: is the sample mean and sx is the sample standard deviation of the explanatory variable x ; is the sample mean and sy is the sample standard deviation of the response variable y. 34

35. Using the drilling data Find the least-squares regression line.
EXAMPLE Finding the Least-squares Regression Line
Using the drilling data
Find the least-squares regression line.
Predict the drilling time if drilling starts at 130 feet.
Is the observed drilling time at 130 feet above, or below, average.
Draw the least-squares regression line on the scatter diagram of the data. 35

36. We agree to round the estimates of the slope and intercept to four decimal places.
(b)
(c) The observed drilling time is 6.93 seconds. The predicted drilling time is seconds. The drilling time of 6.93 seconds is below average. 36

37. Interpretation of Slope:
(d)
Interpretation of Slope:
The slope of the regression line is For each additional foot of depth we start drilling, the time to drill five feet increases by minutes, on average. 37

38. Interpretation of the y-Intercept: The y-intercept of the regression line is To interpret the y-intercept, we must first ask two questions:
1. Is 0 a reasonable value for the explanatory variable? 2. Do any observations near x = 0 exist in the data set?
A value of 0 is reasonable for the drilling data (this indicates that drilling begins at the surface of Earth. The smallest observation in the data set is x = 35 feet, which is reasonably close to 0. So, interpretation of the y-intercept is reasonable.
The time to drill five feet when we begin drilling at the surface of Earth is minutes. 38

39. Find the least-squares regression line for the data.
EXAMPLE
Find the least-squares regression line for the data.
Draw the least-squares regression line on the scatter diagram of the data.
Predict the distance a golf ball will travel when hit with a club-head speed of 103 mph.
Determine the residual for the predicted value in c. Is the distance in table above average or below average among all balls hit with a swing speed of 103 mph?
Club Head Speed (mph)
Distance (yards)
100
257
102
264
103
274
101
266
105
277
263 99
258
275 39

40. If the least-squares regression line is used to make predictions based on values of the explanatory variable that are much larger or much smaller than the observed values, we say the researcher is working outside the scope of the model. Never use a least-squares regression line to make predictions outside the scope of the model because we can’t be sure the linear relation continues to exist. 40

41. Diagnostics on the Least-squares Regression Line
Section
Diagnostics on the Least-squares
Regression Line
4.3

42. If R2 = 0 the line has no explanatory value
The coefficient of determination, R2, measures the proportion of total variation in the response variable that is explained by the least-squares regression line.
The coefficient of determination is a number between 0 and 1, inclusive. That is, 0 < R2 < 1.
If R2 = 0 the line has no explanatory value
If R2 = 1 means the line explains 100% of the variation in the response variable. 42

43. The data to the right are based on a study for drilling rock
The data to the right are based on a study for drilling rock. The researchers wanted to determine whether the time it takes to dry drill a distance of 5 feet in rock increases with the depth at which the drilling begins. So, depth at which drilling begins is the predictor variable, x, and time (in minutes) to drill five feet is the response variable, y.
Source: Penner, R., and Watts, D.G. “Mining Information.” The American Statistician, Vol. 45, No. 1, Feb. 1991, p. 6. 43

44. Time 7 44

45. Mean Standard Deviation Depth 126.2 52.2 Time 6.99 0.781
Sample Statistics
Mean Standard Deviation
Depth
Time
Correlation Between Depth and Time: 0.773
Regression Analysis
The regression equation is
Time = Depth 45

46. The mean time to drill an additional 5 feet: 6.99 minutes
Suppose we were asked to predict the time to drill an additional 5 feet, but we did not know the current depth of the drill. What would be our best “guess”?
ANSWER:
The mean time to drill an additional 5 feet: minutes 46

47. Now suppose that we are asked to predict the time to drill an additional 5 feet if the current depth of the drill is 160 feet?
ANSWER:
Our “guess” increased from 6.99 minutes to 7.39 minutes based on the knowledge that drill depth is positively associated with drill time. 47

48. 48

49. The difference between the observed value of the response variable and the mean value of the response variable is called the total deviation and is equal to
The difference between the predicted value of the response variable and the mean value of the response variable is called the explained deviation and is equal to
The difference between the observed value of the response variable and the predicted value of the response variable is called the unexplained deviation and is equal to 49

50. Unexplained Deviation
Total Deviation
Unexplained Deviation
Explained Deviation = + 50

51. Unexplained Deviation
Total Deviation
Unexplained Deviation
Explained Deviation = +
𝑅 2 = 𝐸𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 𝑇𝑜𝑡𝑎𝑙 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 51

52. Because the linear correlation coefficient, r, is 0.773, we have that
EXAMPLE Determining the Coefficient of Determination
Find and interpret the coefficient of determination for the drilling data.
Because the linear correlation coefficient, r, is 0.773, we have that
R2 = = = 59.75%.
So, 59.75% of the variability in drilling time is explained by the least-squares regression line. 52

53. Find and interpret the coefficient of determination for golf data.
EXAMPLE
Find and interpret the coefficient of determination for golf data.
Since r = ,
R2 = = .8811=88.11% 53

54. Draw a scatter diagram for each of these data sets
Draw a scatter diagram for each of these data sets. For each data set, the variance of y is 54

55. Data Set A Data Set B Data Set C
Data Set A: 99.99% of the variability in y is explained by the least-squares regression line
Data Set B: 94.7% of the variability in y is explained by the least-squares regression line
Data Set C: 9.4% of the variability in y is explained by the least-squares regression line 55

56. To determine whether the variance of the residuals is constant.
Residuals play an important role in determining the adequacy of the linear model. In fact, residuals can be used for the following purposes:
To determine whether a linear model is appropriate to describe the relation between the predictor and response variables.
To determine whether the variance of the residuals is constant.
To check for outliers. 56

57. If a plot of the residuals against the predictor variable shows a discernable pattern, such as a curve, then the response and predictor variable may not be linearly related. 57

58. 58

59. EXAMPLE Is a Linear Model Appropriate?
A chemist has a 1000-gram sample of a radioactive material. She records the amount of radioactive material remaining in the sample every day for a week and obtains the following data.
Day Weight (in grams) 59

60. Linear correlation coefficient: – 0.99460

61. Weight (in g) 61

62. Linear model not appropriate (discernable pattern)
Response is Weight (in g)
Linear model not appropriate (discernable pattern) 62

63. This requirement is called constant error variance.
If a plot of the residuals against the explanatory variable shows the spread of the residuals increasing or decreasing as the explanatory variable increases, then a strict requirement of the linear model is violated.
This requirement is called constant error variance. 63

64. Does not display constant error variance64

65. A plot of residuals against the explanatory variable may also reveal outliers. These values will be easy to identify because the residual will lie far from the rest of the plot. 65

66. 66

67. EXAMPLE Residual Analysis
Draw a residual plot of the drilling time data. Comment on the appropriateness of the linear least-squares regression model. 67

68. 68

69. An influential observation is an observation that significantly affects the least-squares regression line’s slope and/or y-intercept, or the value of the correlation coefficient. 69

70. Explanatory, x
Influential observations typically exist when the point is an outlier relative to the values of the explanatory variable. So, Case 3 is likely influential. 70

71. Influence is affected by two factors:
(1) the relative vertical position of the observation (residuals) and
(2) the relative horizontal position of the observation (leverage). 71

72. EXAMPLE Influential Observations
Suppose an additional data point is added to the drilling data. At a depth of 300 feet, it took minutes to drill 5 feet. Is this point influential? 72

73. 73

74. With influential
Without influential 74