2.4 Cautions about Correlation and Regression

Residuals (again!) Recall our discussion about residuals- what is a residual? The idea for line of best fit was to “minimize” the residuals. While this can always be done, when is the line of best fit a good fit (as opposed to some curve)?

Residual Plots As we hinted at in 2.3, the mean of the least squares residuals is always 0. This leads to the idea of a residual plot; that is, a scatterplot of the residuals against the explanatory variable. Residual plots help us assess the fit of a regression line.

We began Section 2.2 with the graph What does the residual plot look like?

No overall pattern means line is a good fit.

We also looked at this graph in section 2.3 Looks great!

But look at residual plot. Seems to be a pattern at first.

Consider the following data: XY Line of best fit is y = x. (2, 20) and (20,2) seem problematic.

XY Line of best fit is y = x

XY Line of best fit is y = x

All data Without (2,20) Without (20,2)

What was the point of all that? Eliminating a value which has a large deviation in y made for the best residual plot, and it didn’t change the least squares regression much. Eliminating a value which has a large deviation in x considerably changed the least squares regression line. An observation is influential for a calculation if removing it would drastically change the result of the calculation. Usually an extreme outlier in the x direction is considered influential but not an extreme outlier in the y direction. This leads to the truth that while correlation can imply causation, it does not necessarily.

Lurking Variable When we consider ice cream sales and drowning deaths these variables, they will show a positive and potentially statistically significant correlation. Certainly there is no causal relationship here, despite what the data implies. There is a third variable that is “lurking” behind all of this; namely summertime. A lurking variable is a variable that is not among the explanatory or response variables in a study and still may yet influence the interpretation of relationships among those variables.

2.5 Data Analysis for Two-Way Tables

Categorical Data Scatter plots are good for quantitative data, but what if we have categorical data? For example, suppose are interested in comparing the number of people in different kinds of colleges according to gender. We use a two-way table to present the data.

StatusMen (in thousands)Women (in thousands) Two-year college, full-time Two-year college, part-time Four-year college, full-time Four-year college, part-time Graduate school Vocation school Easy to read off data. Could have included a “total” row. This two-way table gives the actual numbers. Perhaps percentages would be more helpful. The following is a joint distribution.

StatusMenWomen Two-year college, full-time Two-year college, part-time Four-year college, full-time Four-year college, part-time Graduate school Vocation school This is a joint distribution. It is obtained by dividing the cell entry by the total sample size. Notice that the sum of all the values should be 1 (sometimes there is slight round-off error). If we are interested in the percentage of each variable, we use a marginal distribution.

StatusMenWomenTotal Two-year college, full-time /10421=18% Two-year college, part-time /10421= 7% Four-year college, full-time /10421=60% Four-year college, part-time /10421=6% Graduate school /10421=6% Vocation school /10421=3% Total4842/10421= 46% 5579/10421= 54% This is a marginal distribution. The marginal distribution in gender is given in the bottom row, and the marginal distribution in status is given in the rightmost column. Now a bar graph can help display the data. We’ll graph status of those in college.

StatusMenWomen Two-year college, full-time48%52% Two-year college, part-time46%54% Four-year college, full-time47%53% Four-year college, part-time39%61% Graduate school46%54% Vocation school54%46% This is a conditional distribution in status since we are fixing a status. We could have also looked at a conditional distribution in gender.

Simpson’s Paradox Simpson’s paradox occurs when the success of a group seems reversed when the groups are combined. Let’s look at an example.

Simpson’s Paradox cont Combined Dave Justice 104/411= /140= /551=.270 Derek Jeter 12/48= /582= /630=.310 Justice had a better batting average in 1995 and 1996 than Jeter, but Jeter had a better combined batting average than Justice. Why? Consider the following made-up table of batting averages.

Simpson’s Paradox cont Combined Dave Justice ½=.500 1/1= /3=.666 Derek Jeter 0/1=.00099/100=.99100/101=.990 Notice that the paradox can be attributed to the fact that Jeter’s number of at-bats in 1996 dominates. Notice that the paradox can be attributed to the fact that Jeter’s number of at-bats in 1996 dominates.