1. Class 5: Thurs., Sep. 23
Example of using regression to make predictions and understand the likely errors in the predictions: salaries of teachers and experience
Normal distribution calculations
R squared
Checking the assumptions of the simple linear regression model: residual plots.

2. Teachers’ Salaries and Dating
In U.S. culture, it is usually considered impolite to ask how much money a person makes.
However, suppose that you are single and are interested in dating a particular person.
Of course, salary isn’t the most important factor when considering whom to date but it certainly is nice to know (especially if it is high!)
In this case, the person you are interested in happens to be a high school teacher, so you know a high salary isn’t an issue.
Still you would like to know how much she or he makes, so you take an informal survey of 11 high school teachers that you know.

5. You happen to know that the person you are interested in has been teaching for 8 years.
How can you use this information to better predict your potential date’s salary?
Regression Analysis to the Rescue!
You go back to each of the original 11 teachers you surveyed and ask them for their years of experience.
Simple Linear Regression Model: E(Y|X)= , the distribution of Y given X is normal with mean and standard deviation

7. Predicted salary of your potential date who has been a teacher for 8 years = Estimated Mean salary for teachers of 8 years = *8 = $54,100
How far off will your estimate typically be? Root mean square error = Estimated standard deviation of Y|X = $4,
Notice that the typical error of your estimate of teacher salary using experience, $4,610.93, is less than that of using only information on mean teacher salary, $6,
Regression analysis enables you to better predict your potential date’s salary.

8. More Information About Your Potential Date’s Salary
From the regression model, you predict that your potential date’s salary is $54,100 and the typical error you expect to make in your prediction is $4,611.
Suppose you want to know an interval that will most of the time (say 95% of the time) contain your date’s salary? What’s the chance that your date will make more than $60,000? What’s the chance that your date will make less than $50,000?
We can answer these questions by using the fact that under the simple linear regression model, the distribution of Y|X is normal, here the subpopulation of teachers with 8 years of experience has a normal distribution with mean $54,100 and standard deviation $4,611.

9. 95% interval: For the subpopulation of teachers with 8 years of experience, 95% of the salaries will be within two SDs of the mean. An interval that will contain a randomly chosen teacher’s salary with 8 years of experience 95% of the time is: $54, *$4,611 = ($44,878,$63,322).
What’s the probability that your date will make more than $60,000? If you don’t have any additional information about your date other than his or her number of years of teaching, we can assume that your date is a random draw from the subpopulation of teachers with 8 years of teaching.
According to the simple linear regression model, the subpopulation of teachers with 8 years of experience is estimated to have a normal distribution with mean $54,100 and standard deviation $4,611.

10. Properties of the Normal Distribution (Section 1.3)
Suppose a variable Y has a normal distribution with mean and standard deviation Then
follows a standard normal distribution.
Then the probability that Y is greater than a number c equals
where Z equals standard normal distribution with mean 0 and SD 1.
The probabilities for a standard normal distribution can be found in Table A.
Review Section 1.3 on using the normal tables.

11. Probability that a teacher with 8 years of experience has salary > $60,000:
Probability that a teacher with 8 years of experience has salary between $52,000 and $56,000:

12. R Squared
How much better predictions of your potential date’s salary does the simple linear regression model provide than just using the mean teacher’s salary?
This is the question that R squared addresses.
R squared: Number between 0 and 1 that measures how much of the variability in the response the regression model explains.
R squared close to 0 means that using regression for predicting Y|X isn’t much better than mean of Y, R squared close to 1 means that regression is much better than the mean of Y for predicting Y|X.

13. R Squared Formula
Total sum of squares = = the sum of squared prediction errors for using sample mean of Y to predict Y
Residual sum of squares = , where is the prediction of Yi from the least squares line.

14. What’s a good R squared?
As with correlation, it depends on the context.
A good R2 depends on the context. In precise laboratory work, R2 values under 90% might be too low, but in social science contexts, when a single variable rarely explains great deal of variation in response, R2 values of 50% may be considered remarkably good.
The best measure of whether the regression model is providing predictions of Y|X that are accurate enough to be useful is the root mean square error, which tells us the typical error in using the regression to predict Y from X.

15. Checking the model
The simple linear regression model is a great tool but its answers will only be useful if it is the right model for the data. We need to check the assumptions before using the model.
Assumptions of the simple linear regression model:
Linearity: The mean of Y|X is a straight line.
Constant variance: The standard deviation of Y|X is constant.
Normality: The distribution of Y|X is normal.
Independence: The observations are independent.

16. Checking that the mean of Y|X is a straight line
Scatterplot: Look at whether the mean of Y given X appears to increase or decrease in a straight line.

17. Residual Plot
Residuals: Prediction error of using regression to predict Yi for observation i:
, where
Residual plot: Plot with residuals on the y axis and the explanatory variable (or some other variable on the x axis.

18. Residual Plot in JMP: After doing Fit Line, click red triangle next to Linear Fit and then click Plot Residuals.
What should the residual plot look like if the simple linear regression model holds? Under simple linear regression model, the residuals
should have approximately a normal distribution with mean zero and a standard deviation which is the same for all X.
Simple linear regression model: Residuals should appear as a “swarm” of randomly scattered points about their (which is always zero).
A pattern in the residual plot that for a certain range of X the residuals tend to be greater than zero or tend to be less than zero indicates that the mean of Y|X is not a straight line.

20. Summary
Normal distribution can be used to calculate probability that Y takes on certain values given X
R squared: measure of how much regression improves on ignoring X when predicting Y.
Assumptions of simple linear regression model must be checked in order for model to be used. Residual plots can be used to check the linearity assumption.
Tuesday’s class: Section 2.4 (more on checking assumptions, outliers and influential points, lurking variables).