1. Simple Linear Regression
Correlation
and
Simple Linear Regression
© Scott Evans, Ph.D.

2. Correlation
Used to assess the relationship between two continuous variables.
Example: weight and number of hours of exercise per week
Scatterplots are useful graphical tools to visualize the relationship between two continuous variables.
Plot of y vs. x
Plots are generally recommended as they are informative, easy to understand, and do not cost anything from a statistical sense.
© Scott Evans, Ph.D.

3. Correlation Coefficient
Ranges from –1 (perfect negative correlation) to 1 (perfect positive correlation)
Correlation of –1 means that all of the data lie in a straight line with negative (but further unknown) slope.
Correlation of 1 means that all of the data lie in a straight line with positive (but further unknown) slope.
Correlation of 0, means that the variables are not correlated. A plot of the data would reveal a cloud with no discernable pattern.
© Scott Evans, Ph.D.

4. Correlation Coefficient
 denotes the population correlation coefficient (a parameter)
r denotes the sample correlation coefficient (a statistic)
Measures the strength of a linear relationship
© Scott Evans, Ph.D.

5. Correlation Coefficient
Can conduct hypothesis tests for  using the t distribution.
H0: =0 is often tested (but is not limiting)
CIs for  may also be obtained.
© Scott Evans, Ph.D.

6. Correlation Coefficient
Two major correlation coefficients:
Pearson correlation: parametric
Sensitive to extreme observations
Spearman correlation: nonparametric
Based on ranks
Robust to extreme observations and thus is recommended particularly when “outliers” are present.
I use Spearman almost exclusively
© Scott Evans, Ph.D.

7. Correlation Warnings:
Correlation does NOT necessarily imply causation.
Correlation does NOT measure the magnitude of the regression slope.
© Scott Evans, Ph.D.

8. Simple Linear Regression
Used to describe and estimate the relationship between 2 continuous variables
We attempt to characterize this relationship with two parameters: (1) an intercept, and (2) a slope.
Has several assumptions
© Scott Evans, Ph.D.

9. Simple Linear Regression
The model: yi = 0 + 1xi
yi is the dependent variable
xi is the independent variable (predictor)
0 is the y-intercept
1 is the slope of the line
It describes how steep the line is and which way it leans
It is also the “effect” (e.g., a treatment effect) on y of a 1 unit change in x
Note that if there was no association between x and y, then 1 will be close to 0 (i.e., x has no effect on y)
© Scott Evans, Ph.D.

10. Simple Linear Regression
Plot the data first: Scatterplots
This enables you to see the relationship between the 2 variables
If the relationship is non-linear (as evidenced by the plot) then a simple linear regression model is the wrong approach.
© Scott Evans, Ph.D.

11. Simple Linear Regression
Uses the method of least squares
Identifies the line that minimizes the sum of squared deviations between observed and predicted values.
Produces an ANOVA table with an F test
Note that F simplifies to t when the numerator d.f. = 1.
© Scott Evans, Ph.D.

12. Simple Linear Regression
Can be used to estimate the correlation between x and y.
Hypothesis tests and CIs may be obtained for 1
One may make predictions of the effect of changes in x on y.
© Scott Evans, Ph.D.

13. Simple Linear Regression
We desire to capture all of the structure in the data with the model (finding the systematic component)
Thus leaving only random errors.
Thus if we plotted the errors, we would hope that no discernable pattern exists
If a pattern exists, then we have not captured all of the systematic structure in the data, and we should look for a better model.
© Scott Evans, Ph.D.

14. Multiple Regression Can incorporate (and control for) many variables
A single (continuous) dependent variable
Multiple independent variables (predictors)
These variables may be of any scale (continuous, nominal, or ordinal)
Outcome = function of many variables (e.g., sex, age, race, smoking status, exercise, education, treatment, genetic factors, etc.)
© Scott Evans, Ph.D.

15. Multiple Regression
Multiple regression can estimate the effect of each of these variables while controlling for (adjusting for) the effects of other (potentially confounding variables) in the model
Confounding occurs when the effect of a variable of interest is distorted when you do not control for the effect of another “confounding” variable.
© Scott Evans, Ph.D.

16. Multiple Regression
Interactions (effect modification) may be investigated
The effect of one variable depends on the level of another variable.
Example: the effect of treatment may depend on whether you are male or female
© Scott Evans, Ph.D.

17. Multiple Regression
Indicator variables are created and used for categorical variables.
Selection procedures can help a research choose a final model from a shopping list of potential independent variables.
Backwards
Forwards
Stepwise
Best subsets
© Scott Evans, Ph.D.

18. Multiple Regression
Models require an assessment of model adequacy and goodness of fit.
Examination of residuals (comparison of observed vs. predicted values).
© Scott Evans, Ph.D.

19. Other Regression Models
Generalized Linear Models (GLMs)
A function of the dependent variable is a linear function of the covariates
Multiple regression (link = identity)
Logistic Regression
Used when the dependent variable is binary
Very common in public health/medical studies (e.g., disease vs. no disease)
Poisson Regression
Used when the dependent variable is a count
© Scott Evans, Ph.D.

20. Other Regression Models
(Cox) Proportional Hazards (PH) Regression
Used when the dependent variable is a “event time” with censoring
© Scott Evans, Ph.D.