1. ANOVA and Regression
Brian Healy, PhD

2. Objectives ANOVA Introduction to regression Multiple comparisons
Relationship to correlation/t-test

3. Comments from reviews Please fill them out because I read them
More examples and not just MS
More depth on technical details/statistical theory/equations
First time ever!!
I have made slides from more in depth courses available on-line so that you have access to formulas for t-test, ANOVA, etc.
Talks too fast for non-native speakers

4. Review Types of data p-value Steps for hypothesis test
How do we set up a null hypothesis?
Choosing the right test
Continuous outcome variable/dichotomous explanatory variable: Two sample t-test

5. Steps for hypothesis testing
State null hypothesis
State type of data for explanatory and outcome variable
Determine appropriate statistical test
State summary statistics
Calculate p-value (stat package)
Decide whether to reject or not reject the null hypothesis
NEVER accept null
Write conclusion

6. Example
In previous class, two groups were compared on a continuous outcome
What if we have more than two groups?
Ex. A recent study compared the intensity of structures on MRI in normal controls, benign MS patients and secondary progressive MS patients
Question: Is there any difference among these groups?

8. Two approaches Compare each group to each other group using a t-test
Problem with multiple comparisons
Complete global comparison to see if there is any difference
Analysis of variance (ANOVA)
Good first step even if eventually complete pairwise comparisons

9. Types of analysis-independent samples
Outcome
Explanatory
Analysis
Continuous
Dichotomous
t-test, Wilcoxon test
Categorical
ANOVA, linear regression
Correlation, linear regression
Chi-square test, logistic regression
Logistic regression
Time to event
Log-rank test

10. Global test-ANOVA
As a first step, we can compare across all groups at once
The null hypothesis for ANOVA is that the means in all of the groups are equal
ANOVA compares the within group variance and the between group variance
If the patients within a group are very alike and the groups are very different, the groups are likely different

12. Hypothesis test H0: meannormal=meanBMS=meanSPMS
Outcome variable: continuous
Explanatory variable: categorical
Test: ANOVA
meannormal=0.41; meanBMS= 0.34; meanSPMS=0.30
Results: p=0.011
Reject null hypothesis
Conclusion: At least one of the groups is significantly different than the others

14. Technical aside
Our F-statistic is the ratio of the between group variance and the within group variance
This ratio of variances has a known distribution (F-distribution)
If our calculated F-statistic is high, the between group variance is higher than the within group variance, meaning the differences between the groups are not likely due to chance
Therefore, the probability of the observed result or something more extreme will be low (low p-value)

15. This is the distribution under the null
This small shaded region is the part of the distribution that is equal to or more extreme than the observed value.
The p-value!!!

16. Now what The question often becomes which groups are different
Possible comparisons
All pairs
All groups to a specific control
Pre-specified comparisons
If we do many tests, we should account for multiple comparisons

17. Type I error
Type I error is when you reject the null hypothesis even though it is true (a=P(reject H0|H0 is true))
We accept making this error 5% of the time
If we run a large experiment with 100 tests and the null hypothesis was true in each case, how many times would we expect to reject the null?

18. Multiple comparisons For this problem, three comparisons
NC vs. BMS; NC vs. SPMS; BMS vs. SPMS
If we complete each test at the 0.05 level, what is the chance that we make a type I error?
P(reject at least 1 | H0 is true) = a
P(reject at least 1 | H0 is true) = 1- P(fail to reject all three| H0 is true) = = 0.143
Inflated type I error rate
Can correct p-value for each test to maintain experiment type I error

19. Bonferroni correction
The Bonferroni correction multiples all p-values by the number of comparisons completed
In our experiment, there were 3 comparisons, so we multiply by 3
Any p-value that remains less than 0.05 is significant
The Bonferroni correction is conservative (it is more difficult to obtain a significant result than it should be), but it is an extremely easy way to account for multiple comparisons.
Can be very harsh correction with many tests

20. Other corrections All pairwise comparisons All groups to a control
Tukey’s test
All groups to a control
Dunnett’s test
MANY others
False discovery rate

21. Example
For our three-group comparison, we compare each and get the following results from Tukey’s test
Groups
Mean diff
p-value
Significant
NC vs. BMS
0.075
0.10
NC vs. SPMS
0.114
0.012 *
BMS vs. SPMS
0.039
0.60

23. Questions to ask yourself
What is the null hypothesis?
We would like to test the null hypothesis at the 0.05 level
If well defined prior to the experiment, the correction for multiple comparison if necessary will be clear
Hypothesis generating vs. hypothesis testing

24. Conclusions
If you are doing a multiple group comparison, always specify before the experiment which comparisons are of interest if possible
If the null hypothesis is that all the groups are the same, test global null using ANOVA
Complete appropriate additional comparisons with corrections if necessary
No single right answer for every situation

25. Types of analysis-independent samples
Outcome
Explanatory
Analysis
Continuous
Dichotomous
t-test, Wilcoxon test
Categorical
ANOVA, linear regression
Correlation, linear regression
Chi-square test, logistic regression
Logistic regression
Time to event
Log-rank test

26. Correlation
Is there a linear relationship between IL-10 expression and IL-6 expression?
The best graphical display for this data is a scatter plot

27. Correlation
Definition: the degree to which two continuous variables are linearly related
Positive correlation- As one variable goes up, the other goes up (positive slope)
Negative correlation- As one variable goes up, the other goes down (negative slope)
Correlation (r) ranges from -1 (perfect negative correlation) to 1 (perfect positive correlation)
A correlation of 0 means that there is no linear relationship between the two variables

29. Hypothesis test
H0: correlation between IL-10 expression and IL-6 expression=0
Outcome variable: IL-6 expression- continuous
Explanatory variable: IL-10 expression- continuous
Test: correlation
Summary statistic: correlation=0.51
Results: p=0.011
Reject null hypothesis
Conclusion: A statistically significant correlation was observed between the two variables

31. Technical aside-correlation
The formal definition of the correlation is given by:
Note that this is dimensionless quantity
This equation shows that if the covariance between the two variables is the same as the variance in the two variables, we have perfect correlation because all of the variability in x and y is explained by how the two variables change together

32. How can we estimate the correlation?
The most common estimator of the correlation is the Pearson’s correlation coefficient, given by:
This is a estimate that requires both x and y are normally distributed. Since we use the mean in the calculation, the estimate is sensitive to outliers.

33. Distribution of the test statistic
The standard error of the sample correlation coefficient is given by
The resulting distribution of the test statistic is a t-distribution with n-2 degrees of freedom where n is the number of patients (not the number of measurements)

34. Regression-Everything in one place
All analyses we have done to this point can be completed using regression!!!

35. Quick math review As you remember, the equation of a line is y=mx+b
For every one unit increase in x, there is an m unit increase in y
b is the value of y when x is equal to zero

36. Picture Does there seem to be a linear relationship in the data?
Is the data perfectly linear?
Could we fit a line to this data?

37. How do we find the best line?
Linear regression tries to find the best line (curve) to fit the data Let’s look at three candidate lines
Which do you think is the best?
What is a way to determine the best line to use?

38. What is linear regression?
The method of finding the best line (curve) is least squares, which minimizes the distance from the line for each of points
The equation of the line is y=1.5x + 4

39. Example
For our investigation of the relationship between IL-10 and IL-6, we can set up a regression equation
b0 is the expression of IL-6 when IL-10=0 (intercept)
b1 is the change in IL-6 for every 1 unit increase in IL-10 (slope)
ei is the residual from the line

40. The final regression equation is The coefficients mean
the estimate of the mean expression of IL-6 for a patient with IL-10 expression=0 (b0)
an increase of one unit in IL-10 expression leads to an estimated increase of 0.63 in the mean expression of IL-6 (b1)

41. Tough question
In our correlation hypothesis test, we wanted to know if there was an association between the two measures
If there was no relationship between IL-10 and IL-6 in our system, what would happen to our regression equation?
No effect means that the change in IL-6 is not related to the change in IL-10
b1=0
Is b1 significantly different than zero?

42. Hypothesis test
H0: no relationship between IL-6 expression and IL-10 expression, b1 =0
Outcome variable: IL-6- continuous
Explanatory variable: IL-10- continuous
Test: linear regression
Summary statistic: b1 = 0.63
Results: p=0.011
Reject null hypothesis
Conclusion: A significant correlation was observed between the two variables

44. Wait a second!! Let’s check something
p-value from correlation analysis = 0.011
p-value from regression analysis = 0.011
They are the same!!
Regression leads to same conclusion as correlation analysis
Other similarities as well from models

45. Technical aside-Estimates of regression coefficients
Once we have solved the least squares equation, we obtain estimates for the b’s, which we refer to as
To test if this estimate is significantly different than 0, we use the following equation:

46. Assumptions of linear regression
Linearity
Linear relationship between outcome and predictors
E(Y|X=x)=b0 + b1x1 + b2x22 is still a linear regression equation because each of the b’s is to the first power
Normality of the residuals
The residuals, ei, are normally distributed, N(0, s2)
Homoscedasticity of the residuals
The residuals, ei, have the same variance
Independence
All of the data points are independent
Correlated data points can be taken into account using multivariate and longitudinal data methods

47. Linear regression with dichotomous predictor
Linear regression can also be used for dichotomous predictors, like sex
Last class we compared relapsing MS patients to progressive MS patients
To do this, we use an indicator variable, which equals 1 for relapsing and 0 for progressive. The resulting regression equation for expression is

48. Interpretation of model
The meaning of the coefficients in this case are
b0 is the estimate of the mean expression when R=0, in the progressive group
b0 + b1 is the estimate of the mean expression when R=1, in the relapsing group
b1 is the estimate of the mean increase in expression between the two groups
The difference between the two groups is b1
If there was no difference between the groups, what would b1 equal?

49. Mean in wildtype=b0 Difference between groups=b1
Mean in Progressive group=b0

50. Hypothesis test Null hypothesis: meanprogressive=meanrelapsing (b1=0)
Explanatory: group membership- dichotomous
Outcome: cytokine production-continuous
Test: Linear regression
b1=6.87
p-value=0.199
Fail to reject null hypothesis
Conclusion: The difference between the groups is not statistically significant

51. T-test
As hopefully you remember, you could have tested this same null hypothesis using a two sample t-test
Very similar result to previous class
If we would have assumed equal variance for our t-test, we would have gotten to the same result!!!
ANOVA results can also be tested using regression using more than one indicator

52. Multiple regression
A large advantage of regression is the ability to include multiple predictors of an outcome in one analysis
A multiple regression equation looks just like a simple regression equation.

53. Example
Brain parenchymal fraction (BPF) is a measure of disease severity in MS
We would like to know if gender has an effect on BPF in MS patients
We also know that BPF declines with age in MS patients
Is there an effect of sex on BPF if we control for age?

54. Blue=males; Red=females

55. Blue=males; Red=females

56. Is age a potential confounder?
We know that age has an effect on BPF from previous research
We also know that male patients have a different disease course than female patients so the age at time of sampling may also be related to sex
Age
Sex
BPF

57. Model
The multiple linear regression model includes a term for both age and sex
What are the values genderi takes on?
genderi=0 if the patient is female
genderi=1 if the patient is male

58. Expression Females: Males: What is different about the equations?
BPFi = b0+ b2*agei+ei
Males:
BPFi = (b0+ b1)+ b2*agei+ei
What is different about the equations?
Intercept
What is the same?
Slope
This model allows an effect of gender on the intercept, but not on the change with age

59. Interpretation of coefficients
The meaning of each coefficient
b0: the average BPF when age is 0 and the patient is female
b1: the average difference in BPF between males and female, HOLDING AGE CONSTANT
b2: the average increase in BPF for a one unit increase in age, HOLDING GENDER CONSTANT
Note that the interpretation of the coefficient requires mention of the other variables in the model

60. Estimated coefficients
Here is the estimated regression equation
The average difference between males and females is holding age constant
For every one unit increase in age, the mean BPF decreases units holding sex constant
Are either of these effects statistically significant?
What is the null hypothesis?

61. Hypothesis test H0: No effect of sex, controlling for age b1 =0
Continuous outcome, continuous predictor
Linear regression controlling for sex
Summary statistic: b1 =0.017
p-value=0.37
Since the p-value is more than 0.05, we fail to reject the null hypothesis
We conclude that there is no significant association between sex and BPF controlling for age

62. Hypothesis test H0: No effect of age, controlling for sex b2 =0
Continuous outcome, continuous predictor
Linear regression controlling for sex
Summary statistic: b2 =
p-value=0.00 4
Since the p-value is less than 0.05, we reject the null hypothesis
We conclude that there is a significant association between age and BPF controlling for sex

63. Estimated effect of sex
p-value for sex
Estimated effect of age
p-value for age

65. Conclusions
Although there was a marginally significant association of sex and BPF, this association was not significant after controlling for age
The significant association between age and BPF remained statistically significant after controlling for sex

66. What we learned (hopefully)
ANOVA
Correlation
Basics of regression