1. Mixed models and their uses in meta-analysis
The difference between analysing individual observational units and parameter estimates
What I know, and don’t know
Weighted regression
Weighted regression in meta-analysis
Fixed and random components of statistical models: basics of mixed models
Weighted regression with mixed models
example

2. Choose the line (i.e. choose the values of ) that minimize the absolute distances of each point from it.
Least squares
Maximum likelihood

3. Precision of our measurement of obs 6
Do we still want to treat e6 and e10 the same way?

4. Assign a lower weight on those observations that are less precise

5. Normally, when we conduct our own study, we take physical measurements of (x,y) on each « observational unit »
If we use the same method to take these measurements then the precision (k) will be the same for all.
Minimizing or minimizing will give the same
result.
Usually don’t need to worry about weighting observations
But you do if different observations are measured with different levels of precision.

6. Analysis vs. Meta-analysis
Single analysis:
Define your statistical population (all those « observational units » that could be chosen given your sampling method)
Randomly sample n observational units  from this statistical population
Measure (x, y) on each of the n observational units
Analyse with a statistical model
Precision of measurements usually the same for all observational units
Meta-analysis:
Define your statistical population (all those « publications » that could be chosen given your sampling method)
Randomly sample n publications  from this statistical population
Obtain measures (x, y) from each of the n publications
Analyse with a statistical model
Precision of measurements usually different in each publication

7. With weighting of 1/variance
Without weighting:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)
x *
---
Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: on 9 degrees of freedom
Multiple R-squared: , Adjusted R-squared:
F-statistic: on 1 and 9 DF, p-value:
x y vars
With weighting of 1/variance
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)
x ***
---
Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: on 9 degrees of freedom
Multiple R-squared: , Adjusted R-squared:

8. A quick review of some basic concepts in statistical modelling …

9. A statistical model contains « parameters » that have to be estimated
A statistical model contains « parameters » that have to be estimated. What is a parameter estimate?
A normal distribution has two « parameters ».
The true values of these are unknown, but can be estimated, with some degree of error, from the observations. The parameter sigma is completely determined once we know alpha, beta and x.

10. Least-squares estimate of slope:
The precision (standard error) of the slope estimate:
How much the y’s vary from the line
Sample size
How much the x’s vary

11. Sample size
Vary range of x’s
Residual variation
simulation
Changing sample size
Changing range of x
Changing residual variation of y

12. If we repeat our study many times in exactly the same way, our estimates will randomly vary around the true value following approximately a normal distribution (central limit theorem)
The standard error of a parameter is a quantification of the precision of estimation of that parameter.
It tells us how much our estimate of this parameter will vary if we were to repeat our estimates many times.
Larger standard errors mean less precision in our estimate, because our estimate will tend to differ more (randomly, centered on the true value) from the true value.

13. Q: how does the slope of y~x change with temperature across studies?
Problem 1: we know that the precision of the estimates of the slope will differ across studies, depending on
The sample sizes used
The range of x values used
Everything besides temperature that can affect the residual variation of y’s given x.
Problem 2: we rarely have available the actual observations on each observational unit. We only have statistical summaries in the form of parameter estimates (here, slopes).
But, we have the parameter estimate (slopes, intercepts, means) and their standard errors (usually…).

14. Example: Does limb length decrease relative to body size as one goes further north (Allen’s rule)?
slope
temperature
We collect slope estimates (b) and their standard errors (se.b) from many different studies (one per study for now), and also get the mean annual temperature of the site in which the study was done.
Study
Slope
se.b
temperature 1
0.75
0.1 20 2
0.66
0.05 15 3
0.8
0.21 18
fit<-lm(slopes~temperature,data=dat,weights=1/se.b^2)

15. The weights argument can be used in a linear model
Now, what happens if you have more than one line (« observational unit ») for each study?
For instance, if studies report slope estimates for different species, or locations….
The parameter estimates within a given study will be correlated because they were taken by the same people, at the same time, in the same geographical locations, etc.

16. Calculating weights of composite effect variables.
Let’s say that we have chosen to measure our « effect » as the difference between two means (say, before and after):
Each of these two means will have its associated standard errors – s(m1) and s(m2) – but we need the standard error of the difference between these two means; i.e. s(E). Here is an approximate formula for combining standard errors (it assumes that the m’s are independent):

17. Standard errors of common composite functions
m1 & m2 are two parameter estimates (means, slopes, intercepts, correlation coefficients… and « k » is a constant

18. A simulation to show what happens if we ignore different amounts of measurement error (i.e. different levels of precision) between studies.
Model 1:
Actual data without measurement error:
X = 20 random values from N(mu=20,sigma=10)
Model 2:
We don’t have « y » but, rather, estimates of y (y1) that each have the same amount of measurement error (sigma=5)
Model 3:
We don’t have « y » but, rather, estimates of y (y2) that each have different amounts of measurement error (sigma=5)
y2=y +N(0,5)

19. Parameter
No measurement error
Constant measurement error
Variable measurement error
Ignore it
weights
Intercept
1.4468
3.0710 SE
2.3179
3.2849
36.481
3.3428 P
0.541
0.133
0.361
0.370
Slope
1.1265
1.2334
1.972
0.9946
0.1098
0.1555
1.727
0.2015 p
6e-9
3e-7
0.269
1e-4
SEresiduals
4.569
6.47
71.86
7.049 R2
0.854
0.775
0.068
0.575
True relationship:

20. Main message so far: IF
you are extracting « effect sizes » from different papers that are parameter estimates (means, slopes, correlation coefficients…) and not the actual direct measurements
you only have one « effect size » per paper
THEN
You must also get the standard errors of these effect sizes (which measures their measurement error)
And you must use these standard errors to weight each effect size so that papers with more precision (i.e. smaller standard errors) are given more weight.

21. Now, mixed models…
What happens if you are extracting « effect sizes » from different papers that are parameter estimates (means, slopes, correlation coefficients…) and not the actual direct measurements
And you only have more than one « effect size » per paper

22. We measure two variables, x and y, on some soil samples taken in the valley outside the field station.

23. This part applies to all observation: it is the fixed component
This part differs randomly for each observation: it is the random component

24. We measure two variables, x and y, on some soil samples taken in two locations: the valley and the hillside outside the field station.

25. Doesn’t seem right! Looks like two groups, each with a different slope and intercept
Typical analysis: ANCOVA

26. ANCOVA. Still has only one source of random variation
ANCOVA. Still has only one source of random variation. The differences between the two groups are not modelled as random differences.
Reference slope
Reference intercept
How much intercepts of each group differ from the reference intercept: « Treatment Contrasts »
How much slopes of each group differ from the reference slope: « Contrasts »

27. We randomly choose a series of locations in the park
We measure two variables, x and y, on randomly chosen soil samples taken in each location.
𝛼 𝑖 =𝛼+ 𝛿 𝑖 𝛽 𝑖 =𝛽+ 𝜙 𝑖
Relationship location i y
Average relationship b a x

28. …
i randomly chosen locations
j randomly chosen samples nested within each location
Equation for location i
Fixed effect random effect

29. Mixed models in R
Function lmer() in the lme4 package
Function lme() in the nlme package
If you use lmer(), you should also install the package « lmerTest »
Relationship location i y
Average relationship b a
lmer(y~x+(1|group),data=…)
lmer(y~x+(-1+x|group),data=…)
lmer(y~x+(x|group),data=…) x

31. …
i randomly chosen locations
j randomly chosen samples nested within each location
Simplest case: Variance components
Random variation of the mean values of y between locations
Random variation of the individual values of y within each location

32. Total random variation:
Proportion of random variation due to differences between locations
Proportion of random variation due to differences between observations within each location

33. Basic format of the lmer function
First example:
76 soils randomly chosen across southern Quebec
Each soil randomly divided into 4 pots
(2 harvests each pot; harvest dates NOT randomly chosen)
4 individuals per pot randomly planted
Go to R code…
#to use the lmer function for mixed models
library(lme4)
#to obtain probabilities for t-tests, F-tests etc.
library(lmerTest)
Basic format of the lmer function
fit<-lmer(Y~fixed effects +(random parameters of fixed part|nesting structure),data= ,…)

34. 76 soils across southern Quebec: 0.8184+0.000+6.4e-15
vc.model.wheat<-lmer(log(masse_en_g)~1+(1|sol)+(1|pot),data=dat)
summary(vc.model.wheat)
Linear mixed model fit by REML ['lmerMod']
Formula: log(masse_en_g) ~ 1 + (1 | sol) + (1 | pot)
Data: dat
REML criterion at convergence:
Scaled residuals:
Min Q Median Q Max
Random effects:
Groups Name Variance Std.Dev.
pot (Intercept) 0.000e e+00
sol (Intercept) 6.434e e-08
Residual e e-01
Number of obs: 1192, groups: pot, 304; sol, 76
Fixed effects:
Estimate Std. Error t value
(Intercept)
76 soils across southern Quebec: e-15
4 pots for each soil:
4 individuals per harvest:
Source of variation % added variation
Soils %
Pots %
Individuals ~100%
Mean ln(biomass) per plant
Almost all variation in ln(biomass) comes from plants growing in the same pot. What could explain this variation? These different plants within a pot have different ages at harvest.

35. Step 2: We include the plant ages (harvest dates)
Step 2: We include the plant ages (harvest dates). We didn’t randomly choose these harvest dates, rather we chose two dates, and so we include this as a fixed effect.
vc.model.wheat<-lmer(log(masse_en_g)~age_jour+(1|sol)+(1|pot),data=dat)
REML criterion at convergence: 727.5
Scaled residuals:
Min Q Median Q Max
Random effects:
Groups Name Variance Std.Dev.
pot (Intercept)
sol (Intercept)
Residual
Number of obs: 1192, groups: pot, 304; sol, 76
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) e e e <2e-16 ***
age_jour e e e <2e-16 ***
---
Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Problems: (1) this model doesn’t allow the slopes to randomly vary across soils (i.e. it assumes that all of these soils allow the same growth rate; (2) it does allow random intercepts. The intercept is the mean ln(mass) at time 0 (i.e. when the mass of the seedling when it begins emerging from the seed). This is unlikely to change between soils.

36. Step 3: There are two fixed harvest dates, so we include this as a fixed effect, fix the intercept (since this is the mean ln(biomass) at time 0, and shouldn’t vary across soils and pots), and allow random slopes
vc.model.wheat<-lmer(log(masse_en_g)~age_jour+(-1+age_jour|sol)+(-1+age_jour|pot),data=dat)
Random effects:
Groups Name Variance Std.Dev.
pot age.jour 3.343e
sol age.jour 2.583e
Residual e
Number of obs: 1192, groups: pot, 304; sol, 76
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) <2e-16 ***
age.jour <2e-16 ***
---

37. Nesting structures
Nested classification
Cross-classification (or « non-nested » classification)

38. Including weighting for differing precisons in meta-analysis:
lmer(y~x+(1|group),weights=SE2,…)
This is the opposite of the « weights » function in lm():
weights
an optional vector of ‘prior weights’ to be used in the fitting process. Should be NULL or a numeric vector. …. In particular, the diagonal of the residual covariance matrix is the squared residual standard deviation parameter sigma times the vector of inverse weights. Therefore, if the weights have relatively large magnitudes, then in order to compensate, the sigma parameter will also need to have a relatively large magnitude.

39. Exemple: Facultative adjustment of the offspring sex ratio and male attractiveness:
a systematic review and meta-analysis. Biol. Rev. (2017), 92, pp. 108–134.