1. Mixed models Various types of models and their relation
Mixed effect models: simple case
Mixed effect models: Estimation of parameters
Tests of hypothesis
R functions for mixed effect models

2. Various forms of models and relation between them
Classical statistics (Observations are random, parameters are unknown constants)
LM: Assumptions 1) independent 2) Normal distribution, 3) constant unknown coefficients
Repeated measures: Assumptions 1) and 3) is modified
LMM: Assumptions 1) and 3) are modified
GLMM: Assumption 2) Exponential family and assumptions 1) and 3) are modified
GLM: assumption 2) Exponential family
Time series
Maximum likelihood: All assumptions can be modified
NLM: Can be applied to all
LM - Linear model
GLM - Generalised linear model
LMM - Linear mixed model
GLMM - Generalised linear mixed model
NLM - Non-linear model
Conceptual difference
Bayesian statistics: Coefficients as well as observations are random

3. Mixed effect models: motivation
In linear and generalised linear models we assumed that 1) observations are independent on each other and have the same variances 2) Distribution is normal; 3) Parameters are constant (in linear model case):
y = X+;  has Normal distribution N(0,2I);  is a vector unknown constants. This type of model is called fixed effect models.
Topic of the last lecture (Lecture 10: Generalised linear models) was about the effect of removing one of the assumptions, namely assumption that observation are from population with normal distribution. What happens if we remove assumption 1) and 3). Then problem becomes more complicated and in general we need nx(n+1)/2 number of parameters to describe covariance structure of observations. Mixed effect models deal with these type of problems. In general this type of models bring classical statistics to a new level and allows to tackle such problems as: clustered data, repeated measures, hierarchical data.

4. Mixed effect models: Example
Let us assume that we have a clinical trial. There is a drug. We want to test the effect of the different doses of the drug. We are interested only these dose levels. We randomly take n person and give to each of them one of the doses. Then the result of the experiment could be written:
yij=+i+ij
Where i is i-th dose, j is j-th person,  is average effect of the drug and  is effect of the drug specific to this particular dose,  is error. Our interest lies on effects of these doses and these doses alone. This type of model is fixed effect model.
Now let us assume these doses were tested in 20 different clinics. Clinics were chosen randomly. Then we can write the model:
yijk=+i+bj +cij ijk
i is i-th dose, j is j-th clinic, k is the k-th patient. Since doses are only those doses we are interested in they are fixed, 20 clinics have been chosen randomly form population of all clinics they are random. We can not guarantee that effect of clinic and effect of dose is additive that is why we add c - interaction between clinics and doses. Since clinics are random then c must be random also. This is an example of mixed effect model. To solve the problem we need to find estimations overall effect (), effects of dose () and distribution of clinics (distribution of b and c).

5. Mixed or random
It is often a challenging problem to decide if we should use fixed or mixed effect models. For example in drug and clinics case if we are going to use these drugs in all clinics (in case of successful results) then we should consider clinics as random but if drugs are very expensive and specialised and they are going to be used only in these clinics then we cannot consider these clinics as random. Then they should be considered as a fixed.
Sometimes choice between random and fixed could be dictated by the amount of the data, information we have. If we have enough data to make inference about the population then we can use mixed effect models. If we do not have enough data then we can make inference only about different levels (e.g. doses of drugs, different clincis) of the variable of interest.

6. Mixed effect models: Simple model
Let us consider model:
yij=+ai+xj+ij
M is overall intercept  constant coefficient on x (describes dependence of y on x), a is random intercept specific to i and  is random error. Let us assume that distribution of  is N(0, ) and the distribution of all ai-s are identically and independently distributed (i.i.d.) random variables with N(0, a). Now we can write the distribution of y:
E(yij) =  +xi
Var(yij) = a2+2
Cov(yij,yij’) = a2
Cov(yi’j,yij’) =0 for i’i
We see that only two parameters are sufficient to describe the whole covariance structure of the observations. Now we can write multivariate normal distribution for joint probability distribution of the observations.

7. Mixed effect models: Simple model
If we use notation V as variance of the observation then we can write of the distribution of the observation and therefore for likelihood:
L(y|m,b,, a) = N(+x,V)
Now the problem is to estimate parameters by maximising this likelihood function. The problem is usually solved iteratively: estimate parameters involved in mean assuming V constant and then estimate parameters involved in V.

8. Mixed effect models: Tests of hypothesis
There are number of hypothesis that can be tested: 1) hypothesis involving parameters included in the mean -  and ; 2) hypothesis about parameters included in the covariance - V part: e.g. a=0. For these tests likelihood ratio test is used. In this particular case, both tests, after some manipulations come to F statistic.

9. General linear mixed effect models
General mixed effect models can be written:
y=X+Zu+
Where u is random variable with distribution N(0,D),  has distribution N(0, ), a is fixed. Then we can write:
E(y)=X
V(y)=Z DZT+2 I
So if the distribution is the normal distribution then we can build joint probability distribution of all observations and therefore the likelihood function. Note that fixed effects are involved only in mean values (just like in linear model), random effect modify the covariance matrix of the observations, it is no longer diagonal and it means that observations are dependent on each other.
Above equations are general form of the linear mixed effect models.

10. Simpler forms of linear mixed effect models
If the structure of the data is known then it is possible to simplify covariance of the above described model. For example if we have two group of variables that are not dependent on each other. For example: let us assume we want to analyse performances of pupils in maths. We take n schools, in each school k classes and in each class l boys and m girls. In the model we would include one constant parameter for boys and one for girls (since these are only two options), then we would take random effect of schools (we are interested in all schools) and classes in these schools (we are interested in all classes in this school). Now it is reasonable to assume that there is no correlation between classes and schools. If class does not belong to the school then I do not know where correlation could come from, if class in the school then since school is considered as a random effect then correlation between classes and this school would be absorbed by the covariance of the school. So we have variance-covariance of schools and that of classes.
Thinking about the system considerable simplifies the model we want to build.

11. Predicting random effects
In mixed models we estimate parameters of fixed effects and distribution for random effects. Sometimes it is interesting to predict random effects. The expressions for fixed effect coefficients and for so called best linear unbiased prediction (BLUP) is
est=(XT V-1X)-1XTV-1y
upredict=DZTV-1(y-Xest)= DZTV-1 (I- (XT V-1X)-1XTV-1)y
var(upredict)=DZTV-1(I- (XT V-1X)-1XTV-1)ZD
Using these facts one can design tests of hypotheses, confidence intervals about u.

12. R commands for linear mixed models
Commands for linear mixed models are in the library nlme:
library(nlme)
data(Orthodont)
lm1 = lme(distance~age+Sex,data=Orthodont)
lm1
summary(lm1)

13. References Demidenko E (2004) Mixed Models: Theory and applications
McCullagh CE, Searle SR, (2001) Generalized, linear and mixed models

14. Exercise
Take the data set esoph form and analyse using generalised linear model. Hints how to analyse this data set is at the end of the help page for this data set:
?esoph