1. EM for Inference in MV Data
BMTRY 726
6/7/2018

2. Last Note on Hypothesis Testing and Confidence Regions/Intervals
Recall we started with some data on n individuals
where we assume X1, X2,…, Xn ~NID(m,S)
We then said we could develop hypothesis tests and confidence regions based on the statistic
if we understand the distribution of T2

3. For Large n
When n is large
This implies
Therefore

4. Large n When n is large, we worry less about the normality
assumption and the distribution of T 2 ~
So if we want to test
We reject H0 if
We can use this to get an approximate confidence
region for m.
Only assumption is that Xi iid and m and S exist.

5. Missing Observations
Missing data are common in studies collecting multivariate data on individuals
So what happens if we have the following data

6. Missing Observations Types of missing data:
MCAR: missing completely at random
-missing observations independent of the actual measurement
MAR: missing at random
-missingness depends on observed values but not the missing values
NMAR: not missing at random
-informative, we can’t ignore why the data are missing

7. Missing Observations
No single uniform method to deal with missing data.
-complete case analysis
-replace missing with sample mean
-multiple imputation
Easiest to work with the MCAR assumption
Imputation: sometimes we do impute missing values. This can be “dangerous” as the variance will often be underestimated. Also often is used for missing values which can bias the results towards rejecting the null.

8. Basic EM Framework
EM is useful when maximum likelihood calculations are easy BUT there are some things we don’t know…
-Have a model for complete data X with associated pdf
with unknown parameter q
-BUT we don’t observe all of X
-We want to maximize the observed-data likelihood with respect to q
*Note: we are assuming data are missing at random!

9. Key Ideas of EM
Compute the expectation of the complete data log-likelihood conditional on the current estimate of parameters
Maximize the resulting log-likelihood to obtain the next estimate of the parameters
Iterate to some level of convergence

10. Key Ideas of EM
EM for exponential families: -Compute the expectation of sufficient statistics conditional on the current estimate of the parameters -Use the resulting estimates of the sufficient statistics to re-estimate -Iterate to some level of convergence.

11. EM and Univariate Normal
Suppose Begin by making an initial guess

12. EM and Univariate Normal
E-step k: compute expectations of sufficient statistics, conditional on Xobs and the current parameter estimate M-step k: maximize the resulting log-likelihood

13. Multivariate Normal Missing Data
Say we have:
If we impute missing valued of BP and LDL
separately, we loose the correlation structure
Imputation using the EM algorithm allows us to
include correlation among the X’s.
We make an initial guess about q and apply our
guess to the missing data. We repeat this until
convergence.

14. EM Algorithm for MVN
In MVN data, the EM algorithm is based on the sufficient statistics

15. EM Algorithm for MVN First estimate based on the data presented
E-Step: For each vector , use the estimates of the mean of the conditional distribution of and to estimate the missing values
These estimates can be calculated based on
properties of a MVN distribution…

16. EM Algorithm for MVN
E-Step: Estimates of missing values

17. EM Algorithm for MVN
E-Step: Estimates of missing values These are used to find the sufficient statistics T1 and T2

18. EM Algorithm for MVN
M-Step: Compute the revised maximum likelihood estimate based on our sufficient statistics from the E-step.

19. Example
Consider an observed sample of 4 subjects and 3 X ’s

20. Example
Update missing values in row 1

21. Example
Update missing values in row 3

22. Example
So what is our updated X after our first iteration?

23. Example
Now estimate the sufficient statistics Lets start with T1

24. Example
Now find T2

25. Example Revise the MLE estimates using the sufficient statistics
Use these estimates and go through the algorithm again

26. Programming an EM algorithm
Start with pseudo-code… what information do you need to capture and what output do you expect?