1. The Multivariate Normal Distribution, Part 2 BMTRY 726 1/14/2014

2. Multivariate Normal PDF Recall the pdf for the MVN distribution Where – x is a p -length vector of observed variables –  is also a p -length vector and E(x)=  –  is a p x p matrix, and Var(x)=  Note,  must also be positive definite

3. Univariate and Bivariate Normal

4. Contours of Constant Density Recall projections of f(x) onto the hyperplane created by x are called contours of constant density Properties include: – P-dimensional ellipsoid defined by: – Centered at  – Axes lengths:

5. Bivariate Examples

6. Why Multivariate Normal Recall, statisticians like the MVN distribution because… – Mathematically simple – Multivariate central limit theorem applies – Natural phenomena are often well approximated by a MVN distribution So what are some “fun” mathematical properties that make is so nice?

7. Properties of MVN Result 4.2: If then has a univariate normal distribution with mean and variance

8. Example

9. Properties of MVN Result 4. 3: Any linear transformation of a multivatiate normal random vector has a normal distribution So if and and B is a k x p matrix of constants then

10. Spectral Decomposition Given  is a non-negative definite, symmetric, real matrix, then  can be decomposed according to: Where the eigenvalues are The eigenvectors of  are e 1, e 2,...,e p And these satisfy the expression

11. Where Recall that Then And

12. Definition: The square root of  is And Also

13. From this it follows that the inverse square root of  is Note This leads us to the transformation to the canonical form: If

14. Marginal Distributions Result 4.4: Consider subsets of X i ’s in X. These subsets are also distributed (multivariate) normal. If Then the marginal distributions of X 1 and X 2 is

15. Example Consider, find the marginal distribution of the 1 st and 3 rd components

16. Example Consider, find the marginal distribution of the 1 st and 3 rd components

17. Marginal Distributions cont’d The converse of result 4.4 is not always true, an additional assumption is needed. Result 4.5(c): If… and X 1 is independent of X 2 then

18. Result 4. 5(a): If X 1(q x 1) and X 2(p-q x 1) are independent then Cov(X 1,X 2 ) = 0 (b) If Then X 1(q x 1) and X 2(p-q x 1) are independent iff

19. Example Consider Are x 1 and x 2 independent of x 3 ?

20. Conditional Distributions Result 4.6: Suppose Then the conditional distribution of X 1 given that X 2 = x 2 is a normal distribution Note the covariance matrix does not depend on the value of x 2

21. Proof of Result 4.6

23. Multiple Regression Consider The conditional distribution of Y|X=x is univariate normal with

24. Example Consider find the conditional distribution of the 1 st and 3 rd components

25. Example

27. Result 4.7: If and  is positive definite, then Proof:

28. Result 4.7: If and  is positive definite, then Proof cont’d:

29. Result 4.8: If are mutually independent with Then Where vector of constants And are n constants. Additionally if we have and which are r x p matrices of constants we can also say

30. Sample Data Let’s say that X 1, X 2, …, X n are i.i.d. random vectors If the data vectors are sampled from a MVN distribution then

31. Multivariate Normal Likelihood We can also look at the joint likelihood of our random sample

32. Some needed Results (1) Given A > 0 and are eigenvalues of A (a) (b) (c) (2) From (c) we can show that:

33. Some needed Results (2) Proof that:

34. Some needed Results (2) Proof that:

35. Some needed Results (1) Given A > 0 and are eigenvalues of A (a) (b) (c) (2) From (c) we can show that: (3) Given  pxp > 0, B pxp > 0 and scalar b > 0

36. MLE’s for.

39. Next Time Sample means and covariance The Wishart distribution Introduction of some basic statistical tests