The Multivariate Normal Distribution, Part 2 BMTRY 726 1/14/2014

Multivariate Normal PDF Recall the pdf for the MVN distribution Where – x is a p -length vector of observed variables –  is also a p -length vector and E(x)=  –  is a p x p matrix, and Var(x)=  Note,  must also be positive definite

Univariate and Bivariate Normal

Contours of Constant Density Recall projections of f(x) onto the hyperplane created by x are called contours of constant density Properties include: – P-dimensional ellipsoid defined by: – Centered at  – Axes lengths:

Bivariate Examples

Why Multivariate Normal Recall, statisticians like the MVN distribution because… – Mathematically simple – Multivariate central limit theorem applies – Natural phenomena are often well approximated by a MVN distribution So what are some “fun” mathematical properties that make is so nice?

Properties of MVN Result 4.2: If then has a univariate normal distribution with mean and variance

Example

Properties of MVN Result 4. 3: Any linear transformation of a multivatiate normal random vector has a normal distribution So if and and B is a k x p matrix of constants then

Spectral Decomposition Given  is a non-negative definite, symmetric, real matrix, then  can be decomposed according to: Where the eigenvalues are The eigenvectors of  are e 1, e 2,...,e p And these satisfy the expression

Where Recall that Then And

Definition: The square root of  is And Also

From this it follows that the inverse square root of  is Note This leads us to the transformation to the canonical form: If

Marginal Distributions Result 4.4: Consider subsets of X i ’s in X. These subsets are also distributed (multivariate) normal. If Then the marginal distributions of X 1 and X 2 is

Example Consider, find the marginal distribution of the 1 st and 3 rd components

Example Consider, find the marginal distribution of the 1 st and 3 rd components

Marginal Distributions cont’d The converse of result 4.4 is not always true, an additional assumption is needed. Result 4.5(c): If… and X 1 is independent of X 2 then

Result 4. 5(a): If X 1(q x 1) and X 2(p-q x 1) are independent then Cov(X 1,X 2 ) = 0 (b) If Then X 1(q x 1) and X 2(p-q x 1) are independent iff

Example Consider Are x 1 and x 2 independent of x 3 ?

Conditional Distributions Result 4.6: Suppose Then the conditional distribution of X 1 given that X 2 = x 2 is a normal distribution Note the covariance matrix does not depend on the value of x 2

Proof of Result 4.6

Multiple Regression Consider The conditional distribution of Y|X=x is univariate normal with

Example Consider find the conditional distribution of the 1 st and 3 rd components

Example

Result 4.7: If and  is positive definite, then Proof:

Result 4.7: If and  is positive definite, then Proof cont’d:

Result 4.8: If are mutually independent with Then Where vector of constants And are n constants. Additionally if we have and which are r x p matrices of constants we can also say

Sample Data Let’s say that X 1, X 2, …, X n are i.i.d. random vectors If the data vectors are sampled from a MVN distribution then

Multivariate Normal Likelihood We can also look at the joint likelihood of our random sample

Some needed Results (1) Given A > 0 and are eigenvalues of A (a) (b) (c) (2) From (c) we can show that:

Some needed Results (2) Proof that:

Some needed Results (2) Proof that:

Some needed Results (1) Given A > 0 and are eigenvalues of A (a) (b) (c) (2) From (c) we can show that: (3) Given  pxp > 0, B pxp > 0 and scalar b > 0

MLE’s for.

Next Time Sample means and covariance The Wishart distribution Introduction of some basic statistical tests