Main topics in the course on probability theory The concept of probability – Repetition of basic skills Multivariate random variables – Chapter 1 Conditional distributions – Chapter 2 Transforms – Chapter 3 Order variables – Chapter 4 The multivariate normal distribution – Chapter 5 The exponential family of distributions - Slides Convergence in probability and distribution – Chapter 6 Probability theory 2011

Objectives Provide a solid understanding of major concepts in probability theory Increase the ability to derive probabilistic relationships in given probability models Facilitate reading scientific articles on inference based on probability models Probability theory 2011

The concept of probability – Repetition of basic skills “Gut: Introduction” + More Whiteboard Probability theory 2011

Multivariate random variables Gut: Chapter 1 Slides Probability theory 2011

Joint distribution function - Copula provides a complete description of the two-dimensional distribution of the random vector (X , Y) Probability theory 2011

Joint distribution function Probability theory 2011

Joint probability density Probability theory 2011

Joint probability function Probability theory 2011

Marginal distributions Marginal probability density of X Probability theory 2011

Independent events Independent stochastic variables Sufficient that Independence Independent events Independent stochastic variables Sufficient that Probability theory 2011

Covariance Assume that E(X) = E(Y) = 0. Then, E(XY) can be regarded as a measure of covariance between X and Y More generally, we set Cov(X , Y) = 0 if X and Y are independent. The converse need not be true. Probability theory 2011

Covariance rules Probability theory 2011

Covariance and correlation Scale-invariant covariance Probability theory 2011

Proof: Assume that Then, observe that Inequalities Proof: Assume that Then, observe that Probability theory 2011

Functions of random variables Let Y = a + bX Derive the relationship between the probability density functions of Y and X Probability theory 2011

Functions of random variables Let X be uniformly distributed on (0,1) and set Derive the probability density function of Y Probability theory 2011

Functions of random variables Let X have an arbitrary continuous distribution, and suppose that g is a (differentiable) strictly increasing function. Set Then and Probability theory 2011

Linear functions of random vectors Let (X1, X2) have a uniform distribution on D = {(x , y); 0 < x <1, 0 < y <1} Set Then Probability theory 2011

Functions of random vectors Let (X1, X2) have an arbitrary continuous distribution, and suppose that g is a (differentiable) one-to-one transformation. Set Then where h is the inverse of g. Proof: Use the variable transformation theorem Probability theory 2011

Random number generation Uniform distribution Bin(2; 0.5) Po(4) Exp(1) Probability theory 2011

Random number generation - the inversion method Let F denote the cumulative distribution function of a probability distribution. Let Z be uniformly distributed on the interval (0,1) Then, X = F-1(Z) will have the cumulative distribution function F. How can we generate normally distributed random numbers? Probability theory 2011

Random number generation: method 3 ( the envelope-rejection method) Generate x from a probability density g(x) such that cg(x)  f(x) Draw u from a uniform distribution on (0,1) Accept x if u < f(x)/cg(x) *************************** Justification: Let X denote a random number from the probability density g. Then How can we generate normally distributed random numbers? Probability theory 2011

Random number generation - LCGs Linear congruential generators are defined by the recurrence relation Numerical Recipes in C advocates a generator of this form with: a = 1664525, b = 1013904223, M = 232 Drawback: Serial correlation Probability theory 2011

Exercises: Chapter I 1.3, 1.8, 1.14, 1.18, 1.30, 1.31, 1.33 Probability theory 2011