1. Main topics in the course on probability theory
The concept of probability – Repetition of basic skills
Multivariate random variables – Chapter 1
Conditional distributions – Chapter 2
Transforms – Chapter 3
Order variables – Chapter 4
The multivariate normal distribution – Chapter 5
The exponential family of distributions - Slides
Convergence in probability and distribution – Chapter 6
Probability theory 2011

2. Objectives
Provide a solid understanding of major concepts in probability theory
Increase the ability to derive probabilistic relationships in given probability models
Facilitate reading scientific articles on inference based on probability models
Probability theory 2011

3. The concept of probability – Repetition of basic skills
“Gut: Introduction” + More
Whiteboard
Probability theory 2011

4. Multivariate random variables
Gut: Chapter 1
Slides
Probability theory 2011

5. Joint distribution function - Copula
provides a complete description of the two-dimensional distribution of the random vector (X , Y)
Probability theory 2011

6. Joint distribution function
Probability theory 2011

7. Joint probability density
Probability theory 2011

8. Joint probability function
Probability theory 2011

9. Marginal distributions
Marginal probability density of X
Probability theory 2011

10. Independent events Independent stochastic variables Sufficient that
Independence
Independent events
Independent stochastic variables
Sufficient that
Probability theory 2011

11. Covariance
Assume that E(X) = E(Y) = 0. Then, E(XY) can be regarded as a measure of covariance between X and Y
More generally, we set
Cov(X , Y) = 0 if X and Y are independent. The converse need not be true.
Probability theory 2011

12. Covariance rules
Probability theory 2011

13. Covariance and correlation
Scale-invariant covariance
Probability theory 2011

14. Proof: Assume that Then, observe that
Inequalities
Proof: Assume that
Then, observe that
Probability theory 2011

15. Functions of random variables
Let Y = a + bX
Derive the relationship between the probability density functions of Y and X
Probability theory 2011

16. Functions of random variables
Let X be uniformly distributed on (0,1) and set
Derive the probability density function of Y
Probability theory 2011

17. Functions of random variables
Let X have an arbitrary continuous distribution, and suppose that g is a (differentiable) strictly increasing function. Set
Then
and
Probability theory 2011

18. Linear functions of random vectors
Let (X1, X2) have a uniform distribution on
D = {(x , y); 0 < x <1, 0 < y <1}
Set
Then
Probability theory 2011

19. Functions of random vectors
Let (X1, X2) have an arbitrary continuous distribution, and suppose that g is a (differentiable) one-to-one transformation. Set
Then
where h is the inverse of g.
Proof: Use the variable transformation theorem
Probability theory 2011

20. Random number generation
Uniform distribution
Bin(2; 0.5)
Po(4)
Exp(1)
Probability theory 2011

21. Random number generation - the inversion method
Let F denote the cumulative distribution function of a probability distribution.
Let Z be uniformly distributed on the interval (0,1)
Then, X = F-1(Z) will have the cumulative distribution function F.
How can we generate normally distributed random numbers?
Probability theory 2011

22. Random number generation: method 3 ( the envelope-rejection method)
Generate x from a probability density g(x) such that cg(x)  f(x)
Draw u from a uniform distribution on (0,1)
Accept x if u < f(x)/cg(x)
***************************
Justification:
Let X denote a random number
from the probability density g.
Then
How can we generate normally distributed random numbers?
Probability theory 2011

23. Random number generation - LCGs
Linear congruential generators are defined by the recurrence relation
Numerical Recipes in C advocates a generator of this form with:
a = , b = , M = 232
Drawback: Serial correlation
Probability theory 2011

24. Exercises: Chapter I
1.3, 1.8, 1.14, 1.18, 1.30, 1.31, 1.33
Probability theory 2011