1. Joint Distribution of two or More Random Variables
Sometimes more than one measurement (r.v.) is taken on each member of the sample space. In cases like this there will be a few random variables defined on the same probability space and we would like to explore their joint distribution.
Joint behavior of 2 random variable (continuous or discrete), X and Y determined by their joint cumulative distribution function
n – dimensional case
week 7

2. Discrete case
Suppose X, Y are discrete random variables defined on the same probability space.
The joint probability mass function of 2 discrete random variables X and Y is the function pX,Y(x,y) defined for all pairs of real numbers x and y by
For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and
week 7

3. Example for illustration
Toss a coin 3 times. Define,
X: number of heads on 1st toss, Y: total number of heads.
The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}.
We display the joint distribution of X and Y in the following table
Can we recover the probability mass function for X and Y from the joint table?
To find the probability mass function of X we sum the appropriate rows of the table of the joint probability function.
Similarly, to find the mass function for Y we sum the appropriate columns.
week 7

4. Marginal Probability Function
The marginal probability mass function for X is
The marginal probability mass function for Y is
Case of several discrete random variables is analogous.
If X1,…,Xm are discrete random variables on the same sample space with joint probability function
The marginal probability function for X1 is
The 2-dimentional marginal probability function for X1 and X2 is
week 7

5. Example
Roll a die twice. Let X: number of 1’s and Y: total of the 2 die.
There is no available form of the joint mass function for X, Y. We display
the joint distribution of X and Y with the following table.
The marginal probability mass function of X and Y are
Find P(X ≤ 1 and Y ≤ 4)
week 7

6. Independence of random variables
Recall the definition of random variable X: mapping from Ω to R such that
for
By definition of F, this implies that (X > 1.4) is and event and for the
discrete case (X = 2) is an event.
In general is an event for any set A that is formed
by taking unions / complements / intersections of intervals from R.
Definition
Random variables X and Y are independent if the events and
are independent.
week 7

7. Theorem
Two discrete random variables X and Y with joint pmf pX,Y(x,y) and marginal mass function pX(x) and pY(y), are independent if and only if
Proof:
Question: Back to the rolling die 2 times example, are X and Y independent?
week 7

8. Conditional Probability on a joint discrete distribution
Given the joint pmf of X and Y, we want to find
and
Back to the example on slide 5,
These 11 probabilities give the conditional pmf of Y given X = 1.
week 7

9. Definition
For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass
function pX(x) and pY(y). If x is a number such that pX(x) > 0, then the
conditional pmf of Y given X = x is
Is this a valid pmf?
Similarly, the conditional pmf of X given Y = y is
Note, from the above conditional pmf we get
Summing both sides over all possible values of Y we get
This is an extremely useful application of the law of total probability.
Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x).
week 7

10. Example
Suppose we roll a fair die; whatever number comes up we toss a coin that many times. What is the distribution of the number of heads?
Let X = number of heads, Y = number on die.
We know that
Want to find pX(x).
The conditional probability function of X given Y = y is given by
for x = 0, 1, …, y.
By the Law of Total Probability we have
Possible values of x: 0,1,2,…,6.
week 7

11. Example
Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of success p on any trail. Let X = number of failures before 1st success and
Y = number of failures before 2nd success (including those before the 1st success).
The marginal pmf of X and Y are
for x = 0, 1, 2, ….
and
for y = 0, 1, 2, ….
The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is the probability that the 1st success occurred after x failure and the 2nd success after y – x more failures (after 1st success), e.g.
The joint pmf is then given by
Find the marginal probability functions of X, Y. Are X, Y independent?
What is the conditional probability function of X given Y = 5
week 7

12. Some Notes on Multiple Integration
Recall: simple integral cab be regarded as the limit as N  ∞ of
the Riemann sum of the form
where ∆xi is the length of the ith subinterval of some partition of [a, b].
By analogy, a double integral
where D is a finite region in the xy-plane, can be regarded as the limit as
N  ∞ of the Riemann sum of the form
where Ai is the area of the ith rectangle in a decomposition of D into rectangles.
week 7

13. Evaluation of Double Integrals
Idea – evaluate as an iterated integral. Restrict D now to a region such that any vertical line cuts its boundary in 2 points. So we can describe D by
yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
Theorem
Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
If f(x,y) is continuous on D then
Comment: When evaluating a double integral, the shape of the region or the form of f (x,y) may dictate that you interchange the role of x and y and integrate first w.r.t x and then y.
week 7

14. Examples
1) Evaluate where D is the triangle with vertices (0,0), (2,0) and (0,1).
2) Evaluate where D is the region in the 1st quadrate bounded
by y = 0, y = x and x2 + y2 = 1.
3) Integrate over the set of points in the positive quadrate for
which y > 2x.
week 7

15. The Joint Distribution of two Continuous R.V’s
Definition
Random variables X and Y are (jointly) continuous if there is a non-negative function fX,Y(x,y) such that
for any “reasonable” 2-dimensional set A.
fX,Y(x,y) is called a joint density function for (X, Y).
In particular , if A = {(X, Y): X ≤ x, Y ≤ x}, the joint CDF of X,Y is
From Fundamental Theorem of Calculus we have
week 7

16. Properties of joint density function
for all
It’s integral over R2 is
week 7

17. Example Consider the following bivariate density function
Check if it’s a valid density function.
Compute P(X > Y).
week 7

18. Properties of Joint Distribution Function
For random variables X, Y , FX,Y : R2  [0,1] given by FX,Y (x,y) = P(X ≤ x,Y ≤ x)
FX,Y (x,y) is non-decreasing in each variable i.e.
if x1 ≤ x2 and y1 ≤ y2 .
and
week 7

19. Marginal Densities and Distribution Functions
The marginal (cumulative) distribution function of X is
The marginal density of X is then
Similarly the marginal density of Y is
week 7

20. Example Consider the following bivariate density function
Check if it is a valid density function.
Find the joint CDF of (X, Y) and compute P(X ≤ ½ ,Y ≤ ½ ).
Compute P(X ≤ 2 ,Y ≤ ½ ).
Find the marginal densities of X and Y.
week 7

21. Generalization to higher dimensions
Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then
Marginal density of X is given by:
Marginal density of X, Y is given by :
week 7

22. Example Given the following joint CDF of X, Y
Find the joint density of X, Y.
Find the marginal densities of X and Y and identify them.
week 7

23. Example Consider the joint density where λ is a positive parameter.
Check if it is a valid density.
Find the marginal densities of X and Y and identify them.
week 7