1. Distributions of Functions of Random Variables November 18, 2015
Probability and Statistical Inference (9th Edition) Chapter 5 (Part 1/2)
Distributions of Functions of Random Variables
November 18, 2015

2. Outline 5.1 Functions of One Random Variable
5.2 Transformation of Two Random Variables
5.3 Several Random Variables
5.4 The Moment-Generating Function Technique

3. Functions of One Random Variable
Let X be a continuous random variable with pdf f(x). If we consider a function of X, say, Y=u(X), then Y must also be a random variable with its own distribution
The cdf of Y is
G(y) = P(Y<=y) = P(u(X)<=y)
The pdf of Y is
g(y) = G’(y)
(where apostrophe ’ denotes derivative)

4. Functions of One Random Variable
Change-of-variable technique
Let X be a continuous random variable with pdf f(x) with support c1<x<c2. We begin this discussion by taking Y=u(X) as a continuous increasing function of X with inverse function X=v(Y). Say the support of X maps onto the support of Y, d1=u(c1)<y<d2=u(c2). Then, the cdf of Y is
Thus,

5. Functions of One Random Variable
The derivative, g(y)=G’(y), of such an expression is given by
Suppose now the function Y=u(X) and its inverse X=v(Y) are continuous decreasing functions. Then,
Thus,

6. Functions of One Random Variable
Thus, for both increasing and decreasing cases, we can write the pdf of Y:

7. Functions of One Random Variable
Example 1
All intervals on the distribution's support are equally probable => uniform distribution.

8. Functions of One Random Variable
Theorem 1: Suppose that a random variable X has a continuous distribution for which the cdf is F. Then, the random variable Y=F(X) has a uniform distribution
Random variables from any given continuous distribution can be converted to random variables having a uniform distribution, and vice versa

9. Functions of One Random Variable
cdf F of N(0,1)
pdf of N(0,1)
X ~ N(0,1)
Y = F(X) ~ U(0,1)

10. Functions of One Random Variable
Theorem 1 (converse statement): If U is a uniform random variable on (0,1), then random variable X=F-1(U) has cdf F
(where F is a continuous cdf and F-1 is its inverse function)
Proof:
P(X<=x) = P(F-1(U)<=x) = P(U<=F(x)) = F(x)

11. Functions of One Random Variable
Theorem 1 (converse statement) can be used to generate random variables of any distribution
To generate values of X which are distributed according to the cdf F:
1. Generate a random number u from U (uniform random variable on (0,1))
2. Compute the value x such that F(x) = u
3. Take x to be the random number distributed according to the cdf F

12. Functions of One Random Variable
Example 2 (the transformation Y=u(X) is not one-to-one): Let Y=X2, where X is Cauchy, then
where
Thus,
In this case of two-to-one transformation, there is a need to sum two terms, each of which is similar to the one-to-one case

13. Functions of One Random Variable
Consider the discrete case
Let X be a discrete random variable with pmf f(x)=P(X=x). Let Y=u(X) be a one-to-one transformation with inverse X=v(Y). Then, the pmf of Y is
Note that, in the discrete case, the derivative |v’(y)| is not needed

14. Functions of One Random Variable
Example 3: Let X be a uniform random variable on {1,2,…,n}. Then Y=X+a is a uniform random variable on {a+1,a+2,…,a+n}

15. Transformations of Two Random Variables
If X1 and X2 are two continuous random variables with joint pdf f(x1,x2), and if Y1=u1(X1,X2), Y2=u2(X1,X2) has the single-valued inverse X1=v1(Y1,Y2), X2=v2(Y1,Y2), then the joint pdf of Y1 and Y2 is
where | | denotes absolute value and J is the Jacobian given by
where | | denotes the determinant of a matrix
The derivative is replaced by the Jacobian.

16. Transformations of Two Random Variables
Example 1: Let X1 and X2 be independent random variables, each with pdf
Hence, their joint pdf is
Let Y1=X1-X2, Y2=X1+X2. Thus, x1=(y1+y2)/2, x2=(y2-y1)/2, and the Jacobian

17. Transformations of Two Random Variables
Then, the joint pdf of Y1 and Y2 is

18. Transformations of Two Random Variables
Example 2 (Box-Muller Transformation): Let X1 and X2 be i.i.d. U(0,1). Let
Thus,
where Q=Z12+Z22, and the Jacobian

19. Transformations of Two Random Variables
Since the joint pdf of X1 and X2 is
it follows that the joint pdf of Z1 and Z2 is
This is the joint pdf of two i.i.d. N(0,1) random variables
Hence, we can generate two i.i.d. N(0,1) random variables from two i.i.d. U(0,1) random variables using this method

20. Random Samples
Assume that we conduct an experiment n times independently. Let Xk be the random variable corresponding to the outcome of the k-th run of experiment. Then X1, X2,…, Xn form a set of random samples of size n

21. Random Samples
For example, if we toss a die n times and let X1, X2,…, Xn be the random variables corresponding to the outcome of the k-th tossing. Then X1, X2,…, Xn form a set of random samples of size n

22. Theorems about Independent Random Variables
Let X1, X2,…, Xn be n independent discrete random variables and h() be a function of n variables. Then, the expected value of random variable Z=h(X1, X2,…, Xn) is equal to

23. Theorems about Independent Random Variables
Likewise, if X1, X2,…, Xn are independent continuous random variables, then

24. Theorems about Independent Random Variables
Theorem: If X1, X2,…, Xn are independent random variables and, for i = 1, 2,…, n, E[hi(Xi)] exists, then
E[h1(X1) h2(X2) … hn(Xn)]
= E[h1(X1)] E[h2(X2)]… E[hn(Xn)]

25. Theorems about Independent Random Variables
Proof for the discrete cases:
The proof for the continuous cases can be derived similarly

26. Theorems about Independent Random Variables
Theorem: Assume that X1, X2,…, Xn are n independent random variables with respective means μ1, μ2,…, μn and variances σ12, σ22,…, σn2. Then, the mean and variance of random variable
where a1,a2,…,an are real constants, are
and

27. Theorems about Independent Random Variables
Proof:

28. Theorems about Independent Random Variables
Since Xi and Xj are independent where i≠j,
Therefore,

29. Moment-Generating Function Technique
Let X be a random variable. The moment-generating function (mgf) of X is defined as
It is called mgf because all of the moments of X can be obtained by successively differentiating MX(t)

30. Moment-Generating Function Technique
For example,
Thus,
Similarly,

31. Moment-Generating Function Technique
In general, the nth derivative of MX(t) evaluated at t=0 equals E[Xn], i.e.,
where denotes the nth derivative of

32. Moment-Generating Function Technique
Moment generating function uniquely determines the distribution. That is, there exists a one-to-one correspondence between the moment generating function (mgf) and the distribution function (pmf/pdf) of a random variable

33. Moment-Generating Function Technique
Example 1 (mgf of N(0,1)):
where the last equality follows from the fact that the expression in the integral is the pdf of a normal random variable with mean t and variance 1 which integrates to one

34. Moment-Generating Function Technique
Exercise (mgf of N(m,s2)):

35. Moment-Generating Function Technique
Theorem: If X1,X2,…,Xn are independent random variables with respective mgfs, Mi(t), i=1,2,…,n, then the mgf of is

36. Moment-Generating Function Technique
Proof:

37. Moment-Generating Function Technique
Corollary: If X1,X2,…,Xn correspond to independent random samples from a distribution with mgf M(t), then

38. Moment-Generating Function Technique
The mgf of the sum of independent random variables is just the product of the individual mgfs

39. Moment-Generating Function Technique
Example 2:
Recall that, let Z1, Z2, …, Zn be independent N(0,1). Then, W=Z12+Z22+…+Zn2 has a distribution that is chi-square with n degrees of freedom, denoted by
Let X1, X2, …, Xn be independent chi-square random variables with r1, r2, …, rn degrees of freedom, respectively. Show that Y=X1+X2+…+Xn is

40. Moment-Generating Function Technique
Use the moment-generating function technique:
which is the mgf of a
Thus, Y is