1. 3.1 Expectation Expectation Example 3.1-1 ...
The process of averaging when a r.v. is involved
Notation :
“the mathematical expectation of X”, “the expected value of X”, “the mean value of X”, “the statistical average of X”
Example 3.1-1
90 people are randomly selected
8, 12, 28, 22, 15, and 5 people have 18¢, 45¢, 64¢, 72¢, 77¢, and 95¢, respectively. -
...
These terms correspond to PDF

2. Expected Value of a Random Variable
3.1 Expectation
Expected Value of a Random Variable
In the previous example, if X is the discrete r.v. “fractional dollar value of pocket coins,” it has 90 discrete values xi that occur with probabilities P(xi), and its expected value E[X] is
xi : the fractional dollar coin
P(xi) : the ratio of the number of people for the given dollar value to the total number of people

3. 3.1 Expectation The expected value of any r.v. X
If X happen to be discrete with N possible values xi having probabilities P(xi) of occurrence

4. 3.1 Expectation
Example 3.1-2
Determine the mean value of the following continuous, exponentially distributed r.v.

5. 3.1 Expectation Expected Value of a Function of a r.v. Example 3.1-3
The expected value of a real function g(·) of X
If X is a discrete r.v.
Example 3.1-3
A particular random voltage V having Rayleigh r.v., a = 0, b = 5
The power Y = g(V) = V2
Find the average power

6. 3.1 Expectation
Example Entropy

7. 3.1 Expectation g(X) : a sum of N functions gn(X), n = 1,2,…,N
The expected value of the sum of N functions of a r.v. X
= the sum of the N expected values of the individual function of the r.v.
Conditional Expected Value
Conditional expected value of X

8. Conditional pdf & mean (rolling a die)
3.1 Expectation
Conditional pdf & mean (rolling a die)
1/6
1/3

9. Moments about the Origin
The nth moment denoted by mn
The function g(X) = Xn, n = 0,1,… in
m0 = 1 : the area of the function fX(x)
m1 = E[X] ; the expected value of X
The 1st order moment is very similar to the center of gravity

10. 3.2 Moments
Consider a stick with a uniform density of 1 kg/m and length of 4 m  total weight = 1 kg/m x 4 m = 4 kg
Where is the center of gravity ?
If the density is not uniform, then fx(x) shall be a function of the density of stick. 5 1

11. 3.2 Moments Central Moments Moments about the mean value of X
The function g(X) = (X-E[X])n, n = 0,1,…
0 = 1 : the area of fX(x)
1 = 0

12. 3.2 Moments Variance and Skew Variance Standard deviation
The 2nd central moment 2 :
Standard deviation
The positive square root of variance
A measure of the spread in the function fX(x) about the mean
Variance can be found from a knowledge of the 1st and 2nd moments

13. 3.2 Moments Example 3.2-1 Let X have the exponential density function
Find the variance of X

14. The third central moment
3.2 Moments
The third central moment
A measure of the asymmetry of fX(x) about
The skew of the density function
Zero skew : if a density is symmetric about
The normalized third central moment
is known as the skewness of the density function, or, alternatively, as the coefficient of skewness

15. 3.2 Moments Example 3.2-2 Continue Example 3.2-1. Exponential density
Compute the skew and coefficient of skewness

16. Chebychev’s Inequality
3.2 Moments
Chebychev’s Inequality
For a r.v. X with mean and variance

17. 3.2 Moments The Week Law of Large Numbers
If X1, X2, ..., Xn is a sequence of independent r. v. with identical PDF’s and E[Xi]=, Var[Xi]=2,
Ex) How many samples should be taken if we want to have a prob. of at least 0.95 that the sample mean will not deviate by more than /10 from the true mean ?

18. 3.2 Moments
Example 3.2-3
Find the largest probability that any r.v.’s values are smaller than its mean by 3 standard deviation or larger than its mean by the same amount

19. An alternative form of Chebychev’s inequality
3.2 Moments
An alternative form of Chebychev’s inequality
If for a r.v., then for any
If the variance of a r.v. X approaches zero, the probability approaches 1, i.e. X will equal its mean value
Markov’s Inequality
For a nonnegative r.v. X

20. 3.4 Transformations of a R.V.
We may wish to transform (change) one r.v. X into a new r.v. Y by means of a transformation
The density function fX(x) or distribution function FX(x) of X is known, and the problem is to determine the density function fY(y) or distribution function FY(y) of Y.
The problem : viewed as a “black box” with input X, output Y, and “transfer characteristic” Y = T(X).
X : discrete, continuous, or mixed r.v.
T : linear, nonlinear, segmented, staircase, etc

21. 3.4 Transformations of a R.V.
A R.V. and an increasing transformation A B C D E 1 2 3 4 5 9 16 25
Sample
Space
X(s) Y

22. 3.4 Transformations of a R.V.
Monotonic Transformations of a Continuous Random Variable
A transformation T is called
Monotonically increasing if T(x1)<T(x2) for any x1<x2
Monotonically decreasing if T(x1)>T(x2) for any x1<x2
Consider the increasing transformation
T : continuous and differentiable at all values of x which fX(x)0
Let Y have a value y0 corresponding to the value x0 of X in shown Figure
T-1 : the inverse of the transformation T

23. 3.4 Transformations of a R.V.
The one-to-one correspondence between X and Y
The probability of the event {Yy0} must equal the probability of the event {Xx0}
Differentiate both side w.r.t y0

24. 3.4 Transformations of a R.V.
A R. V. and a decreasing transformation A B C D E 1 2 3 4 5 -1 -4 -9
-16
-25
Sample
Space
X(s) Y

25. 3.4 Transformations of a R.V.
Consider the decreasing transformation

26. 3.4 Transformations of a R.V.
Example 3.4-1
Y = T(X) = aX+b, a,b : any real constant
X = T-1(Y) = (Y-b)/a, dx/dy = 1/a
X : Gaussian r.v.
A linear transformation of a Gaussian r.v. produces another Gaussian r.v
See also that Ex.5-1 in Papoulis’ (pp. 124)

27. Note) a=2, b=1

28. 3.4 Transformations of a R.V.
A R. V. and a non-monotonic transformation A B C D E 1 2 3 4 5 5 8 9
Sample
Space
X(s) Y

29. 3.4 Transformations of a R.V.
Nonmonotonic Transformations of a Continuous r.v. : A transformation may not be monotonic
There may now be more than one interval of values of X that corresponds to the event P{Yy0}
For the values y0 shown in Figure, the event {Yy0} corresponds to the event {Xx1 and x2  X  x3}
The probability of the event {Yy0} now equals the probability of the event {x values yielding Yy0}  {x|Yy0}
poof) see pp. 130, Papoulis’

30. 3.4 Transformations of a R.V.
Example 3.4-2
Find fY(y) for the square-law transformation Y=T(X)=cX2, c>0
(Sol-1) cdf  differentiate cdf  pdf
See also that Ex. 5-2 on pp. 125, Papoulis’

31. 3.4 Transformations of a R.V.
(Sol-2) using the formula

32. 3.4 Transformations of a R.V.
Transformation of a Discrete r.v.
If X is a discrete r.v. while Y = T(X) is a continuous transformation
If the transformation is monotonic, there is one-to-one correspondence between X and Y so that a set {yn} corresponds to the set {xn} through the equation yn=T(xn)
The probability P(yn) equals P(xn)
If T is not monotonic, the above procedure remains valid except there now exists the possibility that more than one value xn corresponds to a value yn
In such a case P(yn) will equal the sum of the probabilities of the various xn which yn = T(xn)

33. 3.4 Transformations of a R.V.
Example 3.4-3
Consider a discrete r.v. X having values x = -1, 0, 1, and 2 with respective probabilities 0.1, 0.3, 0.4, and 0.2
Consider Y = 2-X2+(X3/3)
Find the density of Y
The values of X map to respective values of Y given by y = 2/3, 2, 4/3 and 2/3
The two values x = -1 and x = 2 map to one value y = 2/3
The probability of {Y = 2/3} is the sum of probabilities P{X = -1} and P{X = 3}

34. 3.5 Computer Generation of One Random Variable
How to generate the samples with the distribution we want
Suppose that we have the samples following uniform distribution
How do we do for it?
linear transformation T(x)

35. 3.5 Computer Generation of One Random Variable
We assume initially that T(X) is a monotonically nondecreasing functions so that applies
For uniform X, FX(x) = x where 0<x<1
We solve for the inverse the above eq.
Since any distribution function FY(y) is nondecreasing, its inverse is nondecreasing, and the initial assumption is always satisfied
Given a specified distribution FY(y) for Y, we find the inverse function by solving FY(y) = x for y  this result is T(x)

36. 3.5 Computer Generation of One Random Variable
Example 3.5-1
Find the transformation required to generate the Rayleigh random variable with a = 0
We solve for y and find

37. 3.5 Computer Generation of One Random Variable
Example 3.5-2
Find the required transformation to convert (0,1) uniform r.v. to a r.v. with the arcsine distribution

38. 3.5 Computer Generation of One Random Variable
Example 3.5-3

39. Homework assignment Programming Due : next Tuesday
2-1) Generate random numbers on [0,1] and plot their density function
Hint) c-functions : rand(), srand()
2-2) Transform the random numbers of 1) using the linear transform obtained in Prob)3.5-1, and plot their density function
Due : next Tuesday