1. Continuous Random Variables

2. Introduction
Most of physical measurements, do not produce discrete set of values but a continuum.
The rainfall data
Maximum temperature
The number of possible temperatures in the interval [0, 40] is infinite and uncountable.
We can “round off” the temperatures then {0,1,2,…, 40}, but we loose precision.
We are interested in the probability of any interval [20,25] or the union of interval [20,25] U [35,40].
Then

3. Definition of a Continuous RV
A continuous RV X is defined as a mapping from the experimental sample space S to a numerical (or measurement) sample space SX (subset of the real line)
The set is uncountable, but we cannot assign a nonzero probability and expect the the sum or probabilities to be one.

4. Definition of a Continuous RV
Let’s assign probabilities to interval, to solve this dilemma.
Each value of X is equally likely so that interval of the same length are equally likely
the probability of the dart landing in the interval [a, b].
What is the probability of all equal length interval are no the same.
A champion dart thrower would be more likely to obtain a value near x = 0 than near x = 1.
Let’s extend PMF to calculate interval probabilities for continuous RV.

5. Definition of a Continuous RV
Consider first a possible approximation of P[a ≤ X ≤ b] = b – a by a uniform PMF as where Δx = 1/M, so that Mx = 1. If a = 0.38 and b = 0.52 then the true value P[a ≤ X ≤ b] = 0.14.

6. Definition of a Continuous RV
with Δx = 1/M, we have
And defining pX(x) = 1 for 0 < x < 1 and zero otherwise, we can write this as
Finally, letting Δx  0 to yield no errors in the approximation, the sum becomes
Note pX(x) is defined to be 1 for all 0 < x < 1.

7. Definition of a Continuous RV
To interpret pX(x) we have for x0 = kx for k an integer:
pX(x) is the probability per unit length and is termed
the probability density function.

8. Definition of a Continuous RV
Since the value of an integral may be interpreted as the area under a curve, the probability is found by determining the area under the PDF curve.

9. Definition of a Continuous RV
The advanced dart thrower will have higher probability to hit near the bullseye i.e. the PDF is nonuniform.

10. Mass analogy
Consider a slice of cheese with length 2m, height 1m, depth 1m and 1kg mass.
If its mass is 1kg, then its overall density D is
However, its linear density or mass per meter ΔM/Δx will change with x.

11. Mass analogy
To determine the linear density we compute ΔM/Δx versus x. Note
Hence, ΔM/Δx = ΔV/Δx = x0/2, this is the same even as Δx  0. Thus,
and to obtain the mass for any wedge we need to integrate ΔM/Δx
Where m(x) = x/2 is the linear mass density of the mass per unit length.

12. The PDF and Its properties
The PDF must have certain properties that the probabilities obtained by satisfy the axioms.
Property 1. PDF must be nonnegative
Proof: If pX(x) < 0 on some small interval [x0 – Δx/2, x0 + Δx/2], then
Which violates Axiom 1 that P[E] ≥ 0 for all events E.
Property 2. PDF must integrate to one.
Proof:

13. Important PDFs Uniform PDF The shorthand notation is X ~ U(a,b).
The Matlab uses rand to produce realization.
Hence, in simulating a coin toss with a probability of heads p = 0.75,
Matlab implementation

14. Important PDFs Exponential PDF for λ > 0.
This PDF has discontinuity (just like uniform PDF)
The PDF can exceed one in value.
Recall from property 2 that The area under the PDF cannot exceed one
This PDF is often used as a model for the lifetime of a product.
If X is the failure time in days of a lightbulb, then P[X > 100] is the probability that the light bulb will fail after 100 days or it will last for at least 100 days.

15. Important PDFs Gaussian or Normal PDF
where σ2 > 0 and -∞< μ < ∞.
A standard normal PDF is one for which μ = 0 and σ2 = 1.
The shorthand notation is X ~ N(μ , σ) . MATLAB generates a realization of a standard normal RV using randn.
To find the probability of the outcome of a Gaussian RV lying with an interval requires numerical integration since the integral
cannot be evaluate analytically.

16. Important PDFs
The normal PDF is commonly used to model noise in a communication system, as well as for numerous other applications. The normal PDF arises as the PDF of a large number of independent RV that have been added together.

17. Important PDFs Laplacian
where σ2 > 0. The Laplacian PDF is easily integrated to find the probability of an interval. This PDF is used as a speech model.

18. Important PDFs Cauchy PDF
It can be easily integrated to find the probability of any interval.
It arises as the PDF if the ration of two independent N(0 , 1) RV.
The Cauchy distribution is often used in statistics as the canonical example of a "pathological" distribution. Both its mean and its variance are undefined.

19. Important PDFs
Gamma PDF is a very general PDF that is used for nonnegative RV
Where λ>0, α>0, and Γ(z) is the Gamma function which is defined as
Γ(z) is an extension of the factorial function with its argument shifted down by 1, to real and complex numbers.
Γ(z) is a smooth curve that connects the points (x, y) given by  y = (x − 1)! at the positive integer values for x.
Used as a normalization factor

20. Properties of Gamma function
Property 1:
Proof:
Property 2:
Proof: Follow from property 1 with z = N – 1 since
Property 3 2π

21. Important PDFs
The Gamma PDF reduce to many well known PDF for appropriate choices of the parameters.
Exponential for α = 1
Chi-squared PDF with N degrees of freedom for α = N/2, λ = ½.
Erlang α = N

22. Important PDFs Chi-squared PDF
pX(x)
Describes probability of the sum of the squares of k independent standard normal random variables.
It is one of the most widely used probability distributions in inferential statistics, e.g., in hypothesis testing or in construction of confidence intervals.

23. Important PDFs Erlang PDF
The PDF is given by two parameters: the shape k, which is a positive integer, and the rate λ, which is a positive real number.
Events that occur independently with some average rate are modeled with a Poisson process. The waiting times between k occurrences of the event are Erlang distributed.
This PDF can be used to determine the probability of packet loss or delay
Erlang PDF
λ = 2.0
λ = 2.0
λ = 2.0
λ = 1.0
λ = 0.5

24. Important PDFs: Examples of Gamma PDF
α = 1
λ = 2.0
Exponential
α = 2
λ = 2.0
α = 3
λ = 2.0
α = 5
λ = 1.0
α = 9
λ = 0.5
Chi-squared
The gamma PDF generalizes the Erlang PDF by allowing k to be any real number α

25. Important PDFs Rayleigh PDF
Rayleigh distribution naturally arises is when wind velocity is analyzed into its orthogonal 2-dimensional vector components.
Assuming that the magnitudes of each component are uncorrelated, normally distributed with equal variance, and zero mean, then the overall wind speed (vector magnitude) will be characterized by a Rayleigh distribution.

26. Cumulative Distribution Functions
The cumulative distribution function (CDF) for a continuous RV is
and is evaluated using the PDF as

27. Cumulative Distribution Functions
Uniform CDF
Exponential CDF
Chi-square CDF
Rayleigh CDF
continuous

28. Gaussian CDF The CDF of Gaussian PDF with μ = 0 and σ2 = 1
This cannot be evaluated further but can be found numerically.
Φ(x) represents the area under the PDF to the left of the point x.

29. Gaussian CDF
It is sometimes more convenient, however, to have knowledge of the area to the right.
Shaded area = Φ(1)
Shaded area = Q(1)

30. Gaussian CDF Properties of the Q functions Matlab implementation
Q(-x) = 1 – Q(x)

31. Probability of error in a com. system
The probability of error for PSK digital communication system is given by
where N(0, 1).
To explicitly evaluate Pe we have that

32. Probability of error in a com. system
It is also sometimes important to determine A to yield a given Pe. This is found as A = 2Q-1(Pe), where Q-1 is the inverse of the Q . It is defined as the value of x necessary to yield a given value of Q(x). Q(x) can be approximated as
Note that the are under the standard normal Guassian PDF is mostly contained in the interval [-3, 3].
Since the PDF is symmetric 99.8% of the probability lies within this interval

33. Gaussian CDF If X ~ N(μ, σ2), then the right-tail probability becomes
CDF can be used to find probabilities of intervals

34. Probability of interval for exponential PDF
Since FX(x) = 1 – exp(-λx) for x ≥ 0, we have for a > 0 and b > 0
Which should be compared to
It is easy to see that finding probability using CDF (*) is easier than PDF (**) that is an advantage of CDF over PDF.
(*)
(**)

35. Relationship between PDF and CDF
We obtained the CDF form the PDF, lets recover PDF from CDF.
Consider a small interval [x0 – dx/2, x0 + dx/2] for a RV X and
Then, we have
See next slide

36. Relationship between PDF and CDF
So that Hence, we can obtain the PDF from the CDF by differentiation or This is the fundamental theorem of calculus.

37. Transformations The PDF of a transformed continuous RV is given by
Example: Given Y = 2X, where X ~ U(1,2), then g(x) = 2x and g-1(y) = y/2. Substituting it to (*) we get
The principal difference between transformation of discrete RV and continuous RV is Jacobian factor dg-1(y)/dy.
(*)

38. Transformations
Jacobian is needed to account for the change in scaling due to the mapping of a given length interval into an interval of a different length. 4
zoomed up
A total of 50 realizations were are obtained for X and Y. The generated X outcomes are shown on the x-axis and the resultant Y outcomes obtained from y = 2x are shown on the y-axis.
The density of points on the y-axis is less than that on the x-axis. 3 1 2

39. PDF for linear(affine) transformation
To determine the PDF of Y = aX + b, for a and b constants first assume that SX = {x : -∞ < x < ∞} and SY = {y : -∞ < y < ∞}.
We gave g(x) = ax + b so that the inverse function g-1 is found by solving y = ax + b for x, so that
and

40. PDF for linear transformation of normal RV
Consider X ~ N(0, 1) and the transformation
Then, letting we have
and therefore Y ~ N(μ, σ2).
A linear transformation of a Gaussian RV results in another Gaussian RV with different parameters.

41. Many-to-one transformation
When transformation is not one-to-one, we will have multiple solutions for in y = g(x).
An example is for y = x2 for which solutions are
In this case we must add the PDFs to yield
In general, if y = g(x) have solutions xi = gi-1(y) for i = 1,2,…,M, then

42. Alternative approach to determine PDF of a transformed RV
An alternative means of finding the PDF of a transformed RV is to first find the CDF and then differentiate it. Example: CDF approach to determine PDF of Y = X2 for X ~ N(0, 1). Then differentiating we have
using chain rule
Since pX(-x) = pX(x) for X ~ N(0, 1)

43. Example: PDF of Y=X2 for X ~ N(0,1)
Since -∞ < x < ∞ we must Y ≥ 0. Net because and we have
Which reduces to
Y ~ χ12 Chi square RV with N = 1
The PDF is undefined at y = 0 since pY(0)  ∞, although the total area under the PDF is equal to 1.

44. Setting Clipping Levels for Speech Signals
To prevent clipping in the sound communication system due to analog-to-digital conversion it is necessary to to determine the highest amplitude of the speech signal that can be expected to occur.
The distribution of amplitudes is modeled by Laplacian PDF i.e.
A design requirement might be to transmit a speech signal without clipping 99% of time. A model for a clipper is

45. Setting Clipping Levels for Speech Signals
Clipping will occur whenever |X| > A. To satisfy the design requirement (not clipping for 99% of time), we should choose A so that pclip ≤ 0.01.
And since the Laplacian PDF is symmetric about x = 0 this is just

46. Setting Clipping Levels for Speech Signals
Hence, if this probability is to be no more than 0.01, we must have or solving for A produces the requirement that As example consider a speech signal with σ2 = 1, assume a clipping level of A = 1. The probability of clipping is

47. Setting Clipping Levels for Speech Signals
50 outcomes of a Laplacian model with σ2 = 1
With A = 1, we would expect about 500.2431≈12 instances of clipping.
To meet specification we should have that
clipping level to clip only 1% of times.

48. Exercises Find probability of interval [a, b] for exponential PDF.
A wedge of cheese is sliced from x= a to x = b. If a = 0 and b = 0.2, what is the mass of cheese in the wedge? How about it a = 1.8 and b= 2?
Which of the function below are valid PDFs? If a function is not a PDF, why not?

49. Exercises
Determine the value of c to make the following function a valid PDF
A Gaussian mixture PDF is defined as
for σ12 ≠ σ22. What are the possible values for α1 and α2 so that thus is a valid PDF?
A memory chip has a projected lifetime X in days that is modeled as X ~ exp(0.001). What is the probability that it will fail within one year?