Continuous Random Variables

Introduction Most of physical measurements, do not produce discrete set of values but a continuum. The rainfall data Maximum temperature The number of possible temperatures in the interval [0, 40] is infinite and uncountable. We can “round off” the temperatures then {0,1,2,…, 40}, but we loose precision. We are interested in the probability of any interval [20,25] or the union of interval [20,25] U [35,40]. Then

Definition of a Continuous RV A continuous RV X is defined as a mapping from the experimental sample space S to a numerical (or measurement) sample space SX (subset of the real line) The set is uncountable, but we cannot assign a nonzero probability and expect the the sum or probabilities to be one.

Definition of a Continuous RV Let’s assign probabilities to interval, to solve this dilemma. Each value of X is equally likely so that interval of the same length are equally likely the probability of the dart landing in the interval [a, b]. What is the probability of all equal length interval are no the same. A champion dart thrower would be more likely to obtain a value near x = 0 than near x = 1. Let’s extend PMF to calculate interval probabilities for continuous RV.

Definition of a Continuous RV Consider first a possible approximation of P[a ≤ X ≤ b] = b – a by a uniform PMF as where Δx = 1/M, so that Mx = 1. If a = 0.38 and b = 0.52 then the true value P[a ≤ X ≤ b] = 0.14.

Definition of a Continuous RV with Δx = 1/M, we have And defining pX(x) = 1 for 0 < x < 1 and zero otherwise, we can write this as Finally, letting Δx  0 to yield no errors in the approximation, the sum becomes Note pX(x) is defined to be 1 for all 0 < x < 1.

Definition of a Continuous RV To interpret pX(x) we have for x0 = kx for k an integer: pX(x) is the probability per unit length and is termed the probability density function.

Definition of a Continuous RV Since the value of an integral may be interpreted as the area under a curve, the probability is found by determining the area under the PDF curve.

Definition of a Continuous RV The advanced dart thrower will have higher probability to hit near the bullseye i.e. the PDF is nonuniform.

Mass analogy Consider a slice of cheese with length 2m, height 1m, depth 1m and 1kg mass. If its mass is 1kg, then its overall density D is However, its linear density or mass per meter ΔM/Δx will change with x.

Mass analogy To determine the linear density we compute ΔM/Δx versus x. Note Hence, ΔM/Δx = ΔV/Δx = x0/2, this is the same even as Δx  0. Thus, and to obtain the mass for any wedge we need to integrate ΔM/Δx Where m(x) = x/2 is the linear mass density of the mass per unit length.

The PDF and Its properties The PDF must have certain properties that the probabilities obtained by satisfy the axioms. Property 1. PDF must be nonnegative Proof: If pX(x) < 0 on some small interval [x0 – Δx/2, x0 + Δx/2], then Which violates Axiom 1 that P[E] ≥ 0 for all events E. Property 2. PDF must integrate to one. Proof:

Important PDFs Uniform PDF The shorthand notation is X ~ U(a,b). The Matlab uses rand to produce realization. Hence, in simulating a coin toss with a probability of heads p = 0.75, Matlab implementation

Important PDFs Exponential PDF for λ > 0. This PDF has discontinuity (just like uniform PDF) The PDF can exceed one in value. Recall from property 2 that The area under the PDF cannot exceed one This PDF is often used as a model for the lifetime of a product. If X is the failure time in days of a lightbulb, then P[X > 100] is the probability that the light bulb will fail after 100 days or it will last for at least 100 days.

Important PDFs Gaussian or Normal PDF where σ2 > 0 and -∞< μ < ∞. A standard normal PDF is one for which μ = 0 and σ2 = 1. The shorthand notation is X ~ N(μ , σ) . MATLAB generates a realization of a standard normal RV using randn. To find the probability of the outcome of a Gaussian RV lying with an interval requires numerical integration since the integral cannot be evaluate analytically.

Important PDFs The normal PDF is commonly used to model noise in a communication system, as well as for numerous other applications. The normal PDF arises as the PDF of a large number of independent RV that have been added together.

Important PDFs Laplacian where σ2 > 0. The Laplacian PDF is easily integrated to find the probability of an interval. This PDF is used as a speech model.

Important PDFs Cauchy PDF It can be easily integrated to find the probability of any interval. It arises as the PDF if the ration of two independent N(0 , 1) RV. The Cauchy distribution is often used in statistics as the canonical example of a "pathological" distribution. Both its mean and its variance are undefined.

Important PDFs Gamma PDF is a very general PDF that is used for nonnegative RV Where λ>0, α>0, and Γ(z) is the Gamma function which is defined as Γ(z) is an extension of the factorial function with its argument shifted down by 1, to real and complex numbers. Γ(z) is a smooth curve that connects the points (x, y) given by  y = (x − 1)! at the positive integer values for x. Used as a normalization factor

Properties of Gamma function Property 1: Proof: Property 2: Proof: Follow from property 1 with z = N – 1 since Property 3 2π

Important PDFs The Gamma PDF reduce to many well known PDF for appropriate choices of the parameters. Exponential for α = 1 Chi-squared PDF with N degrees of freedom for α = N/2, λ = ½. Erlang α = N

Important PDFs Chi-squared PDF pX(x) Describes probability of the sum of the squares of k independent standard normal random variables. It is one of the most widely used probability distributions in inferential statistics, e.g., in hypothesis testing or in construction of confidence intervals.

Important PDFs Erlang PDF The PDF is given by two parameters: the shape k, which is a positive integer, and the rate λ, which is a positive real number. Events that occur independently with some average rate are modeled with a Poisson process. The waiting times between k occurrences of the event are Erlang distributed. This PDF can be used to determine the probability of packet loss or delay Erlang PDF λ = 2.0 λ = 2.0 λ = 2.0 λ = 1.0 λ = 0.5

Important PDFs: Examples of Gamma PDF α = 1 λ = 2.0 Exponential α = 2 λ = 2.0 α = 3 λ = 2.0 α = 5 λ = 1.0 α = 9 λ = 0.5 Chi-squared The gamma PDF generalizes the Erlang PDF by allowing k to be any real number α

Important PDFs Rayleigh PDF Rayleigh distribution naturally arises is when wind velocity is analyzed into its orthogonal 2-dimensional vector components. Assuming that the magnitudes of each component are uncorrelated, normally distributed with equal variance, and zero mean, then the overall wind speed (vector magnitude) will be characterized by a Rayleigh distribution.

Cumulative Distribution Functions The cumulative distribution function (CDF) for a continuous RV is and is evaluated using the PDF as

Cumulative Distribution Functions Uniform CDF Exponential CDF Chi-square CDF Rayleigh CDF continuous

Gaussian CDF The CDF of Gaussian PDF with μ = 0 and σ2 = 1 This cannot be evaluated further but can be found numerically. Φ(x) represents the area under the PDF to the left of the point x.

Gaussian CDF It is sometimes more convenient, however, to have knowledge of the area to the right. Shaded area = Φ(1) Shaded area = Q(1)

Gaussian CDF Properties of the Q functions Matlab implementation Q(-x) = 1 – Q(x)

Probability of error in a com. system The probability of error for PSK digital communication system is given by where N(0, 1). To explicitly evaluate Pe we have that

Probability of error in a com. system It is also sometimes important to determine A to yield a given Pe. This is found as A = 2Q-1(Pe), where Q-1 is the inverse of the Q . It is defined as the value of x necessary to yield a given value of Q(x). Q(x) can be approximated as Note that the are under the standard normal Guassian PDF is mostly contained in the interval [-3, 3]. Since the PDF is symmetric 99.8% of the probability lies within this interval

Gaussian CDF If X ~ N(μ, σ2), then the right-tail probability becomes CDF can be used to find probabilities of intervals

Probability of interval for exponential PDF Since FX(x) = 1 – exp(-λx) for x ≥ 0, we have for a > 0 and b > 0 Which should be compared to It is easy to see that finding probability using CDF (*) is easier than PDF (**) that is an advantage of CDF over PDF. (*) (**)

Relationship between PDF and CDF We obtained the CDF form the PDF, lets recover PDF from CDF. Consider a small interval [x0 – dx/2, x0 + dx/2] for a RV X and Then, we have See next slide

Relationship between PDF and CDF So that Hence, we can obtain the PDF from the CDF by differentiation or This is the fundamental theorem of calculus.

Transformations The PDF of a transformed continuous RV is given by Example: Given Y = 2X, where X ~ U(1,2), then g(x) = 2x and g-1(y) = y/2. Substituting it to (*) we get The principal difference between transformation of discrete RV and continuous RV is Jacobian factor dg-1(y)/dy. (*)

Transformations Jacobian is needed to account for the change in scaling due to the mapping of a given length interval into an interval of a different length. 4 zoomed up A total of 50 realizations were are obtained for X and Y. The generated X outcomes are shown on the x-axis and the resultant Y outcomes obtained from y = 2x are shown on the y-axis. The density of points on the y-axis is less than that on the x-axis. 3 1 2

PDF for linear(affine) transformation To determine the PDF of Y = aX + b, for a and b constants first assume that SX = {x : -∞ < x < ∞} and SY = {y : -∞ < y < ∞}. We gave g(x) = ax + b so that the inverse function g-1 is found by solving y = ax + b for x, so that and

PDF for linear transformation of normal RV Consider X ~ N(0, 1) and the transformation . Then, letting we have and therefore Y ~ N(μ, σ2). A linear transformation of a Gaussian RV results in another Gaussian RV with different parameters.

Many-to-one transformation When transformation is not one-to-one, we will have multiple solutions for in y = g(x). An example is for y = x2 for which solutions are In this case we must add the PDFs to yield In general, if y = g(x) have solutions xi = gi-1(y) for i = 1,2,…,M, then

Alternative approach to determine PDF of a transformed RV An alternative means of finding the PDF of a transformed RV is to first find the CDF and then differentiate it. Example: CDF approach to determine PDF of Y = X2 for X ~ N(0, 1). Then differentiating we have using chain rule Since pX(-x) = pX(x) for X ~ N(0, 1)

Example: PDF of Y=X2 for X ~ N(0,1) Since -∞ < x < ∞ we must Y ≥ 0. Net because and we have Which reduces to Y ~ χ12 Chi square RV with N = 1 The PDF is undefined at y = 0 since pY(0)  ∞, although the total area under the PDF is equal to 1.

Setting Clipping Levels for Speech Signals To prevent clipping in the sound communication system due to analog-to-digital conversion it is necessary to to determine the highest amplitude of the speech signal that can be expected to occur. The distribution of amplitudes is modeled by Laplacian PDF i.e. A design requirement might be to transmit a speech signal without clipping 99% of time. A model for a clipper is

Setting Clipping Levels for Speech Signals Clipping will occur whenever |X| > A. To satisfy the design requirement (not clipping for 99% of time), we should choose A so that pclip ≤ 0.01. And since the Laplacian PDF is symmetric about x = 0 this is just

Setting Clipping Levels for Speech Signals Hence, if this probability is to be no more than 0.01, we must have or solving for A produces the requirement that As example consider a speech signal with σ2 = 1, assume a clipping level of A = 1. The probability of clipping is

Setting Clipping Levels for Speech Signals 50 outcomes of a Laplacian model with σ2 = 1 With A = 1, we would expect about 500.2431≈12 instances of clipping. To meet specification we should have that clipping level to clip only 1% of times.

Exercises Find probability of interval [a, b] for exponential PDF. A wedge of cheese is sliced from x= a to x = b. If a = 0 and b = 0.2, what is the mass of cheese in the wedge? How about it a = 1.8 and b= 2? Which of the function below are valid PDFs? If a function is not a PDF, why not?

Exercises Determine the value of c to make the following function a valid PDF A Gaussian mixture PDF is defined as for σ12 ≠ σ22. What are the possible values for α1 and α2 so that thus is a valid PDF? A memory chip has a projected lifetime X in days that is modeled as X ~ exp(0.001). What is the probability that it will fail within one year?