1. EMIS 7300 SYSTEMS ANALYSIS METHODS FALL 2005
Dr. John Lipp
Copyright © John Lipp

2. Copyright  2002 - 2005 Dr. John Lipp
Session 1 Outline
Part 1: The Statistics You Thought You Knew.
Part 2: Probability Theory.
Part 3: Discrete Random Variables.
Part 4: Continuous Random Variables.
EMIS 7300
Copyright  Dr. John Lipp

3. Today’s Session Topics
Part 4: Continuous Random Variables.
Continuous Random Variables.
Continuous PDF and CDF.
Stochastic Expectation.
Mixed Random Variables.
Uniform Random Variables.
Gaussian Random Variables.
Central Limit Theorem.
Chi-Square Random Variables.
Exponential Random Variables
Rayleigh Random Variables.
Cauchy Random Variables.
EMIS 7300
Copyright  Dr. John Lipp

4. Continuous Random Variables
The elementary outcomes are mapped to real numbers, i.e., x = X(E), where x is a real number and E is an outcome in a sample space S. E S
x = X(E) x -3 -2 -1 1 2 3
EMIS 7300
Copyright  Dr. John Lipp

5. Continuous Random Variables (cont.)
The number of possible x values is infinite.
Question: can the mean calculation
be performed when the xi are real numbers?
Or put differently, can the real numbers be made into a list such that {x1, x2, x3, …, x} contains all possible real numbers to which the random variable maps?
EMIS 7300
Copyright  Dr. John Lipp

6. Continuous Random Variables (cont.)
Proof by contradiction.
Consider just the real numbers between 0 and 1.
Write them as decimals, e.g., …
Assume that the (infinite) list of numbers shown below contains all the real numbers between 0 and 1.
{ …, …, …, …, …, …}
Find a real number between 0 and 1 not in the list!
EMIS 7300
Copyright  Dr. John Lipp

7. Continuous RV - Cumulative Distribution Function
For a discrete random variable, P(X = xi) = fX[xi].
For a (pure) continuous random variable P(X = x) = 0!
Therefore the definition of PDF and CDF needs to be redefined for continuous random variables.
Start by defining the cumulative distribution function (CDF) as P(X  x) = FX(x). S E
X(E)  x x -3 -2 -1 1 2 3
EMIS 7300
Copyright  Dr. John Lipp

8. Continuous RV - Cumulative Distribution Function (cont.)
The CDF for a continuous RV has some basic properties:
FX(-) = P(X  -) = 0.
FX(+) = P(X  +) = 1.
FX(x)  FX(y) when x  y (monotone increasing).
FX(x) may contain jump discontinuities (special case).
FX(x) is right-continuous. 1 x -3 -2 -1 1 2 3
EMIS 7300
Copyright  Dr. John Lipp

9. Continuous RV - Probability Distribution Function
Use the Fundamental Theorem of Calculus,
where
The function fX(x) is the probability density function (PDF) for a continuous random variable. Note
The PDF really has no meaning outside of an integral. Specifically, P(X = x)  fX(x). [Exception: the special case of jump discontinuities in the CDF.].
EMIS 7300
Copyright  Dr. John Lipp

10. Continuous RV - Probability Distribution Function (cont.)
The PDF has some basic properties:
fX(x)  0 (because FX(x) is monotone increasing).
fX(x) may contain Dirac delta functions (from the jump discontinuities in FX(x)).
The area under fX(x) is unity,
Expectation is now an integral which involves the PDF:
FX(x)
fX(x)
EMIS 7300
Copyright  Dr. John Lipp

11. The Mean, Mode, Median, and Variance
The mean of a continuous random variable is
The mode is the value of x at the maximum (peak) of fX(x).
The median is the value of x at which it is equally probable for X to have a value above or below the median, that is,
The mode, median, or mean are not necessarily the same values!
The variance of a continuous random variable is
EMIS 7300
Copyright  Dr. John Lipp

12. Copyright  2002 - 2005 Dr. John Lipp
Mixed Distributions
The CDF can be separated as the sum of two functions: a continuous function and a sum of step functions. 1
FXc(x) x -3 -2 -1 1 2 3 1
FXd(x) x -3 -2 -1 1 2 3
EMIS 7300
Copyright  Dr. John Lipp

13. Mixed Distributions (cont.)
The continuous portion represents a “pure” continuous random variable and the step portion represents a “pure” discrete random variable.
The latter’s continuous PDF is a sum of Dirac delta functions, 1
fXd(x) = a(x - x1) x -3 -2 -1 1 2 3
EMIS 7300
Copyright  Dr. John Lipp

14. Mixed Distributions (cont.)
Recall the definition of discrete PDF, fX[xi] = P(X = xi). Then,
Using the sifting property of the Dirac delta function yields the expected result for the discrete portion of a continuous CDF,
where u(x) is the unit step function.
EMIS 7300
Copyright  Dr. John Lipp

15. Time to Gather some Data
Aim at the bull’s eye using the dart gun’s sights.
Measure the X and Y displacements from the bull’s eye.
Collect 10 data sample pairs for your dart gun.
Plot the data on the provided probability paper. Y X
EMIS 7300
Copyright  Dr. John Lipp

16. Distribution: Uniform
A most simple distribution. The name comes from its PDF.
Vital statistics (mean, variance, median, NO mode): 1 1
b - a x a b -3 -2 -1 1 2 3
EMIS 7300
Copyright  Dr. John Lipp

17. Distribution: Gaussian
The Gaussian distribution is also known as the normal distribution and is often written in shorthand as N(,  2).
PDF:
s = 1.5
m = 1
95.5%
99.7%
68.3%
EMIS 7300
Copyright  Dr. John Lipp

18. Distribution: Gaussian (cont.)
CDF:
Mean is  and variance is  2.
The sum of any number of Gaussian random variables (and constants), independent or otherwise, and most any linear operation on a Gaussian random variable, is a Gaussian random variable.
Insert Plot of CDF here.
EMIS 7300
Copyright  Dr. John Lipp

19. Distribution: Gaussian (cont.)
Example:
Let X be N(x, x2). Define Y = mX + b. Then Y is Gaussian with mean and variance
y = E{Y} = E{mX + b} = mx + b
y2 = E{Y 2}- y2 = E{(mX + b)2} – (mx + b)2
= E{m 2 X 2 + 2mbX + b 2} – (mx + b)2
= m 2(x2 + x2) + 2mbx + b 2 – (mx + b)2
= m2x2
Thus, the PDF of Y is
EMIS 7300
Copyright  Dr. John Lipp

20. Distribution: Gaussian (cont.)
A normal random variable N(0,1) is known as the standard normal random variable. It is often denoted as Z.
If X ~ N(x, x2) then a standard RV is
EMIS 7300
Copyright  Dr. John Lipp

21. Gaussian (Normal) Probability Distribution (cont.)
The sum of any number of normally distributed data values (and constants), from independent experiments or otherwise, and most any linear operation on normally distributed data, is also normally distributed.
The sample mean of normally distributed data {x1, x2, …, xn} is normally distributed:
Sum the data, then
Divide by a scalar, n, a linear operation.
Neither the sample variance nor the sample standard deviation of normally distributed data {x1, x2, …, xn} are normally distributed:
Both involve the sum of squared normal data.
EMIS 7300
Copyright  Dr. John Lipp

22. “Remove” the mean and variance from Y, that is, standardize:
Central Limit Theorem
Let Y be the sum (or average) of a “very large” number of mutually independent random variables Xi with identical PDFs and finite means and variances.
“Remove” the mean and variance from Y, that is, standardize:
Z tends towards a standard normal random variable.
In general, the sum of even a small number of independent identically distributed (i.i.d.) random variables tends towards a Gaussian distribution.
Needs Better Explanation!
(Work out more details)
EMIS 7300
Copyright  Dr. John Lipp

23. Central Limit Theorem (cont.)
Consider the random variable UN defined to be the sum of i.i.d. uniform random variables ranged between -1 and +1,
Use the following theorem:
If X and Y are independent random variables, the PDF of Z = X + Y is the convolution of the PDFs of X and Y,
Repeatedly applying this theory to generate UN ...
Show Sum
As Pictures
EMIS 7300
Copyright  Dr. John Lipp

24. Central Limit Theorem (cont.)
EMIS 7300
Copyright  Dr. John Lipp

25. Statistical Modeling (cont.)
It is possible for both the response’s mean and variance to be functions of the independent variables, the distribution to be other than the normal, in various combinations:
Mean
Variance
PDF
Constant
Normal
m(x1,x2,…,xm)
s 2(x1,x2,…,xm)
Other
Data
Transforms
approximate
simpler
forms
Increasing Analysis Difficulty
EMIS 7300
Copyright  Dr. John Lipp

26. Distribution: Chi-Square
Let X be a zero mean (x = 0) Gaussian random variable with variance x2.
Then Y = X 2 has a chi-square distribution (with one degree of freedom).
PDF:
Insert Plot of PDF here.
EMIS 7300
Copyright  Dr. John Lipp

27. Distribution: Chi-Square (cont.)
The mean is x2 and the variance is 2x4.
The median is  x2. (Found numerically.)
Mode = 0.
Show how it is easy to show the mean.
EMIS 7300
Copyright  Dr. John Lipp

28. Distribution: Chi-Square (cont.)
EMIS 7300
Copyright  Dr. John Lipp

29. Distribution: Exponential
Let X1 and X2 be zero mean (x = 0) independent Gaussian random variables with equal variances x2.
Then Y = X12 + X22 has an exponential distribution with parameter  = 2 x2.
PDF:
Insert Plot of PDF here.
EMIS 7300
Copyright  Dr. John Lipp

30. Distribution: Exponential (cont.)
CDF:
The mean is , the variance is 2 2, the median is - ln(2), and the mode = 0.
Let U be a random variable uniformly distributed between and 1. Then Y = -  ln(U) is exponentially distributed.
Note: The textbook has a different “definition.”
Insert Plot of CDF here.
EMIS 7300
Copyright  Dr. John Lipp

31. Distribution: Chi-Square with n-degrees of freedom
Let X1, X2, …, Xn be zero mean (x = 0) independent Gaussian random variables with equal variances x2.
Then Y = X12 + X22 + … + Xn2 has a chi-square distribution with n-degrees of freedom.
PDF:
EMIS 7300
Copyright  Dr. John Lipp

32. Distribution: Chi-Square with n-degrees of freedom (cont.)
The gamma function (r) is a “continuous” generalization of the factorial (needed to count continuous things)
The mean is nx2 and the variance is 2nx4.
Show that for n = 2 it is consistent with exponential.
Show motivation: analysis of standard deviation / variance.
EMIS 7300
Copyright  Dr. John Lipp

33. Distribution: Rayleigh
Let X1 and X2 be zero mean (x = 0) independent Gaussian random variables with equal variances x2.
Then has a Rayleigh distribution ( = x2).
The PDF :
EMIS 7300
Copyright  Dr. John Lipp

34. Distribution: Rayleigh (cont.)
CDF:
The mean is , the variance (2 - /2), the mode , the median 2 ln(2).
The Rayleigh distribution has important applications in miss distance calculations, radar and other coherent signal systems.
EMIS 7300
Copyright  Dr. John Lipp

35. Distribution: Rayleigh (cont.)
A ballistic shell is required to hit within 1 meter of a target.
After several trials it is observed that the downrange and crossrange miss distances are independent and Gaussian with a standard deviation of 0.8 meters.
Thus, the radial miss distance, the RSS (root sum squares) of the downrange and crossrange, is Rayleigh distributed with  = 0.64.
The Ph is then the CDF of 1,
EMIS 7300
Copyright  Dr. John Lipp

36. Copyright  2002 - 2005 Dr. John Lipp
Distribution: Cauchy
Let X1 and X2 be zero mean (x = 0) independent Gaussian random variables with equal variances x2.
Then Y = X1 / X2 is Cauchy distributed with parameter  = 1.
PDF:
EMIS 7300
Copyright  Dr. John Lipp

37. Distribution: Cauchy (cont.)
The Cauchy distribution has no mean or variance! Why?
The mode and median are both zero.
EMIS 7300
Copyright  Dr. John Lipp

38. Copyright  2002 - 2005 Dr. John Lipp
Homework
Mandatory (answers in the back of the book): R 
Mandatory: Subtract the X and Y sample means from your dart gun data and compute the radial miss distance and angle errors. Using probability paper, attempt to determine the appropriate PDF of the radial and angle errors.
EMIS 7300
Copyright  Dr. John Lipp