EMIS 7300 SYSTEMS ANALYSIS METHODS FALL 2005 Dr. John Lipp Copyright © 2002 - 2005 John Lipp

Copyright  2002 - 2005 Dr. John Lipp Session 1 Outline Part 1: The Statistics You Thought You Knew. Part 2: Probability Theory. Part 3: Discrete Random Variables. Part 4: Continuous Random Variables. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Today’s Session Topics Part 4: Continuous Random Variables. Continuous Random Variables. Continuous PDF and CDF. Stochastic Expectation. Mixed Random Variables. Uniform Random Variables. Gaussian Random Variables. Central Limit Theorem. Chi-Square Random Variables. Exponential Random Variables Rayleigh Random Variables. Cauchy Random Variables. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous Random Variables The elementary outcomes are mapped to real numbers, i.e., x = X(E), where x is a real number and E is an outcome in a sample space S. E S x = X(E) x -3 -2 -1 1 2 3 EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous Random Variables (cont.) The number of possible x values is infinite. Question: can the mean calculation be performed when the xi are real numbers? Or put differently, can the real numbers be made into a list such that {x1, x2, x3, …, x} contains all possible real numbers to which the random variable maps? EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous Random Variables (cont.) Proof by contradiction. Consider just the real numbers between 0 and 1. Write them as decimals, e.g., 0.598248766… Assume that the (infinite) list of numbers shown below contains all the real numbers between 0 and 1. {0.887871659879013691591325…, 0.698173286190238298539754…, 0.213498077356359723983947…, 0.162390283987279837459734…, 0.903743761618759869834982…, …} Find a real number between 0 and 1 not in the list! EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous RV - Cumulative Distribution Function For a discrete random variable, P(X = xi) = fX[xi]. For a (pure) continuous random variable P(X = x) = 0! Therefore the definition of PDF and CDF needs to be redefined for continuous random variables. Start by defining the cumulative distribution function (CDF) as P(X  x) = FX(x). S E X(E)  x x -3 -2 -1 1 2 3 EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous RV - Cumulative Distribution Function (cont.) The CDF for a continuous RV has some basic properties: FX(-) = P(X  -) = 0. FX(+) = P(X  +) = 1. FX(x)  FX(y) when x  y (monotone increasing). FX(x) may contain jump discontinuities (special case). FX(x) is right-continuous. 1 x -3 -2 -1 1 2 3 EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous RV - Probability Distribution Function Use the Fundamental Theorem of Calculus, where The function fX(x) is the probability density function (PDF) for a continuous random variable. Note The PDF really has no meaning outside of an integral. Specifically, P(X = x)  fX(x). [Exception: the special case of jump discontinuities in the CDF.]. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Continuous RV - Probability Distribution Function (cont.) The PDF has some basic properties: fX(x)  0 (because FX(x) is monotone increasing). fX(x) may contain Dirac delta functions (from the jump discontinuities in FX(x)). The area under fX(x) is unity, Expectation is now an integral which involves the PDF: FX(x) fX(x) EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

The Mean, Mode, Median, and Variance The mean of a continuous random variable is The mode is the value of x at the maximum (peak) of fX(x). The median is the value of x at which it is equally probable for X to have a value above or below the median, that is, The mode, median, or mean are not necessarily the same values! The variance of a continuous random variable is EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Copyright  2002 - 2005 Dr. John Lipp Mixed Distributions The CDF can be separated as the sum of two functions: a continuous function and a sum of step functions. 1 FXc(x) x -3 -2 -1 1 2 3 1 FXd(x) x -3 -2 -1 1 2 3 EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Mixed Distributions (cont.) The continuous portion represents a “pure” continuous random variable and the step portion represents a “pure” discrete random variable. The latter’s continuous PDF is a sum of Dirac delta functions, 1 fXd(x) = a(x - x1) x -3 -2 -1 1 2 3 EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Mixed Distributions (cont.) Recall the definition of discrete PDF, fX[xi] = P(X = xi). Then, Using the sifting property of the Dirac delta function yields the expected result for the discrete portion of a continuous CDF, where u(x) is the unit step function. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Time to Gather some Data Aim at the bull’s eye using the dart gun’s sights. Measure the X and Y displacements from the bull’s eye. Collect 10 data sample pairs for your dart gun. Plot the data on the provided probability paper. Y X EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Uniform A most simple distribution. The name comes from its PDF. Vital statistics (mean, variance, median, NO mode): 1 1 b - a x a b -3 -2 -1 1 2 3 EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Gaussian The Gaussian distribution is also known as the normal distribution and is often written in shorthand as N(,  2). PDF: s = 1.5 m = 1 95.5% 99.7% 68.3% EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Gaussian (cont.) CDF: Mean is  and variance is  2. The sum of any number of Gaussian random variables (and constants), independent or otherwise, and most any linear operation on a Gaussian random variable, is a Gaussian random variable. Insert Plot of CDF here. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Gaussian (cont.) Example: Let X be N(x, x2). Define Y = mX + b. Then Y is Gaussian with mean and variance y = E{Y} = E{mX + b} = mx + b y2 = E{Y 2}- y2 = E{(mX + b)2} – (mx + b)2 = E{m 2 X 2 + 2mbX + b 2} – (mx + b)2 = m 2(x2 + x2) + 2mbx + b 2 – (mx + b)2 = m2x2 Thus, the PDF of Y is EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Gaussian (cont.) A normal random variable N(0,1) is known as the standard normal random variable. It is often denoted as Z. If X ~ N(x, x2) then a standard RV is . EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Gaussian (Normal) Probability Distribution (cont.) The sum of any number of normally distributed data values (and constants), from independent experiments or otherwise, and most any linear operation on normally distributed data, is also normally distributed. The sample mean of normally distributed data {x1, x2, …, xn} is normally distributed: Sum the data, then Divide by a scalar, n, a linear operation. Neither the sample variance nor the sample standard deviation of normally distributed data {x1, x2, …, xn} are normally distributed: Both involve the sum of squared normal data. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

“Remove” the mean and variance from Y, that is, standardize: Central Limit Theorem Let Y be the sum (or average) of a “very large” number of mutually independent random variables Xi with identical PDFs and finite means and variances. “Remove” the mean and variance from Y, that is, standardize: Z tends towards a standard normal random variable. In general, the sum of even a small number of independent identically distributed (i.i.d.) random variables tends towards a Gaussian distribution. Needs Better Explanation! (Work out more details) EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Central Limit Theorem (cont.) Consider the random variable UN defined to be the sum of i.i.d. uniform random variables ranged between -1 and +1, Use the following theorem: If X and Y are independent random variables, the PDF of Z = X + Y is the convolution of the PDFs of X and Y, Repeatedly applying this theory to generate UN ... Show Sum As Pictures EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Central Limit Theorem (cont.) EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Statistical Modeling (cont.) It is possible for both the response’s mean and variance to be functions of the independent variables, the distribution to be other than the normal, in various combinations: Mean Variance PDF Constant Normal m(x1,x2,…,xm) s 2(x1,x2,…,xm) Other Data Transforms approximate simpler forms Increasing Analysis Difficulty EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Chi-Square Let X be a zero mean (x = 0) Gaussian random variable with variance x2. Then Y = X 2 has a chi-square distribution (with one degree of freedom). PDF: Insert Plot of PDF here. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Chi-Square (cont.) The mean is x2 and the variance is 2x4. The median is  0.4550 x2. (Found numerically.) Mode = 0. Show how it is easy to show the mean. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Chi-Square (cont.) EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Exponential Let X1 and X2 be zero mean (x = 0) independent Gaussian random variables with equal variances x2. Then Y = X12 + X22 has an exponential distribution with parameter  = 2 x2. PDF: Insert Plot of PDF here. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Exponential (cont.) CDF: The mean is , the variance is 2 2, the median is - ln(2), and the mode = 0. Let U be a random variable uniformly distributed between 0 and 1. Then Y = -  ln(U) is exponentially distributed. Note: The textbook has a different “definition.” Insert Plot of CDF here. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Chi-Square with n-degrees of freedom Let X1, X2, …, Xn be zero mean (x = 0) independent Gaussian random variables with equal variances x2. Then Y = X12 + X22 + … + Xn2 has a chi-square distribution with n-degrees of freedom. PDF: EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Chi-Square with n-degrees of freedom (cont.) The gamma function (r) is a “continuous” generalization of the factorial (needed to count continuous things) The mean is nx2 and the variance is 2nx4. Show that for n = 2 it is consistent with exponential. Show motivation: analysis of standard deviation / variance. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Rayleigh Let X1 and X2 be zero mean (x = 0) independent Gaussian random variables with equal variances x2. Then has a Rayleigh distribution ( = x2). The PDF : EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Rayleigh (cont.) CDF: The mean is , the variance (2 - /2), the mode , the median 2 ln(2). The Rayleigh distribution has important applications in miss distance calculations, radar and other coherent signal systems. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Rayleigh (cont.) A ballistic shell is required to hit within 1 meter of a target. After several trials it is observed that the downrange and crossrange miss distances are independent and Gaussian with a standard deviation of 0.8 meters. Thus, the radial miss distance, the RSS (root sum squares) of the downrange and crossrange, is Rayleigh distributed with  = 0.64. The Ph is then the CDF of 1, EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Copyright  2002 - 2005 Dr. John Lipp Distribution: Cauchy Let X1 and X2 be zero mean (x = 0) independent Gaussian random variables with equal variances x2. Then Y = X1 / X2 is Cauchy distributed with parameter  = 1. PDF: EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Distribution: Cauchy (cont.) The Cauchy distribution has no mean or variance! Why? The mode and median are both zero. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp

Copyright  2002 - 2005 Dr. John Lipp Homework Mandatory (answers in the back of the book): 4-1 4-13 4-29 4-41 4-47 4-79 4-101 R  Mandatory: Subtract the X and Y sample means from your dart gun data and compute the radial miss distance and angle errors. Using probability paper, attempt to determine the appropriate PDF of the radial and angle errors. EMIS 7300 Copyright  2002 - 2005 Dr. John Lipp