PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Department of Computer Engineering Sharif University of Technology Mean, Variance, Moments and Characteristic Functions

Sharif University of Technology 2 Mean and Variance  p.d.f of a r.v represents complete information about it.  quite often it is desirable to characterize the r.v in terms of its average behavior.  We introduce two parameters -Mean -Variance  represent the overall properties of the r.v and its p.d.f.

Sharif University of Technology 3 Mean  Mean or the Expected Value of a r.v X is defined as  If X is a discrete-type r.v,  Mean represents the average value of the r.v in a very large number of trials. This is the midpoint of the interval (a,b). Example: Uniform Distribution

Sharif University of Technology 4 Mean Example: Exponential Distribution

Sharif University of Technology 5 Mean Example: Poisson Distribution

Sharif University of Technology 6 Mean Example: Binomial Distribution

Sharif University of Technology 7 Mean Example: Normal Distribution

Sharif University of Technology 8 Functions of r.vs  Given  suppose defines a new r.v with p.d.f  Y has a mean  To determine we don’t need to determine  Recall that for any y, where represent the multiple solutions of the equation  This can be rewritten as where the terms form nonoverlapping intervals.

Sharif University of Technology 9 Functions of r.vs – continued  Hence  and hence as y covers the entire y -axis,  the corresponding Δ x ’s are nonoverlapping, and they cover the entire x -axis.  As  Which in the discrete case, reduces to:  So, is not required to evaluate for

Sharif University of Technology 10 Functions of r.vs – continued  Determine the mean of where X is a Poisson r.v. Example is known as the kth moment of r.v X

Sharif University of Technology 11 Another Measure  Mean alone will not be able to truly represent the p.d.f of any r.v.  Consider two Gaussian r.vs and  Despite having the same mean, their p.d.fs are quite different as one is more concentrated around the mean.  We need an additional parameter to measure this spread around the mean. (a) (b)

Sharif University of Technology 12 Variance and Standard Deviation  For a r.v X with mean (positive or negative) represents the deviation of the r.v from its mean.  So, consider the quantity and its average value represents the average mean square deviation of X around its mean.  Define  Using with we have:  The standard deviation represents the root mean square spread of the r.v X around its mean. variance of the r.v X standard deviation of X

Sharif University of Technology 13 Variance and Standard Deviation This can be used to compute Example: Poisson Distribution

Sharif University of Technology 14 Variance and Standard Deviation  To simplify this, we can make use  For a normal p.d.f. This gives  Differentiating both sides of this equation with respect to we get  or Example: Normal Distribution

Sharif University of Technology 15 Moments  in general are known as the moments of the r.v X, and are known as the central moments of X.  Clearly, and  It is easy to relate and  generalized moments of X about a,  absolute moments of X.

Sharif University of Technology 16 Variance and Standard Deviation  It can be shown that Example: Normal Distribution

Sharif University of Technology 17 Characteristic Function  The characteristic function of a r.v X is defined as  and for all  For discrete r.vs:

Sharif University of Technology 18 Example Example: Poisson Distribution

Sharif University of Technology 19 Example Example: Binomial Distribution

Sharif University of Technology 20 Characteristic Function and Moments  Taking the first derivative of this equation with respect to , and letting it to be equal to zero, we get  Similarly, the second derivative gives

Sharif University of Technology 21 Characteristic Function and Moments – continued  repeating this procedure k times, we obtain the k th moment of X to be  We can use these results to compute the mean, variance and other higher order moments of any r.v X.

Sharif University of Technology 22 Example These results agree with previous ones, but the efforts involved in the calculations are very minimal. Example: Poisson Distribution So,

Sharif University of Technology 23 Example Example: Mean and Variance of Binomial Distribution So,

Sharif University of Technology 24 Example The characteristic function of a Gaussian r.v itself has the “Gaussian” bell shape. Example: Normal(Guassian) Distribution

Sharif University of Technology 25 Example - continued  Note the reverse roles of in and (b) (a) For we have and therefore:

Sharif University of Technology 26 Example Example: Cauchy Distribution Which diverges to infinity. Similarly: With So the double sided integral for mean is undefined. It concludes that the mean and variance of a Cauchy r.v are undefined.

Sharif University of Technology 27 we can start with the definition of So the desired probability is: Chebychev Inequality  Consider an interval of width 2 ε symmetrically centered around its mean μ.  What is the probability that X falls outside this interval? Chebychev inequality

Sharif University of Technology 28 Chebychev Inequality  So, the knowledge of is not necessary. We only need  With we obtain  Thus with we get the probability of X being outside the 3σ interval around its mean to be for any r.v.  Obviously this cannot be a tight bound as it includes all r.vs.  For example, in the case of a Gaussian r.v, from the Table,  Chebychev inequality always underestimates the exact probability.

Sharif University of Technology 29 Moment Identities  Suppose X is a discrete random variable that takes only nonnegative integer values. i.e.,  Then  Similarly,  Which gives: These equations are at times quite useful in simplifying calculations.

Sharif University of Technology 30 Example: Birthday Pairing  In a group of n people, what is A) The probability that two or more persons will have the same birthday? B) The probability that someone in the group will have the birthday that matches yours? Solution C = “no two persons have the same birthday” For n=23,

Sharif University of Technology 31 Example – continued B)D = “a person not matching your birthday” let X represent the minimum number of people in a group for a birthday pair to occur. The probability that “the first n people selected from that group have different birthdays” is given by

Sharif University of Technology 32 Example – continued  But the event the “the first n people selected havedifferent birthdays” is the same as the event “ X > n.”  Hence  Using moment identities, this gives the mean value of X to be

Sharif University of Technology 33 Example – continued  Similarly,  Thus, and  Since the standard deviation is quite high compared to the mean value, the actual number of people required could be anywhere from 25 to 40.

Sharif University of Technology 34 Moment Identities for Continuous r.vs if X is a nonnegative random variable with density function f X (x) and distribution function F X (X), then where Similarly