STATISTICS Random Variables and Probability Distributions

STATISTICS Random Variables and Probability Distributions
Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University

Definition of random variable (RV)
For a given probability space ( ,A, P[]), a random variable, denoted by X or X(), is a function with domain  and counterdomain the real line. The function X() must be such that the set Ar, denoted by , belongs to A for every real number r. Unlike the probability which is defined on the event space, a random variable is defined on the sample space. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

is defined whereas is not defined.
Random experiment Sample space Event space Probability space is defined whereas is not defined. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Cumulative distribution function (CDF)
The cumulative distribution function of a random variable X, denoted by , is defined to be 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Consider the experiment of tossing two fair coins
Consider the experiment of tossing two fair coins. Let random variable X denote the number of heads. CDF of X is 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Indicator function or indicator variable
Let  be any space with points  and A any subset of . The indicator function of A, denoted by , is the function with domain  and counterdomain equal to the set consisting of the two real numbers 0 and 1 defined by 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Discrete random variables
A random variable X will be defined to be discrete if the range of X is countable. If X is a discrete random variable with values then the function denoted by and defined by is defined to be the discrete density function of X. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Continuous random variables
A random variable X will be defined to be continuous if there exists a function such that for every real number x. The function is called the probability density function of X. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Properties of a CDF is continuous from the right, i.e. 3/27/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Properties of a PDF 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Example 1 Determine which of the following are valid distribution functions: 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Example 2 Determine the real constant a, for arbitrary real constants m and 0 < b, such that is a valid density function. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Function is symmetric about m.

Characterizing random variables
Cumulative distribution function Probability density function Expectation (expected value) Variance Moments Quantile Median Mode 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Expectation of a random variable
The expectation (or mean, expected value) of X, denoted by or E(X) , is defined by: 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Rules for expectation Let X and Xi be random variables and c be any real constant. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Variance of a random variable

is called the standard deviation of X.
Variance characterizes the dispersion of data with respect to the mean. Thus, shifting a density function does not change its variance. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Rules for variance 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Events A and B are defined to be independent if and only if
Two random variables are said to be independent if knowledge of the value assumed by one gives no clue to the value assumed by the other. Events A and B are defined to be independent if and only if 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Moments and central moments of a random variable

Properties of moments 3/27/2017

Quantile The qth quantile of a random variable X, denoted by , is defined as the smallest number satisfying Discrete Uniform 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Median and mode The median of a random variable is the 0.5th quantile, or The mode of a random variable X is defined as the value u at which is the maximum of 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Note: For a positively skewed distribution, the mean will always be the highest estimate of central tendency and the mode will always be the lowest estimate of central tendency (assuming that the distribution has only one mode). For negatively skewed distributions, the mean will always be the lowest estimate of central tendency and the mode will be the highest estimate of central tendency. In any skewed distribution (i.e., positive or negative) the median will always fall in-between the mean and the mode. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Moment generating function

Usage of MGF MGF can be used to express moments in terms of PDF parameters and such expressions can again be used to express mean, variance, coefficient of skewness, etc. in terms of PDF parameters. Random variables of the same MGF are associated with the same type of probability distribution. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The moment generating function of a sum of independent random variables is the product of the moment generating functions of individual random variables. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Expected value of a function of a random variable

If Y=g(X) 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Y Y=g(X) y X x1 x2 x3 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Theorem 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Chebyshev Inequality 3/27/2017

The Chebyshev inequality gives a bound, which does not depend on the distribution of X, for the probability of particular events described in terms of a random variable and its mean and variance. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Probability density functions of discrete random variables
Discrete uniform distribution Bernoulli distribution Binomial distribution Negative binomial distribution Geometric distribution Hypergeometric distribution Poisson distribution 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Discrete uniform distribution
N ranges over the possible integers. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Bernoulli distribution
1-p is often denoted by q. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Binomial distribution
Binomial distribution represents the probability of having exactly x success in n independent and identical Bernoulli trials. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Negative binomial distribution
Negative binomial distribution represents the probability of achieving the r-th success in x independent and identical Bernoulli trials. Unlike the binomial distribution for which the number of trials is fixed, the number of successes is fixed and the number of trials varies from experiment to experiment. The negative binomial random variable represents the number of trials needed to achieve the r-th success. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Geometric distribution
Geometric distribution represents the probability of obtaining the first success in x independent and identical Bernoulli trials. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Hypergeometric distribution
where M is a positive integer, K is a nonnegative integer that is at most M, and n is a positive integer that is at most M. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Let X denote the number of defective products in a sample of size n when sampling without replacement from a box containing M products, K of which are defective. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Poisson distribution The Poisson distribution provides a realistic model for many random phenomena for which the number of occurrences within a given scope (time, length, area, volume) is of interest. For example, the number of fatal traffic accidents per day in Taipei, the number of meteorites that collide with a satellite during a single orbit, the number of defects per unit of some material, the number of flaws per unit length of some wire, etc. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Assume that we are observing the occurrence of certain happening in time, space, region or length. Also assume that there exists a positive quantity which satisfies the following properties: 1. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

2. 3. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The probability of success (occurrence) in each trial.

Comparison of Poisson and Binomial distributions

Example Suppose that the average number of telephone calls arriving at the switchboard of a company is 30 calls per hour. (1) What is the probability that no calls will arrive in a 3-minute period? (2) What is the probability that more than five calls will arrive in a 5-minute interval? Assume that the number of calls arriving during any time period has a Poisson distribution. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Assuming time is measured in minutes
Poisson distribution is NOT an appropriate choice. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Assuming time is measured in seconds
Poisson distribution is an appropriate choice. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

represents the time length measured in scale of .
The first property provides the basis for transferring the mean rate of occurrence between different observation scales. The “small time interval of length h” can be measured in different observation scales. represents the time length measured in scale of . is the mean rate of occurrence when observation scale is used. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

If the first property holds for various observation scales, say , then it implies the probability of exactly one happening in a small time interval h can be approximated by The probability of more than one happenings in time interval h is negligible. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

probability that more than five calls will arrive in a 5-minute interval
Occurrences of events which can be characterized by the Poisson distribution is known as the Poisson process. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Probability density functions of continuous random variables
Uniform or rectangular distribution Normal distribution (also known as the Gaussian distribution) Exponential distribution (or negative exponential distribution) Gamma distribution (Pearson Type III) Chi-squared distribution Lognormal distribution 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Uniform or rectangular distribution

PDF of U(a,b) 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Normal distribution (Gaussian distribution)

Z 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

X~N(μ1, σ1) Z~N(0,1) Y~N(μ2, σ2) 3/27/2017

Commonly used values of normal distributions

Exponential distribution (negative exponential distribution)
Mean rate of occurrence in a Poisson process. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Gamma distribution represents the mean rate of occurrence in a Poisson process. is equivalent to  in the exponential density. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The exponential distribution is a special case of gamma distribution with
The sum of n independent identically distributed exponential random variables with parameter has a gamma distribution with parameters 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Pearson Type III distribution (PT3)
, and are the mean, standard deviation and skewness coefficient of X, respectively. It reduces to Gamma distribution if = 0. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The Pearson type III distribution is widely applied in stochastic hydrology.
Total rainfall depths of storm events can be characterized by the Pearson type III distribution. Annual maximum rainfall depths are also often characterized by the Pearson type III or log-Pearson type III distribution. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Chi-squared distribution
The chi-squared distribution is a special case of the gamma distribution with 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Log-Normal Distribution Log-Pearson Type III Distribution (LPT3)
A random variable X is said to have a log-normal distribution if Log(X) is distributed with a normal density. A random variable X is said to have a Log-Pearson type III distribution if Log(X) has a Pearson type III distribution. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Lognormal distribution

Approximations between random variables
Approximation of binomial distribution by Poisson distribution Approximation of binomial distribution by normal distribution Approximation of Poisson distribution by normal distribution 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Approximation of binomial distribution by Poisson distribution

Approximation of binomial distribution by normal distribution
Let X have a binomial distribution with parameters n and p. If , then for fixed a<b, is the cumulative distribution function of the standard normal distribution. It is equivalent to say that as n approaches infinity X can be approximated by a normal distribution with mean np and variance npq. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Approximation of Poisson distribution by normal distribution
Let X have a Poisson distribution with parameter . If , then for fixed a<b It is equivalent to say that as  approaches infinity X can be approximated by a normal distribution with mean  and variance . 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Example Suppose that two fair dice are tossed 600 times. Let X denote the number of times that a total of 7 dots occurs. What is the probability that ? 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Transformation of random variables
[Theorem] Let X be a continuous RV with density fx. Let Y=g(X), where g is strictly monotonic and differentiable. The density for Y, denoted by fY, is given by 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Proof: Assume that Y=g(X) is a strictly monotonic increasing function of X.

Example Let X be a gamma random variable with
Y is also a gamma random variable with scale parameter and shape parameter . 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Definition of the location parameter

Example of location parameter

Definition of the scale parameter

Example of scale parameter

Simulation Given a random variable X with CDF FX(x), there are situations that we want to obtain a set of n random numbers (i.e., a random sample of size n) from FX(.) . The advances in computer technology have made it possible to generate such random numbers using computers. The work of this nature is termed “simulation”, or more precisely “stochastic simulation”. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Pseudo-random number generation
Pseudorandom number generation (PRNG) is the technique of generating a sequence of numbers that appears to be a random sample of random variables uniformly distributed over (0,1). 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

A commonly applied approach of PRNG starts with an initial seed and the following recursive algorithm (Ross, 2002) modulo m where a and m are given positive integers, and the above equation means that is divided by m and the remainder is taken as the value of 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The quantity is then taken as an approximation to the value of a uniform (0,1) random variable.
Such algorithm will deterministically generate a sequence of values and repeat itself again and again. Consequently, the constants a and m should be chosen to satisfy the following criteria: For any initial seed, the resultant sequence has the “appearance” of being a sequence of independent uniform (0,1) random variables. For any initial seed, the number of random variables that can be generated before repetition begins is large. The values can be computed efficiently on a digital computer. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

A guideline for selection of a and m is that m be chosen to be a large prime number that can be fitted to the computer word size. For a 32-bit word computer, m = and a = result in desired properties (Ross, 2002). 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Simulating a continuous random variable
probability integral transformation 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The cumulative distribution function of a continuous random variable is a monotonic increasing function. 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Example Generate a random sample of random variable V which has a uniform density over (0, 1). Convert to using the above V-to-X transformation. 3/27/2017 Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU

Random number generation in R
R commands for stochastic simulation (for normal distribution pnorm – cumulative probability qnorm – quantile function rnorm – generating a random sample of a specific sample size dnorm – probability density function For other distributions, simply change the distribution names. For examples, (punif, qunif, runif, and dunif) for uniform distribution and (ppois, qpois, rpois, and dpois) for Poisson distribution. 3/27/2017 Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU

Generating random numbers of discrete distribution in R
Discrete uniform distribution R does not provide default functions for random number generation for the discrete uniform distribution. However, the following functions can be used for discrete uniform distribution between 1 and k. rdu<-function(n,k) sample(1:k,n,replace=T) # random number ddu<-function(x,k) ifelse(x>=1 & x<=k & round(x)==x,1/k,0) # density pdu<-function(x,k) ifelse(x<1,0,ifelse(x<=k,floor(x)/k,1)) # CDF qdu <- function(p, k) ifelse(p <= 0 | p > 1, return("undefined"), ceiling(p*k)) # quantile 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Similar, yet more flexible, functions are defined as follows
dunifdisc<-function(x, min=0, max=1) ifelse(x>=min & x<=max & round(x)==x, 1/(max-min+1), 0) >dunifdisc(23,21,40) >dunifdisc(c(0,1)) punifdisc<-function(q, min=0, max=1) ifelse(q<min, 0, ifelse(q>max, 1, floor(q-min+1)/(max-min+1))) >punifdisc(0.2) >punifdisc(5,2,19) qunifdisc<-function(p, min=0, max=1) floor(p*(max-min+1))+min >qunifdisc( ,2,19) >qunifdisc(0.2) runifdisc<-function(n, min=0, max=1) sample(min:max, n, replace=T) >runifdisc(30,2,19) >runifdisc(30) 3/27/2017 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Binomial distribution

Negative binomial distribution

Geometric distribution

Hypergeometric distribution

Poisson distribution 3/27/2017

An example of stochastic simulation
The travel time from your home (or dormitory) to NTU campus may involve a few factors: Walking to bus stop (stop for traffic lights, crowdedness on the streets, etc.) Transportation by bus Stop by 7-11 or Starbucks for breakfast (long queue) Walking to campus 3/27/2017 Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU

All Xi’s are independently distributed.
Gamma distribution with mean 30 minutes and standard deviation 10 minutes. Exponential distribution with a mean of 20 minutes. All Xi’s are independently distributed. If you leave home at 8:00 a.m. for a class session of 9:10, what is the probability of being late for the class? 3/27/2017 Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU

STATISTICS Random Variables and Probability Distributions

Similar presentations

Presentation on theme: "STATISTICS Random Variables and Probability Distributions"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

STATISTICS Random Variables and Probability Distributions

Similar presentations

Presentation on theme: "STATISTICS Random Variables and Probability Distributions"— Presentation transcript:

Similar presentations

About project

Feedback