Some standard univariate probability distributions

Slides:



Advertisements
Similar presentations
Order Statistics The order statistics of a set of random variables X1, X2,…, Xn are the same random variables arranged in increasing order. Denote by X(1)
Advertisements

Exponential Distribution. = mean interval between consequent events = rate = mean number of counts in the unit interval > 0 X = distance between events.
Lecture (11,12) Parameter Estimation of PDF and Fitting a Distribution Function.
Log-linear and logistic models Generalised linear model ANOVA revisited Log-linear model: Poisson distribution logistic model: Binomial distribution Deviances.
Discrete Uniform Distribution
Statistics review of basic probability and statistics.
Random Variable A random variable X is a function that assign a real number, X(ζ), to each outcome ζ in the sample space of a random experiment. Domain.
Discrete Probability Distributions
Use of moment generating functions. Definition Let X denote a random variable with probability density function f(x) if continuous (probability mass function.
Chapter 1 Probability Theory (i) : One Random Variable
Engineering Probability and Statistics - SE-205 -Chap 4 By S. O. Duffuaa.
Descriptive statistics Experiment  Data  Sample Statistics Sample mean Sample variance Normalize sample variance by N-1 Standard deviation goes as square-root.
Probability Densities
Maximum likelihood Conditional distribution and likelihood Maximum likelihood estimations Information in the data and likelihood Observed and Fisher’s.
QA-2 FRM-GARP Sep-2001 Zvi Wiener Quantitative Analysis 2.
FRM Zvi Wiener Following P. Jorion, Financial Risk Manager Handbook Financial Risk Management.
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
Statistics.
Log-linear and logistic models Generalised linear model ANOVA revisited Log-linear model: Poisson distribution logistic model: Binomial distribution Deviances.
Log-linear and logistic models
Some standard univariate probability distributions
A random variable that has the following pmf is said to be a binomial random variable with parameters n, p The Binomial random variable.
The moment generating function of random variable X is given by Moment generating function.
3-1 Introduction Experiment Random Random experiment.
Continuous Random Variables and Probability Distributions
Class notes for ISE 201 San Jose State University
Maximum likelihood (ML)
Some standard univariate probability distributions
Discrete Random Variables and Probability Distributions
Normal and Sampling Distributions A normal distribution is uniquely determined by its mean, , and variance,  2 The random variable Z = (X-  /  is.
Chapter 21 Random Variables Discrete: Bernoulli, Binomial, Geometric, Poisson Continuous: Uniform, Exponential, Gamma, Normal Expectation & Variance, Joint.
Distribution Function properties. Density Function – We define the derivative of the distribution function F X (x) as the probability density function.
Exponential Distribution & Poisson Process
Moment Generating Functions 1/33. Contents Review of Continuous Distribution Functions 2/33.
Standard Statistical Distributions Most elementary statistical books provide a survey of commonly used statistical distributions. The reason we study these.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Review of Exam 2 Sections 4.6 – 5.6 Jiaping Wang Department of Mathematical Science 04/01/2013, Monday.
DATA ANALYSIS Module Code: CA660 Lecture Block 3.
Chapter 3 Basic Concepts in Statistics and Probability
Random Variables & Probability Distributions Outcomes of experiments are, in part, random E.g. Let X 7 be the gender of the 7 th randomly selected student.
Statistics for Engineer Week II and Week III: Random Variables and Probability Distribution.
Moment Generating Functions
Some standard univariate probability distributions Characteristic function, moment generating function, cumulant generating functions Discrete distribution.
Copyright © 2010, 2007, 2004 Pearson Education, Inc. Review and Preview This chapter combines the methods of descriptive statistics presented in.
Random Sampling, Point Estimation and Maximum Likelihood.
Functions of Random Variables. Methods for determining the distribution of functions of Random Variables 1.Distribution function method 2.Moment generating.
Continuous Distributions The Uniform distribution from a to b.
Problem: 1) Show that is a set of sufficient statistics 2) Being location and scale parameters, take as (improper) prior and show that inferences on ……
Stats Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes.
Lecture 3: Statistics Review I Date: 9/3/02  Distributions  Likelihood  Hypothesis tests.
Expectation. Let X denote a discrete random variable with probability function p(x) (probability density function f(x) if X is continuous) then the expected.
Random Variable The outcome of an experiment need not be a number, for example, the outcome when a coin is tossed can be 'heads' or 'tails'. However, we.
Sampling and estimation Petter Mostad
Chapter 4 Continuous Random Variables and Probability Distributions  Probability Density Functions.2 - Cumulative Distribution Functions and E Expected.
Chapter 5 Sampling Distributions. Introduction Distribution of a Sample Statistic: The probability distribution of a sample statistic obtained from a.
Statistics Sampling Distributions and Point Estimation of Parameters Contents, figures, and exercises come from the textbook: Applied Statistics and Probability.
Chapter 31Introduction to Statistical Quality Control, 7th Edition by Douglas C. Montgomery. Copyright (c) 2012 John Wiley & Sons, Inc.
Chapter 5 Probability Distributions 5-1 Overview 5-2 Random Variables 5-3 Binomial Probability Distributions 5-4 Mean, Variance and Standard Deviation.
Central Limit Theorem Let X 1, X 2, …, X n be n independent, identically distributed random variables with mean  and standard deviation . For large n:
Chap 5-1 Chapter 5 Discrete Random Variables and Probability Distributions Statistics for Business and Economics 6 th Edition.
Introduction to Probability - III John Rundle Econophysics PHYS 250
Probability Distributions: a review
Engineering Probability and Statistics - SE-205 -Chap 4
The Exponential and Gamma Distributions
Chapter 7: Sampling Distributions
Multinomial Distribution
Moment Generating Functions
Further Topics on Random Variables: 1
Continuous Distributions
1/2555 สมศักดิ์ ศิวดำรงพงศ์
Presentation transcript:

Some standard univariate probability distributions Characteristic function, moment generating function, cumulant generating functions Discrete distribution Continuous distributions Some distributions associated with normal References

Characteristic function, moment generating function, cumulant generating functions Characteristic function is defined as expectation of the function - e(itx) Moment generating function is defined as (expectation of e(tx)): Moments can be calculated in the following way. Obtain derivative of M(t) and take the value of it at t=0 Cumulant generting function is defined as logarithm of the characteristic function

Discrete distributions: Binomial Let us assume that we carry out experiment and the result of the experiment can be “success” or “failure”. The probability of “success” in one experiment is p. Then probability of failure is q=1-p. We carry out experiments n times. Distribution of k successes is binomial: Characteristic function: Moment generating function: Find the first and the second moments.

Discrete distributions: Poisson When number of the trials (n) is large and the probability of successes (p) is small and np is finite and tends to  then the binomial distribution converges to Poisson distribution: Poisson distribution is used to describe the distribution of an event that occurs rarely (rare events) in a short time period. It is used in counting statistics to describe the number of registered photons. Characteristic function is: What is the first moment?

Discrete distributions: Negative Binomial Consider an experiment: Probability of “success” is p and probability of failure is q=1-p. We carry out experiment until k-th success. We want to find the probability of j failures. (It is called sequential sampling. Sampling is carried out until stopping rule - k successes - is satisfied). If we have j failures then it means that the number of trials is k+j. Last trial was success. Then the probability that we will have j failures is: It is called negative binomial because coefficients have the from of negative binomial series: p-k=(1-q)-k Characteristic function is: What is the moment generating function? What is the first moment?

Continuous distributions: uniform The simplest form of the continuous distribution is the uniform with density: Cumulative distribution function is: Moments and other properties are calculated easily.

Continuous distributions: exponential Density of exponential distribution has the form: This distribution has two origins. Maximum entropy. If we know that random variable is non-negative and we know its first moment – 1/ then maximum entropy distribution has the exponential form. From Poisson type random processes. If probability distribution of j(t) events occurring during time interval [0;t) is a Poisson with mean value  t then probability of time elapsing till the first event occurs has the exponential distribution. Let Tr denotes time elapsed until r-th event Putting r=1 we get e(- t). Taking into account that P(T1>t) = 1-F1(t) and getting its derivative wrt t we arrive to exponential distribution This distribution together with Poisson is widely used in reliability studies, life testing etc.

Continuous distributions: Gamma Gamma distribution can be considered as generalisation of the exponential distribution. It has the form: It is probability of time t elapsing before r events happens Characteristic function of this distribution is: If there are r independently and identically exponentially distributed random variables then the distribution of their sum is Gamma.

Continuous distributions: Normal Perhaps the most popular and widely used continuous distribution is the normal distribution. Main reason for this is that that usually random variable is the sum of the many random variables. According to central limit theorem under some conditions (for example: random variables are independent. first and second and third moments exist and finite then distribution of sum of random variables converges to normal distribution) Density of the normal distribution has the form There many tables for normal distribution. Its characteristic function is:

Central limit theorem Let assume that we have n independent random variables {Xi}, i= 1,..,n. If first, second and third moments (this condition can be relaxed) are finite then the sum of these random variables for sufficiently large n will be approximately normally distributed. Because of this theorem in many cases assumption of normal distribution is sufficiently good and tests based on this assumption give satisfactory results. Sometimes statements are made that:

Exponential family Exponential family of distributions has the form Many distributions are special case of this family. Natural exponential family of distributions is the subclass of this family: Where A() is natural parameter. If we use the fact that distribution should be normalised then characteristic function of the natural exponential family with natural parameter A() =  can be derived to be: Try to derive it. Hint: use the normalisation factor. Find D and then use expression of characteristic function and D. This distribution is used for fitting generlised linear models.

Exponential family: Examples Many well known distributions belong to this family (All distributions mentioned in this lecture are from exponential family). Binomial Poisson Gamma Normal

Continuous distributions: 2 Normal variables are called standardized if their mean is 0 and variance is 1. Sum of n standardized normal random variables is 2 with n degrees of freedom. Density function is: If there are p linear restraints on the random variables then degree of freedom becomes n-p. Characteristic function for this distribution is: 2 is used widely in statistics for such tests as goodness of fit of model to experiment.

Continuous distributions: t and F-distributions Two more distribution is closely related with normal distribution. We will give them when we will discuss sample and sampling distributions. One of them is Student’s t-distribution. It is used to test if mean value of the sample is significantly different from 0. Another and similar application is for tests of differences of means of two different samples are different. Fisher’s F-distribution is distribution ratio of the variances of two different samples. It is used to test if their variances are different. On of the important application is in ANOVA.

Reference Johnson, N.L. & Kotz, S. (1969, 1970, 1972) Distributions in Statistics, I: Discrete distributions; II, III: Continuous univariate distributions, IV: Continuous multivariate distributions. Houghton Mufflin, New York. Mardia, K.V. & Jupp, P.E. (2000) Directional Statistics, John Wiley & Sons. Jaynes, E (2003) The Probability theory: Logic of Science