2 2.3 Random Variables and Probability Distributions A variable X is said to be a random variable (rv) if for every real number a there exists a probability P(X ≦ a) that X takes on a value less than or equal to aThus P(X = x) is the probability that the random variable X takes the value x.P( x1≦ X ≦ x2 ) is the probability that the random variable X takes values between x1 and x2, both inclusive.
3 2.3 Random Variables and Probability Distributions A formula giving the probabilities for different values of the random variable X is called a probability distribution in the case of discrete random variables.Probability density function (denoted by p.d.f.) for continuous random variables. This is usually denoted by f(x)
4 2.3 Random Variables and Probability Distributions In general, for a continuous random variable, the occurrence of any exact value of X may be regarded as having a zero probability.Hence probabilities are discussed in terms of some ranges.These probabilities are obtained by integrating f(x) over the desired range.
5 2.3 Random Variables and Probability Distributions For instance, if we want Prob (a≦ X ≦b), this is given by
6 2.4 The Normal Probability Distribution and Related Distributions There are some probability distributions for which the probabilities have been tabulated and which are considered suitable descriptions for a wide variety of phenomena.These are the normal distribution and the x2, t, and F distributions
7 2.4 The Normal Probability Distribution and Related Distributions There is also a question of whether the normal distribution is an appropriate one to use to describe economic variables.However, even if the variables are not normally distributed, one can consider transformations of the variables so that the transformed variables are normally distributed.
8 2.4 The Normal Probability Distribution and Related Distributions The Normal Distribution (an example)The normal distribution is a bell-shaped distribution which is used most extensively in statistical applications in a wide variety of fields.Its probability density function is given by
9 2.4 The Normal Probability Distribution and Related Distributions and the correlation between x and x is , thenIn particular,and
10 2.4 The Normal Probability Distribution and Related Distributions If x1 , x2 , …, xn are independent normal variable with mean zero and variance 1, that is, , thenis said to have the distribution with degrees of freedom (d.f.) n, and we will write this as
11 2.4 The Normal Probability Distribution and Related Distributions The subscript n denotes the d.f.The distribution is the distribution of the sum of squares of n independent standard normal variables.
12 2.4 The Normal Probability Distribution and Related Distributions If ,then Z should be defined asThe distribution also has an “additive property,” although it is different from the property of the normal distribution and is much more restrictive.
13 2.4 The Normal Probability Distribution and Related Distributions The property is:If and and and are independent,then
14 2.4 The Normal Probability Distribution and Related Distributions t - DistributionIf and and x and y are independent, has a t-distribution with d.f. n.We write this asThe subscript n again denotes the d.f.
15 2.4 The Normal Probability Distribution and Related Distributions Thus the t-distribution is the distribution of a standard normal variable divided by the square root of an independent averaged variable ( variable divided by its degrees of freedom).The t-distribution is a symmetric probability distribution like the normal distribution but is flatter than the normal and has longer tails.As the d.f n approached infinity, the t-distribution approaches the normal distribution.
16 2.4 The Normal Probability Distribution and Related Distributions F ~DistributionIf and and and are independent,has the F-distribution with d.f. n1 and n2.We write this as
17 2.4 The Normal Probability Distribution and Related Distributions F ~DistributionThe first subscript, n1, refers to the d.f. the numerator, and the second subscript, n2 ,refers to the d.f. of the denominator.The F -distribution is thus the distribution of the ratio of two independent averaged variables.
18 2.5 Classical Statistical Inference Statistical inference is the area that describes the procedures by which we use the observed data to draw conclusions about the population from which the data came or about the process by which the data were generated.
19 2.5 Classical Statistical Inference Broadly speaking, statistical inference can be classified under two headings:Classical inferenceBayesian inference.
20 2.5 Classical Statistical Inference In Bayesian inference we combine sample information with prior information.Suppose that we draw a random sample y1 , y2 , …, yn of size n from a normal population with mean and variance (assumed known), and we want to make inferences about .
21 2.5 Classical Statistical Inference In classical inference we take the sample mean as our estimate of Its variance isThe inverse of this variance is known as the sample precision. Thus the sample precision is
22 2.5 Classical Statistical Inference In Bayesian inference we have prior information onThis is expressed in term of a probability distribution known as the prior distribution.Suppose that the prior distribution is normal with mean and variance , that is, precision
23 2.5 Classical Statistical Inference We now combine this with the sample information to obtain what is known as the posterior distribution ofThis distribution can be shown to be normal.Its mean is a weighted average of the sample.
24 2.5 Classical Statistical Inference Mean and the prior mean , weighted by the sample precision and prior precision, respectively. ThusWhereAlso, the precision (or inverse of the variance) of the posterior distribution of is , that is, the sum of sample precision and prior precision.
25 2.5 Classical Statistical Inference For instance, if the sample mean is 20 with variance 4 and the prior mean is 10 with variance 2, we haveThe posterior mean will lie between the sample mean and the prior mean.The posterior variance will be less than both the sample and prior variance.
26 2.5 Classical Statistical Inference 1. Point estimation.2. Interval estimation.3. Testing of hypotheses.
27 2.6 Properties of Estimators There are some desirable properties of estimators that are often mentioned in the book.These are:1. Unbiasedness.2. Efficiency.3. Consistency.The first two are small-sample properties. The third is a large-sample property.
28 2.6 Properties of Estimators UnbiasednessAn estimator g is said to be unbiased for if ,that is, the mean of the sampling distribution of g is equal to .What this says is that if we calculate g for each sample and repeat this process infinitely many times, the average of all these estimates will be equal to .
29 2.6 Properties of Estimators If , then g is said to be biased and we refer to as the bias.Unbiasedness is a desirable property but not at all costs.Suppose that we have two estimator g1 and g2 can assume values far away from and yet have its mean equal to ,whereas g2 always ranges close to but has its mean slightly away from .
30 2.6 Properties of Estimators Then we might prefer g2 to g1 because it has smaller variance even though it is biased.If the variance of the estimator is large, we can have some unlucky samples where our estimate is far from the true value.Thus the second property we want our estimators to have is a small variance.One criterion that is often suggested is the mean-squared error (MSE), which is defined by
31 2.6 Properties of Estimators EfficiencyThe property of efficiency is concerned with the variance of estimators.Obviously, it is a relative concept and we have to confine ourselves to a particular class.If g is an unbiased estimator and it has the minimum variance in the class of unbiased estimators, g is said to be an efficient estimators.We say that g is an MVUE (a minimum-variance unbiased estimator).
32 2.6 Properties of Estimators If we confine ourselves to linear estimators, that is,where the c’s are constants which we choose so that g is unbiased and has minimum variance, g is called a BLUE (a best linear unbiased estimator).
33 2.6 Properties of Estimators ConsistencyOften it is not possible to find estimators that have desirable small-sample properties such as properties.In such cases, it is customary to look at desirable properties in large samples.These are called asymptotic properties.
34 Three such properties often mentioned are consistency, asymptotic unbiasedness ,and asymptotic efficiency
35 2.6 Properties of Estimators Suppose that is the estimator of base on a sample of size n.Then the sequence of estimators is called a consistent sequence if for any arbitrarily small positive numbers and there is a sample size such that
36 2.6 Properties of Estimators That is, by increasing the sample size n the estimator can be made to lie arbitrarily close to the true value of with probability arbitrarily close to 1.This statement is also written asAnd more briefly we write it as
37 2.6 Properties of Estimators A sufficient condition for to be consistent is that the bias and variance should both tend to zero as the sample size increase.This condition is often useful to check in practice, but it should be noted that the condition is not necessary.An estimator can be consistent even if the bias does not tend to zeroAn example: unbiased and consistent estimate