Uncertainty and confidence Although the sample mean,, is a unique number for any particular sample, if you pick a different sample you will probably get a different sample mean. In fact, you could get many different values for the sample mean, and virtually none of them would actually equal the true population mean, . x

But the sample distribution is narrower than the population distribution, by a factor of √n. Thus, the estimates gained from our samples are always relatively close to the population parameter µ. n Sample means, n subjects  Population, x individual subjects If the population is normally distributed N(µ,σ), so will the sampling distribution of Xbar be N(µ,σ/√n). But recall the Central Limit Theorem, even in the cases when the population in not normally distributed, for large n, we’ll have ~normality for Xbar.

So we’ll use this information to make inferences; i.e., draw conclusions about populations from data in our samples We'll consider two types: –Confidence interval estimation –Tests of significance In both of these cases, we'll consider our data as either being a random sample from a population or as data from a randomized experiment Start with estimation… there are two situations we'll consider –estimating the mean  of a population of measurements –estimating the proportion p of Ss in a population of Ss and Fs

In either case, we'll construct a confidence interval of the form estimate +/- M.O.E., where M.O.E. = margin of error of the estimator. The MOE gives information on how good the estimate is through the variation in the estimator (its standard error) and through the level of confidence in the confidence interval (through a tabulated value). The standard error of an estimator is its estimated standard deviation (treating the estimator as a statistic with a sampling distribution…) Best estimator of  is and we know from the previous chapter that is approximately Best estimator of p is phat and we know from the last chapter that phat is approx.

Red dot: mean value of individual sample 95% of all sample means will be within roughly 2 standard deviations (2*  /√n) of the population parameter  This implies that the population parameter  must be within roughly 2 standard deviations from the sample average, in 95% of all samples. This reasoning is the essence of statistical estimation.

So using this fact, we construct a 95% confidence interval for  as Xbar +/- 1.96(  /sqrt(n)) 1.96(  /sqrt(n)) is the M.O.E. NOTE: MOE = (number from a table)*(Std. Error) A most important figure in this context is the one given on the previous slide (Figure 6.3; eBook 6.1, 3/7)… Be sure you can give a full explanation of what this figure is showing you about confidence intervals In general, we can construct a confidence interval for  with any level of confidence C as Xbar +/- z * (  /sqrt(n)) where z * is the appropriate number from Table A giving confidence C; or you may use the last row of Table D.

Xbar +/- 1.96(  /sqrt(n)) is the 95% CI for  … if the MOE is too large for our purposes, we may want to increase n… in fact, we can set the MOE equal to any number we'd like and solve for n… do the algebra to get n = (1.96  /MOE) 2. Substitute z * for 1.96 to get n for other levels of confidence… Read the cautions on page (eBook 6.1, 7/7). These are very important and we must pay careful attention to them…

A significance test or hypothesis test is a procedure for comparing our data with a hypothesis whose truth we want to assess. The hypothesis is usually a statement about the parameters of a population from which we’ve taken our data. The results of the test are given in a probability statement that measures how well our data and the hypothesis agree with each other…Go over Examples (eBook, section 6.2, 1/8 through 5/8) in detail to see the logic in hypothesis testing…

The general format of a hypothesis test is: STATE THE HYPOTHESES GIVE THE TEST STATISTIC YOU'LL USE IN THE TEST CALCULATE ITS VALUE FOR OUR DATA AND ASSESS HOW LIKELY IT IS TO HAVE OCCURRED, ASSUMING THE HYPOTHESIS IS TRUE STATE THE CONCLUSION IN THE CONTEXT OF THE PROBLEM YOU’RE WORKING… The hypothesis you assume to be true, the one you are comparing against your data is called the null hypothesis and is usually labeled by the symbol H 0. The test is designed to assess the strength of evidence against the null hypothesis. Many times the null hypothesis is a statement of “no effect” or of “no difference”… more later on this

The null hypothesis in Ex. 6.8 is H 0 :There is no difference in the true mean debts and the alternative hypothesis is H a :The true mean debts are not the same NOTE: these hypotheses always refer to a population parameter or a model, not to a particular outcome (“true mean”). The above alternative is two-sided since we don’t really know whether one mean is larger or smaller than the other... The test of hypothesis is based on a test statistic that is a good estimator of the parameter in the null hypothesis; e.g., Xbar estimates , phat estimates p, the difference in sample means estimates the difference in true means, etc. If the test statistic is “far away” from the value of the parameter specified in the null hypothesis, this gives evidence against H 0 – the alternative hypothesis determines which direction we should be looking for the evidence, larger or smaller (or both if two-sided). “far away” is in terms of the s.d. of the estimator...

Whether the test statistic is “likely” or “unlikely” to occur assuming the null hypothesis is true, is determined by computing the p-value of the test; i.e., the probability, assuming the null hypothesis is true, that the TS would take on a value as extreme or more extreme than the one actually observed. See Figures 6.7 and 6.8 on p. 374 (eBook section 6.2, 1/8) Figure 6.7Figure 6.8

Now compare this P-value with the significance level, called , the probability that we regard as decisive, usually picked as.05, but could be different. If our P- value is <= , then we reject the null hypothesis and say that our data are statistically significant at level . Then we usually summarize our conclusion in a sentence or two that tells what our test has found… The box on p. 383 (6.2,6/8) summarizes the z-test for .

Go over Example 6.15, p (eBook, 6.2, 6/8), to see how a p-value is computed when the alternative hypothesis is two-sided. NOTE: Double the area you find in one tail… see Figure 6.11 below. z = 1.78

There is an important relationship between confidence intervals and two-sided hypothesis tests: A level  two-sided significance test rejects H 0 :  =  0 exactly when the hypothesized value  0 falls outside a level (1-  confidence interval for . In other words, if we can’t say that the hypothesized value of mu,    is in our confidence interval, then we would reject a two-sided hypothesis about   . That is, values not in our confidence interval would seem to be not compatible with our data…i.e., they would be rejected by our data…

Ex: Your sample gives a 99% confidence interval of With 99% confidence, could samples be from populations with µ = 0.86? µ = 0.85? 99% C.I. Logic of confidence interval test Cannot reject H 0 :  = 0.85 Reject H 0 :  = 0.86 A confidence interval gives a black and white answer: Reject or don't reject H 0. But it also estimates a range of likely values for the true population mean µ. A P-value quantifies how strong the evidence is against the H 0. But if you reject H 0, it doesn’t provide any information about the true population mean µ.

Use and Abuse of Hypothesis Tests… Null hypothesis asserts “no effect”, “no difference” while the alternative is a research hypothesis asserting that the effect is present or there is a difference… The P-value gives a way of measuring the amount of evidence provided by the data against H 0. “There is no sharp border between “significant” and “not significant”, only increasingly strong evidence as the P-value decreases” See R.A. Fisher’s opinion on choosing the level of significance for a test at the top of page 396… Statistical significance is not the same as practical significance. Don’t forget to explore your data thoroughly before doing hypothesis testing…

Don’t ignore lack of significance – believing an effect is present and not finding it could be important. Badly designed surveys and experiments cannot be improved by hypothesis testing… “The reasoning behind statistical significance works well if you decide what effect you’re seeking, design an experiment or sample to search for it, and use a test of significance to weigh the evidence you get”. But be careful about “searching for significance”… see page 399, example 6.28… many tests run at once on the same data will likely turn up some significant results by chance even if all the null hypotheses are true!

HW: Read section 6.1 Do # , , 6.17, 6.18, , 6.35 HW: Read section 6.2 – take some time to understand the logic of hypothesis testing pay particular attention to the p-value Do # , 6.53, , ,