A significance test or hypothesis test is a procedure for comparing our data with a hypothesis whose truth we want to assess. The hypothesis is usually a statement about the parameters of a population from which we’ve taken our data. The results of the test are given in a probability statement that measures how well our data and the hypothesis agree with each other…Go over Examples in detail to see the logic in hypothesis testing (see the link on my website "Work on Examples in sections 6.1 and 6.2 (.doc)" )

The general format of a hypothesis test is: –STATE THE HYPOTHESES –GIVE THE TEST STATISTIC YOU WILL USE IN THE TEST –CALCULATE ITS VALUE FOR OUR DATA AND ASSESS HOW LIKELY IT IS TO HAVE OCCURRED, ASSUMING THE NULL HYPOTHESIS IS TRUE –STATE THE CONCLUSION IN THE CONTEXT OF THE PROBLEM YOU’RE WORKING… The hypothesis you assume to be true, the one you are comparing against your data is called the null hypothesis and is usually labeled by the symbol H 0. The test is designed to assess the strength of evidence against the null hypothesis. Many times the null hypothesis is a statement of “no effect” or of “no difference”… more later on this

The null hypothesis in Ex. 6.8 is H 0 :There is no difference in the true mean debts and the alternative hypothesis is H a :The true mean debts are not the same NOTE: these hypotheses always refer to a population parameter or a model, not to a particular outcome (“true mean”). The above alternative is two-sided since we don’t really know whether one mean is larger or smaller than the other... The test of hypothesis is based on a test statistic that is a good estimator of the parameter in the null hypothesis; e.g., Xbar estimates , phat estimates p, the difference in sample means estimates the difference in true means, etc. If the test statistic is “far away” from the value of the parameter specified in the null hypothesis, this gives evidence against H 0 – the alternative hypothesis determines which direction we should be looking for the evidence, larger or smaller (or both if two-sided). “far away” is in terms of the s.d. of the estimator...

Whether the test statistic is “likely” or “unlikely” to occur assuming the null hypothesis is true, is determined by computing the p-value of the test; i.e., the probability, assuming the null hypothesis is true, that the TS would take on a value as extreme or more extreme than the one actually observed. See Figures 6.7 and 6.8 on p. 374 (eBook section 6.2, 1/8) Figure 6.7 Figure 6.8

Now compare this P-value with the significance level, called , the probability that we regard as decisive, usually picked as.05, but could be different. If our P- value is <= , then we reject the null hypothesis and say that our data are statistically significant at level . Then we usually summarize our conclusion in a sentence or two that tells what our test has found… The box on p. 383 (6.2,6/8) summarizes the z-test for .

Go over Example 6.15, p (eBook, 6.2, 6/8), to see how a p-value is computed when the alternative hypothesis is two-sided. NOTE: Double the area you find in one tail… see Figure 6.11 below. z = 1.78

HW: Read section 6.2 – take some time to understand the logic of hypothesis testing pay particular attention to the p-value Do # , 6.53, , ,

There is an important relationship between confidence intervals and two-sided hypothesis tests: A level  two-sided significance test rejects H 0 :  =  0 exactly when the hypothesized value  0 falls outside a level (1-  confidence interval for . In other words, if we can’t say that the hypothesized value of mu,    is in our confidence interval, then we would reject a two-sided hypothesis about   . That is, values not in our confidence interval would seem to be not compatible with our data…i.e., they would be rejected by our data…

Ex: Your sample gives a 99% confidence interval of With 99% confidence, could samples be from populations with µ = 0.86? µ = 0.85? 99% C.I. Logic of confidence interval test Cannot reject H 0 :  = 0.85 Reject H 0 :  = 0.86 A confidence interval gives a black and white answer: Reject or don't reject H 0. But it also estimates a range of likely values for the true population mean µ. A P-value quantifies how strong the evidence is against the H 0. But if you reject H 0, it doesn’t provide any information about the true population mean µ.

Use and Abuse of Hypothesis Tests… –Null hypothesis asserts “no effect”, “no difference” while the alternative is a research hypothesis asserting that the effect is present or there is a difference… –The P-value gives a way of measuring the amount of evidence provided by the data against H 0. “There is no sharp border between “significant” and “not significant”, only increasingly strong evidence as the P-value decreases” See R.A. Fisher’s opinion on choosing the level of significance for a test at the top of page 396 (eBook 6.3, 2/6) … –Statistical significance is not the same as practical significance. Don’t forget to explore your data thoroughly before doing hypothesis testing…

Don’t ignore lack of significance – believing an effect is present and not finding it could be important. Badly designed surveys and experiments cannot be improved by hypothesis testing… “The reasoning behind statistical significance works well if you decide what effect you’re seeking, design an experiment or sample to search for it, and use a test of significance to weigh the evidence you get”. But be careful about “searching for significance”… see page 399, example 6.28… many tests run at once on the same data will likely turn up some significant results by chance even if all the null hypotheses are true!