© 2008 McGraw-Hill Higher Education The Statistical Imagination Chapter 9. Hypothesis Testing I: The Six Steps of Statistical Inference

© 2008 McGraw-Hill Higher Education A Hypothesis A hypothesis is a prediction about the relationship between two variables that asserts that differences among the measurements of an independent variable will correspond to differences among the measurements of a dependent variable

© 2008 McGraw-Hill Higher Education Using a Hypothesis to Test a Theory The hypothesis is stated before we gather data The theoretical purpose of a hypothesis test is to corroborate theory by testing preconceived ideas against facts Theory motivates or pushes us to expect certain empirical outcomes

© 2008 McGraw-Hill Higher Education Statistical Inference Statistical inference is drawing conclusions about a population on the basis of sample statistics The logic of hypothesis testing involves deciding whether to accept or reject a statement on the basis of observations of data “Accounting for sampling error” with sampling distributions is key to the process

© 2008 McGraw-Hill Higher Education The Test Effect of the Hypothesis Test The difference between what is observed in a sample and what is hypothesized is called the test effect We ask: What is the probability that the test effect is simply the result of sampling error? The sampling distribution provides a measuring stick to answer the question

© 2008 McGraw-Hill Higher Education The Statistical Purpose of a Hypothesis Test The statistical purpose of a hypothesis test is to determine whether statistical test effects computed from a sample indicate (1) real effects in the population or (2) sampling error

© 2008 McGraw-Hill Higher Education Making Empirical Predictions To prove a hypothesis we must predict two things: 1. A mathematical prediction of a parameter outcome 2. A sampling distribution, a prediction of all possible sampling outcomes factoring in sampling error With these predictions, we determine the probability that our single sample outcome differs significantly from the predicted outcome (i.e., if the effect is real)

© 2008 McGraw-Hill Higher Education Basic Logical Procedure of a Hypothesis Testing a.A question is raised b.Predictions are made on the basis of probability theory c.An event is observed and its effects are measured d.The probability of the test effect occurring is computed e.A conclusion is drawn

© 2008 McGraw-Hill Higher Education Two Sets of Tasks When Testing a Hypothesis 1.Test preparation — deciding what test to use, and organizing the data 2.Test the hypothesis following the six steps of statistical inference

© 2008 McGraw-Hill Higher Education Test Preparation State the research question: A goal that can be stated in terms of a hypothesis Draw a conceptual diagram depicting givens (population under study, sample size, variables and their levels of measurement, provided and calculated parameters and statistics) Select the statistical test

© 2008 McGraw-Hill Higher Education Step 1 of The Six Steps of Statistical Inference Step 1: State the null hypothesis (H 0 ). State the alternative hypothesis (H A ) and stipulate the direction of the test

© 2008 McGraw-Hill Higher Education The Null Hypothesis The null hypothesis, H 0, is a hypothesis stated in such a way that we will know what statistical outcomes will occur in repeated random sampling if this hypothesis is true It is a “statistical” hypothesis: it directs us to the sampling distribution, which provides sampling predictions

© 2008 McGraw-Hill Higher Education What does “null” mean? Null means none H 0 predicts sampling outcomes assuming no effect or no difference Often we can “nullify” (negate the wording) of the research question to determine the H 0

© 2008 McGraw-Hill Higher Education The Alternative Hypothesis (H A ) H A is the statement we accept if H 0 is rejected H A is often a direct statement of the research question

© 2008 McGraw-Hill Higher Education The Direction of a Hypothesis Test Test direction refers to whether we are able to predict the direction our observed sample statistic will fall Direction must be specified before we observe data

© 2008 McGraw-Hill Higher Education The Direction of a Hypothesis Test (cont.) Three possible directional statements: 1.Nondirectional (two-tailed test) 2.Positive direction (one-tailed test) 3.Negative direction (one-tailed test)

© 2008 McGraw-Hill Higher Education The Direction of a Hypothesis Test (cont.) In Step 1, the H A, we specify whether we expect the outcome in our observed sample to fall above (positive, one-tailed) or below (negative, one-tailed) the hypothesized parameter of the H 0 For a nondirectional, two-tailed test, we do not predict a direction and simply assert that the outcome is expected to differ from the hypothesized parameter

© 2008 McGraw-Hill Higher Education When to State a Positive, One-Tailed Test When the content of research question includes terms such as greater than, more, increase, faster, heavier, and gain

© 2008 McGraw-Hill Higher Education When to State a Negative, One-Tailed Test When the content of research question includes terms such as less than, fewer, decrease, slower, lighter, and loss

© 2008 McGraw-Hill Higher Education When to State a Nondirectional, Two-Tailed Test When the content of the research question includes no statements about direction, or simply asserts inequality

© 2008 McGraw-Hill Higher Education Step 2 of The Six Steps of Statistical Inference Step 2: Describe the sampling distribution and draw its curve The sampling distribution is a description of all possible sampling outcomes and a stipulation of the probability of each outcome assuming that the H 0 is true It is built around the H 0

© 2008 McGraw-Hill Higher Education Step 3 of The Six Steps of Statistical Inference Step 3: State the chosen level of significance, alpha (α), and indicate again whether the test is one-tailed or two-tailed. Specify the critical test value The level of significance, alpha, is the amount of sampling error we are willing to tolerate in coming to a conclusion Critical test values are obtained from the statistical tables in Appendix B

© 2008 McGraw-Hill Higher Education Step 4 of The Six Steps of Statistical Inference Step 4: Observe the actual sample; compute the test effects, the test statistic, and the p-value

© 2008 McGraw-Hill Higher Education Step 4 (cont.): The Test Effect The test effect is the difference between the value of the sample statistic and the parameter value predicted by the null hypothesis (H 0 in Step 1) It is a deviation score on the sampling distribution curve

© 2008 McGraw-Hill Higher Education Step 4 (cont.): The Test Statistic The test statistic is a formula for measuring the likelihood of the observed effect It transforms the effect into standard error units so that the result may be compared to critical scores of the statistical tables in Appendix B

© 2008 McGraw-Hill Higher Education Step 4 (cont.) The p-Value The p-value is a measure of the unusualness of a sample outcome when the H 0 is true. E.g., Is it unusual to roll four 7’s in a row with honest dice? Calculation: p-value = probability (p) of sampling outcomes as unusual as or more unusual than the outcome observed under the assumption that the H 0 is true An area in the tail(s) of the curve in Step 2

© 2008 McGraw-Hill Higher Education Step 5 of The Six Steps of Statistical Inference Step 5: Make the rejection decision by comparing the p-value to α If p < α, reject the H 0 and accept the H A at the 1- α level of confidence If p > α, “fail to reject” the H 0

© 2008 McGraw-Hill Higher Education Step 6 of The Six Steps: Interpretation Step 6: Interpret and apply the results, and provide best estimates in everyday terms Fit the interpretation to either a professional or public audience: use as little statistical jargon as possible Frame the interpretation around the H 0 or the H A, whichever survived the hypothesis test

© 2008 McGraw-Hill Higher Education Probability Theory in Hypothesis Testing Computing probabilities is the essential mathematical operation in hypothesis testing Hypothesis testing is based on comparing two probabilities: 1. What actually occurs in our single observed sample 2. What we expect to occur in repeated sampling

© 2008 McGraw-Hill Higher Education A Focus on p-Values: When the p-Value is Large When p > α, we fail to reject the H 0 A large p-value tells us that our observed sample outcome is not much different or “far off” from the outcome predicted by the H 0 A large p-value occurs when the test effect is small, and this suggests that the effect could easily be the result of expected sampling error

© 2008 McGraw-Hill Higher Education A Focus on p-Values: When the p-Value is Small When p < α, we reject the H 0 A small p-value tells us that assuming the H 0 is true, our sample outcome is unusual or “far off” from the outcome predicted by the H 0 A small p-value occurs when the test effect is large leading us to conclude that the test effect did not result from sampling error

© 2008 McGraw-Hill Higher Education Inverse Relationship Between Effect Size and p-Value A small test effect = a large p-value = “fail to reject” the H 0 A large test effect = a small p-value = “reject” the H 0 and accept the H A

© 2008 McGraw-Hill Higher Education The Level of Significance (α) in Hypothesis Testing The level of significance (α) is the critical probability point at which we are no longer willing to say that our sampling outcome resulted from random sampling error α is stated in Step 3 and compared in Step 5 to the p-value. This comparison is called the rejection decision

© 2008 McGraw-Hill Higher Education Critical Test Scores The critical test score (Z α ) is the statistical test score that is large enough to indicate a significant difference between the observed sample statistic and the hypothesized parameter The critical region is the area in the tail(s) of the probability curve that is beyond the critical test score of the stated level of significance

© 2008 McGraw-Hill Higher Education Critical Test Scores (cont.) Z observed is a test statistic Z α is a critical score If │Z observed │ > │ Z α │, then p < α; reject H 0 If │Z observed │ α; fail to reject H 0

© 2008 McGraw-Hill Higher Education Critical Z-scores on the Normal Curve Critical Z-scores are ones of great importance in statistical procedures and are used very frequently Some widely used critical Z-scores are 1.64, 1.96, 2.33, 2.58, 3.08, and 3.30 See if you can match these scores to the level of significance and direction of a hypothesis test

© 2008 McGraw-Hill Higher Education Choosing the Level of Significance Setting the level of significance (α) allows us to control the chances of making a wrong decision or “error” Short of double-checking against data for the entire population, we will never know for sure whether we made the correct rejection decision or made an error

© 2008 McGraw-Hill Higher Education Possible Results of a Rejection Decision Correct decision: Fail to reject a true H 0 Type I error: Rejecting a true H 0 Correct decision: reject a false H 0 Type II error: Failing to reject a false H 0

© 2008 McGraw-Hill Higher Education Managing and Controlling Rejection Decision Errors When we reject H 0, we either made a correct decision or made a Type I error; we could not have made a Type II error When we fail to reject H 0, we made either a correct decision or a Type II error; we could not have made a Type I error

© 2008 McGraw-Hill Higher Education Controlling Type I and Type II Errors Type I error is easily controlled by setting the level of significance (α), because it turns out that α = p [of making a Type I error] β = p [of making a Type II error]; controlling beta (β) is difficult β is indirectly controlled when we set α because the two are inversely related; β is also minimized by using a large sample size

© 2008 McGraw-Hill Higher Education Four Conventional Levels of Alpha (α ) α =.10: High likelihood of rejecting the H 0. Used in exploratory research, where little is known about a topic α =.05: Moderate likelihood of rejecting the H 0. Used in survey research α =.01 and α =.001: Low likelihood of rejecting the H 0. Used in biological, laboratory, and medical research, especially when a Type I error is life- threatening

© 2008 McGraw-Hill Higher Education The Level of Confidence (LOC) for a Hypothesis Test The LOC is the confidence we have that we did not make a Type I error LOC = 1 - level of significance = 1 - α E.g., the.05 level of significance corresponds to a 95% LOC The only time we have 100% confidence in a conclusion is when every subject in a population is observed

© 2008 McGraw-Hill Higher Education Selecting Which Statistical Test to Use Ask: How many variables are we observing for this test? What are the levels of measurement of the variables? Are we dealing with one representative sample from a single population or more? What is the sample size? Are there peculiar circumstances to consider?

© 2008 McGraw-Hill Higher Education When to Use a Large Single-Sample Means Test 1. One variable 2. Interval/ratio level of measurement 3. One representative sample from one population 4. n > 121 cases Sampling distribution will be the normal curve (See Chapter 7)