2 8.1 Hypothesis Testing Logic A statistical method that uses sample data to evaluate the validity of a hypothesis about a population parameter
3 Logic of Hypothesis Test State hypothesis about a …Predict the expected characteristics of the sample based on the …Obtain a random sample from the …Compare the obtained sample data with the prediction made from the hypothesisIf consistent, hypothesis is …If discrepant, hypothesis is …
4 Figure 8.1 Basic Experimental Design FIGURE 8.1 The basic experimental situation for hypothesis testing. It is assumed that the parameter μ is known for the population before treatment. The purpose of the experiment is to determine whether the treatment has an effect on the population mean.
5 Figure 8.2 Unknown Population in Basic Experimental Design FIGURE 8.2 From the point of view of the hypothesis test, the entire population receives the treatment and then a sample is selected from the treated population. In the actual research study, a sample is selected from the original population and the treatment is administered to the sample. From either perspective, the result is a treated sample that represents the treated population.
6 Four Steps in Hypothesis Testing Step 1: Step 2: Step 3: Step 4:
7 Step 1: State Hypotheses Null hypothesis (H0)states that, in the general population, there is no change, no difference, or is no relationshipAlternative hypothesis (H1)states that there is a change, a difference, or there is a relationship in the general population
8 Step 2: Set the Decision Criterion Distribution of sample outcomes is dividedThose likely if H0 is trueThose “very unlikely” if H0 is trueAlpha level, or significance level, is a probability value used to define “very unlikely” outcomesCritical region(s) consist of the extreme sample outcomes that are “very unlikely”Boundaries of critical region(s) are determined by the probability set by the alpha level
9 Figure 8.3 Note “Unlikely” Parts of Distribution of Sample Means FIGURE 8.3 The set of potential samples is divided into those that are likely to be obtained and those that are very unlikely to be obtained if the null hypothesis is true.
10 Figure 8.4 Critical region(s) for α = .05 FIGURE 8.4 The critical region (very unlikely outcomes) for alpha = .05
11 Step 3: Compute Sample Statistics Compute a sample statistic (z-score) to show the exact position of the sampleIn words, z is the difference between the observed sample mean and the hypothesized population mean divided by the standard error of the mean
12 Step 4: Make a decisionIf sample statistic (z) is located in the critical region, the null hypothesis is …If the sample statistic (z) is not located in the critical region, the researcher fails to …
13 8.2 Uncertainty and Errors in Hypothesis Testing Hypothesis testing is an inferential processUses limited information from a sample to make a statistical decision, and then from it …Sample data used to make the statistical decision allows us to make an inference and …Errors are possible
14 Type I ErrorsResearcher rejects a null hypothesis that is actually trueResearcher concludes that a treatment has an effect when it has noneAlpha level is …
15 Type II ErrorsResearcher fails to reject a null hypothesis that is really falseResearcher has failed to detect a real treatment effectType II error probability is not easily identified
16 Table 8.1 Type I error (α) Type II error (β) Actual Situation No Effect =H0 TrueEffect Exists =H0 FalseResearcher’s DecisionReject H0Type I error (α)Decision correctFail to reject H0Type II error (β)
17 Figure 8.5 Location of Critical Region Boundaries FIGURE 8.5 The locations of the critical region boundaries for three different levels of significance: α = .05, α = .01, and α = .001.
18 Figure 8.6 Critical Region for Standard Test FIGURE 8.6 Sample means that fall in the critical region (shaded areas) have a probability less than alpha (p < α). In this case, H0 should be rejected. Sample means that do fall in the critical region have a probability greater than alpha (p > α).
19 8.3 Assumptions for Hypothesis Tests with z-Scores Random …Independent …Value of σ is not changed …Normally distributed …
20 Factors that Influence the Outcome of a Hypothesis Test Size of difference between sample mean and original population meanVariability of the scoresNumber of scores in the sample
21 8.5 Hypothesis Testing Concerns: Measuring Effect Size Although commonly used, some researchers are concerned about hypothesis testingFocus of test is data, not hypothesisSignificant effects are not always substantialEffect size measures the absolute magnitude of a treatment effect, independent of sample sizeCohen’s d measures effect size simply and directly in a standardized way
22 Cohen’s d : Measure of Effect Size Magnitude of dEvaluation of Effect Sized = 0.2Small effectd = 0.5Medium effectd = 0.8Large effect
23 Figure 8.8 When is a 15-point Difference a “Large” Effect? FIGURE 8.8 The appearance of a 15-point treatment effect in two different situations. In part (a), the standard deviation is σ = 100 and the 15-point effect is relatively small. In part (b), the standard deviation is σ = 15 and the 15-point effect is relatively large. Cohen’s d uses the standard deviation to help measure effect size.