Hypothesis Testing A procedure for determining which of two (or more) mutually exclusive statements is more likely true We classify hypothesis tests in terms of –Parametric vs. Non-parametric –Directional vs. Non-directional

Hypothesis Tests Parametric Tests - tests about specific population parameters (μ, σ 2, etc.) –Is μ 1 different from a predetermined value? –Is μ 1 different from μ 2 ? Non-Parametric Tests - tests about the shape of the population (medians?) –Is this population different from another population?

Hypothesis Tests Non-Directional Hypothesis –Is μ 1 different from μ 2 ? –Is the distribution of scores in group 1 different than those in group 2? Directional Hypotheses –Is μ 1 greater than μ 2 ? –Is μ 1 less than μ 2 ? –Is the distribution of scores in group 1 to the right (greater than) of those in group 2?

The Six-Steps of Hypothesis Testing 1. State and Check Assumptions 2. Generate Null and Alternative Hypotheses 3. Chose the Sampling Distribution of the Test Statistic 4. Set Significance Level 5. Compute the Test Statistic 6. Draw Conclusions

1. State and Check Assumptions There are three requirements for hypothesis testing to work –Assumptions about the population –Assumptions about the sample

Assumptions about the Population “Assumption of Normality” - the population is normally distributed or the sample size is sufficiently large so that the CLT comes into play and

Assumptions about Variance Depending of the type of test, some important features of variance may come into play –Is σ or σ 2 known? –“Homogeneity of Variance” - the variance of the populations being compared have equal variance

Assumptions about the Sample The sample has been obtained using independent random sampling

What if the assumptions can’t be met? “Violation” - when assumptions are not met –Violation of Normality = “Non-normal” –Violation of Homogeneity of Variance = “Heterogeneous Variance” “Robust” - a test’s ability to “deal with” violations –“a t-test is robust to violations of normality”

BUT,... No tests are robust to violations of the random sampling assumption. If you do not have a random sample, probability theory will not work and therefore inferential statistical techniques will fail.

2. Generate Null and Alternative Hypotheses These two hypotheses, designated H O and H A (or H 1 ), are mutually exclusive –mutually exclusive - the don’t overlap

Properties of H O Specifies no difference or no change from a standard or theoretical value Always specifies something about a particular population parameter Used in constructing a sampling distribution –For the subsequent quantitative work, the null hypothesis is assumed to be true

Properties of H A (or H 1 ) About the same aspect of the population as H O Usually stated in general terms Mutually exclusive - no overlap with H O Used in making a decision Can be directional or non-directional

Directional vs. Non-directional H A s Non-directional H A - usually stated as “does not equal” or “is different than” Directional H A - stated as “greater than” or “less than” –note that a non-directional hypothesis is equal to the two directional hypotheses “greater than” or “less than”

3. Chose the Sampling Distribution Depending on the type of data, assumptions, and hypotheses certain distributions of the test statistic require selection The second half of this course will be devoted to making the “best” decision about which test statistic to choose (z, t, F, etc.)

4. Set Significance Level Hypothesis testing is sometimes called “significance testing” The significance level is the basis for making our decision –“rejection region” - the value specified by the significance level of the test –“critical value” - the value of the test statistic specified by the significance level that begins the rejection region

Significance Level The probability value associated with the decision rule is called the significance level of the test Significance level is represented by the Greek letter alpha (α) The actual value of α is up to you

What is the significance level? Hypothesis testing entails determining which of two hypotheses (H O and H 1 ) is more likely correct. But “more likely” is a subjective evaluation on your part. If you were to obtain a statistic that was unlikely if H O were assume to be true, would you be willing to accept the H 1 ? How “unlikely” does it need to be for you to be convinced?

Typical Significance level A Typical significance level : α =.05 We are convinced that results are considered as significant different from the H O when they are in the most extreme 5% (a proportion of.05) of all possible outcomes specified in H O

Using α With our significance level, we determine a decision rule –Critical Value – the sufficiently extreme value of the statistic such that if our statistic is more extreme, we reject the H O –p-value – if the probability of our statistic, assuming H O is true, is less than α, we reject the H O

5. Compute the Test Statistic From the random sample obtained, the test statistic is computed using various formulae –z-statistic –binomial probability –t-statistic –F-statistic –etc.

6. Draw Conclusions Based on the test statistic just computed, we do one of two things: –Reject the null hypothesis and accept the alternative hypothesis, or –Do not reject the null hypothesis and do not accept the alternative hypothesis Note that we NEVER accept the null hypothesis, we only fail to reject it!

More on Decisions When completed, a significance test tells us the probability of obtaining our results when the null hypothesis is true p(Results|H o is True) If that probability is small, smaller than our significance level (α), it is likely that H o is not true and we reject it