Download presentation

Presentation is loading. Please wait.

Published byAbraham Dean Modified over 6 years ago

1
Significance Testing Chapter 13 Victor Katch Kinesiology

2
Critical Region The critical region (or rejection region) is the set of all values of the test statistic that cause us to reject the null hypothesis. For example, see the red-shaded region in previous Figure. Victor Katch Kinesiology

3
Significance Level The significance level (denoted by ) is the probability that the test statistic will fall in the critical region when the null hypothesis is actually true. Common choices for are 0.05, 0.01, and 0.10. Victor Katch Kinesiology

4
Critical Value A critical value is any value separating the critical region (where we reject the H0) from the values of the test statistic that does not lead to rejection of the null hypothesis, the sampling distribution that applies, and the significance level . For example, the critical value of z = corresponds to a significance level of = 0.05. Victor Katch Kinesiology

5
**Two-tailed, Right-tailed, Left-tailed Tests**

The tails in a distribution are the extreme regions bounded by critical values. page 373 of text Victor Katch Kinesiology

6
** is divided equally between the two tails of the critical**

Two-tailed Test H0: = H1: is divided equally between the two tails of the critical region Means less than or greater than Victor Katch Kinesiology

7
Right-tailed Test H0: = H1: > Points Right Victor Katch Kinesiology

8
Left-tailed Test H0: = H1: < Points Left Victor Katch Kinesiology

9
P-Value The P-value (or p-value or probability value) is the probability of getting a value of the test statistic that is at least as extreme as the one representing the sample data, assuming that the null hypothesis is true. The null hypothesis is rejected if the P-value is very small, such as 0.05 or less. Victor Katch Kinesiology

10
**Conclusions in Hypothesis Testing**

We always test the null hypothesis. 1. Reject the H0 2. Fail to reject the H0 page 374 of text. Examples at bottom of page and top of page 375 Victor Katch Kinesiology

11
**Accept versus Fail to Reject**

Some texts use “accept the null hypothesis.” We are not proving the null hypothesis. The sample evidence is not strong enough to warrant rejection (such as not enough evidence to convict a suspect). page 374 of text The term ‘accept’ is somewhat misleading, implying incorrectly that the null has been proven. The phrase ‘fail to reject’ represents the result more correctly. Victor Katch Kinesiology

12
**Decision Criterion Traditional method:**

Reject H0 if the test statistic falls within the critical region. Fail to reject H0 if the test statistic does not fall within the critical region. Victor Katch Kinesiology

13
**Decision Criterion P-value method:**

Reject H0 if P-value (where is the significance level, such as 0.05). Fail to reject H0 if P-value > . Victor Katch Kinesiology

14
**Decision Criterion Another option:**

Instead of using a significance level such as 0.05, simply identify the P-value and leave the decision to the reader. Victor Katch Kinesiology

15
**Example: Finding P-values**

Victor Katch Kinesiology

16
**Wording of Final Conclusion**

Victor Katch Kinesiology

17
**Hypothesis testing about:**

a population mean or mean difference (paired data) the difference between means of two populations the difference between two population proportions Three Cautions: 1. Inference is only valid if the sample is representative of the population for the question of interest. 2. Hypotheses and conclusions apply to the larger population(s) represented by the sample(s). 3. If the distribution of a quantitative variable is highly skewed, consider analyzing the median rather than the mean – called nonparametric methods. Victor Katch Kinesiology

18
**Significance Testing Steps in Any Hypothesis Test**

Determine the null and alternative hypotheses. Verify necessary data conditions, and if met, summarize the data into an appropriate test statistic. Assuming the null hypothesis is true, find the p-value. Decide whether or not the result is statistically significant based on the p-value. Report the conclusion in the context of the situation. Victor Katch Kinesiology

19
**Testing Hypotheses About One Mean or Paired Data**

Step 1: Determine null and alternative hypotheses 1. H0: m = m0 versus Ha: m m0 (two-sided) 2. H0: m m0 versus Ha: m < m0 (one-sided) 3. H0: m m0 versus Ha: m > m0 (one-sided) Often H0 for a one-sided test is written as H0: m = m0. Remember a p-value is computed assuming H0 is true, and m0 is the value used for that computation. Victor Katch Kinesiology

20
**Step 2: Verify Necessary Data Condition**

Situation 1: Population of measurements of interest is approximately normal, and a random sample of any size is measured. In practice, use method if shape is not notably skewed or no extreme outliers. Situation 2: Population of measurements of interest is not approximately normal, but a large random sample (n 30) is measured. If extreme outliers or extreme skewness, better to have a larger sample. Victor Katch Kinesiology

21
**Continuing Step 2: The Test Statistic**

The t-statistic is a standardized score for measuring the difference between the sample mean and the null hypothesis value of the population mean: This t-statistic has (approx) a t-distribution with df = n - 1. Victor Katch Kinesiology

22
**Step 3: Assuming H0 true, Find the p-value**

For H1 less than, the p-value is the area below t, even if t is positive. For H1 greater than, the p-value is the area above t, even if t is negative. For H1 two-sided, p-value is 2 area above |t|. Victor Katch Kinesiology

23
Steps 4 and 5: Decide Whether or Not the Result is Statistically Significant based on the p-value and Report the Conclusion in the Context of the Situation These two steps remain the same for all of the hypothesis tests. Choose a level of significance a, and reject H0 if the p-value is less than (or equal to) a. Otherwise, conclude that there is not enough evidence to support the alternative hypothesis. Victor Katch Kinesiology

24
**Example Normal Body Temperature**

What is normal body temperature? Is it actually less than 98.6 degrees Fahrenheit (on average)? Step 1: State the null and alternative hypotheses H0: m = 98.6 Ha: m < 98.6 where m = mean body temperature in human population. Victor Katch Kinesiology

25
**Example Normal Body Temp (cont)**

Data: random sample of n = 18 normal body temps Step 2: Verify data conditions … x no outliers nor strong skewness. Sample mean of is close to sample median of 98.2. Victor Katch Kinesiology

26
**Example Normal Body Temp (cont)**

Step 2: … Summarizing data with a test statistic Test of mu = vs mu < Variable N Mean StDev SE Mean T P Temperature Key elements: Sample statistic: = (under “Mean”) Standard error: (under “SE Mean”) (under “T”) Victor Katch Kinesiology

27
**Example Normal Body Temp (cont)**

Step 3: Find the p-value From output: p-value = 0.015 From Table A.3: p-value is between and Area to left of t = equals area to right of t = The value t = 2.38 is between column headings 2.33 and 2.58 in table, and for df =17, the one-sided p-values are and Victor Katch Kinesiology

28
**Example Normal Body Temp (cont)**

Step 4: Decide whether or not the result is statistically significant based on the p-value Using a = 0.05 as the level of significance criterion, the results are statistically significant because 0.015, the p-value of the test, is less than In other words, we can reject the null hypothesis. Step 5: Report the Conclusion We can conclude, based on these data, that the mean temperature in the human population is actually less than 98.6 degrees. Victor Katch Kinesiology

29
**Paired Data and the Paired t-Test**

Data: two variables for n individuals or pairs; use the difference d = x1 – x2. Parameter: md = population mean of differences Sample estimate: = sample mean of the differences Standard deviation and standard error: sd = standard deviation of the sample of differences; Often of interest: Is the mean difference in the population different from 0? Victor Katch Kinesiology

30
**Steps for a Paired t-Test**

Step 1: Determine null and alternative hypotheses H0: md = 0 versus Ha: md 0 or Ha: md < 0 or Ha: md > 0 Watch how differences are defined for selecting the Ha. Step 2: Verify data conditions and compute test statistic Conditions apply to the differences. The t-test statistic is: Steps 3, 4 and 5: Similar to t-test for a single mean. The df = n – 1, where n is the number of differences. Victor Katch Kinesiology

31
**Example Effect of Alcohol**

Study: n = 10 pilots perform simulation first under sober conditions and then after drinking alcohol. Response: Amount of useful performance time. (longer time is better) Question: Does useful performance time decrease with alcohol use? Step 1: State the null and alternative hypotheses H0: md = 0 versus Ha: md > 0 where md = population mean difference between alcohol and no alcohol measurements if all pilots took these tests. Victor Katch Kinesiology

32
**Example Effect of Alcohol (cont)**

Data: random sample of n = 10 time differences Step 2: Verify data conditions … Boxplot shows no outliers nor extreme skewness. Victor Katch Kinesiology

33
**Example Effect of Alcohol (cont)**

Step 2: … Summarizing data with a test statistic Test of mu = 0.0 vs mu > 0.0 Variable N Mean StDev SE Mean T P Diff Key elements: Sample statistic: = (under “Mean”) Standard error: (under “SE Mean”) (under “T”) Victor Katch Kinesiology

34
**Example Effect of Alcohol (cont)**

Step 3: Find the p-value From output: p-value = 0.013 From Table A.3: p-value is between and The value t = 2.68 is between column headings 2.58 and in the table, and for df =9, the one-sided p-values are and Victor Katch Kinesiology

35
**Example Effect of Alcohol (cont)**

Steps 4 and 5: Decide whether or not the result is statistically significant based on the p-value and Report the Conclusion Using a = 0.05 as the level of significance criterion, we can reject the null hypothesis since the p-value of is less than Even with a small experiment, it appears that alcohol has a statistically significant effect and decreases performance time. Victor Katch Kinesiology

36
**Testing The Difference between Two Means (Independent Samples)**

Step 1: Determine null and alternative hypotheses H0: m1 – m2 = 0 versus Ha: m1 – m2 0 or Ha: m1 – m2 < 0 or Ha: m1 – m2 > 0 Watch how Population 1 and 2 are defined. Step 2: Verify data conditions and compute test statistic Both n’s are large or no extreme outliers or skewness in either sample. Samples are independent. The t-test statistic is: Steps 3, 4 and 5: Similar to t-test for one mean. Victor Katch Kinesiology

37
**Example Effect of Stare on Driving**

Randomized experiment: Researchers either stared or did not stare at drivers stopped at a campus stop sign; Timed how long (sec) it took driver to proceed from sign to a mark on other side of the intersection. Question: Does stare speed up crossing times? Step 1: State the null and alternative hypotheses H0: m1 – m2 = 0 versus Ha: m1 – m2 > 0 where 1 = no-stare population and 2 = stare population. Victor Katch Kinesiology

38
**Example Effect of Stare (cont)**

Data: n1 = 14 no stare and n2 = 13 stare responses Step 2: Verify data conditions … No outliers nor extreme skewness for either group. Victor Katch Kinesiology

39
**Example Effect of Stare (cont)**

Step 2: … Summarizing data with a test statistic Sample statistic: = 6.63 – 5.59 = 1.04 seconds Standard error: Victor Katch Kinesiology

40
**Example Effect of Stare (cont)**

Steps 3, 4 and 5: Determine the p-value and make a conclusion in context. The p-value = 0.013, so we reject the null hypothesis, the results are “statistically significant”. The p-value is determined using a t-distribution with df = 21 (df using Welch approximation formula) and finding area to right of t = Table A.3 => p-value is between and We can conclude that if all drivers were stared at, the mean crossing times at an intersection would be faster than under normal conditions. Victor Katch Kinesiology

41
**The Two Types of Errors and Their Probabilities**

When the null hypothesis is true, the probability of a type 1 error, the level of significance, and the a-level are all equivalent. When the null hypothesis is not true, a type 1 error cannot be made. Victor Katch Kinesiology

42
Type I Error A Type I error is the mistake of rejecting the null hypothesis when it is true. The symbol (alpha) is used to represent the probability of a type I error. Example on page 375 of text Victor Katch Kinesiology

43
Type II Error A Type II error is the mistake of failing to reject the null hypothesis when it is false. The symbol (beta) is used to represent the probability of a type II error. Victor Katch Kinesiology

44
Example: Assume that we a conducting a hypothesis test of the claim p > Here are the null and alternative hypotheses: H0: p = 0.5, and H1: p > 0.5. a) Identify a type I error. b) Identify a type II error. Victor Katch Kinesiology

45
Example: Assume that we a conducting a hypothesis test of the claim p > Here are the null and alternative hypotheses: H0: p = 0.5, and H1: p > Identify a type I error. A type I error is the mistake of rejecting a true null hypothesis, so this is a type I error: Conclude that there is sufficient evidence to support p > 0.5, when in reality p = 0.5. Victor Katch Kinesiology

46
Example: Assume that we a conducting a hypothesis test of the claim p > Here are the null and alternative hypotheses: H0: p = 0.5, and H1: p > Identify a type II error A type II error is the mistake of failing to reject the null hypothesis when it is false, so this is a type II error: Fail to reject p = 0.5 (and therefore fail to support p > 0.5) when in reality p > 0.5. Victor Katch Kinesiology

47
**Type I and Type II Errors**

Victor Katch Kinesiology

48
**Controlling Type I and Type II Errors**

For any fixed , an increase in the sample size n will cause a decrease in For any fixed sample size n , a decrease in will cause an increase in . Conversely, an increase in will cause a decrease in . To decrease both and , increase the sample size. page 377 in text Victor Katch Kinesiology

49
**Power of a Hypothesis Test**

Definition Power of a Hypothesis Test The power of a hypothesis test is the probability (1 - ) of rejecting a false null hypothesis, which is computed by using a particular significance level and a particular value of the population parameter that is an alternative to the value assumed true in the null hypothesis. Victor Katch Kinesiology

50
**Trade-Off in Probability for Two Errors**

There is an inverse relationship between the probabilities of the two types of errors. Increase probability of a type 1 error => decrease in probability of a type 2 error Victor Katch Kinesiology

51
Type 2 Errors and Power Three factors that affect probability of a type 2 error 1. Sample size; larger n reduces the probability of a type 2 error without affecting the probability of a type 1 error. 2. Level of significance; larger a reduces probability of a type 2 error by increasing the probability of a type 1 error. 3. Actual value of the population parameter; (not in researcher’s control. Farther truth falls from null value (in Ha direction), the lower the probability of a type 2 error. When the alternative hypothesis is true, the probability of making the correct decision is called the power of a test. Victor Katch Kinesiology

Similar presentations

© 2021 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google