1. Introduction to Hypothesis Testing
Chapter 11
Introduction to Hypothesis Testing

2. Introduction
The purpose of hypothesis testing is to determine whether there is enough statistical evidence supporting a certain belief about a parameter.
Examples
Is there statistical evidence in a random sample of potential customers, that support the hypothesis that more than p% of all potential customers will purchase a new products?
Is the hypothesis that a certain drug is effective supported by the level of improvement in patients’ conditions after treated with the drug, compared with this of another group of patients who were given a placebo?

3. 11.1 Concepts of Hypothesis Testing
Two hypotheses are defined.
H0: The null hypothesis. Under this hypothesis we specify our current belief about the parameter we test. (m = 170, p = .4, etc.)
H1: The alternative hypothesis. Under this hypothesis we specify a range of values for the parameter tested (m > 170; p ¹ .4; etc.)
effected by some action taken.
This is the hypothesis we try to prove!

4. Concepts of Hypothesis Testing
The two hypotheses are stated, and a test is run to determine whether a sample statistic supports the rejection of H0 in favor of H1.
H0: m = 170
H1: m > 170

5. The Concept of Hypothesis Testing
Let’s assume H0 is true: m = 170
A sample is drawn. Assume the sample mean = 180.
If we have little incentive to believe m > 170 because and m are relatively close.
m = 170

6. The Concept of Hypothesis Testing
Let’s assume H0 is true: m = 170
A sample is drawn. Now assume the sample mean = 250.
If we have much more incentive to believe
m > 170 because falls far above m.
The question is: How far is far?
Is 250 sufficiently larger than 170 for us to believe that m > 170? Click.
m = 170

7. The Concept of Hypothesis Testing
Let’s assume H0 is true: m = 170
You may want to think about it as follows. Click:
This is the probability that
when m > 170
With m = 170… click
If m were greater than 170… click
This is the probability that
when m = 170
m = 170
m > 170
As you can see it becomes more likely that when m > 170

8. The Concept of Hypothesis Testing
We’ll look next at the probability that as a tool to help decide whether we should reject H0.
This idea will be further discussed (with a somewhat more computational flavor) as example 1 is presented next.
Pay attention!

9. 11.2 Testing the Population Mean when the Population Standard Deviation is Known
Example 1: Department Store new Billing System
A new billing system for a department store will be cost- effective only if the mean monthly account is more than $170.
A sample of 400 accounts has a mean of $178.
If the accounts are approximately normally distributed with s = $65, can we conclude that the new system will be cost effective? (can we conclude from the sample result that the accounts population mean is greater than 170?)

10. Testing the Population Mean (s is Known)
Example 1 - Solution
The population of interest is the credit accounts at the store.
We want to show that the mean account for all customers is greater than $170.
This is what you want to prove
H1 : m > 170
The null hypothesis must specify the values of the parameter m, not included in H1
H0 : m £ 170

11. Testing the Population Mean (s is Known)
To better understand the hypotheses testing concept let us ask the following question:
If H0 is true (m = 170) how likely is it a sample of 400 accounts have a sample mean at least as large as 178?
Answer: By the central limit theorem
To illustrate, by sheer chance, out of samples of 400 accounts each only 69 samples will have a sample mean of 178 or more, if indeed m = 170.
It seems there must be another reason (rather than just “chance”) why the event has occurred. Click.
Most likely m > 170, which explains better why That is, H0 should be rejected in favor of H1

12. Types of Errors
Testing the hypotheses, two types of errors may occur when deciding whether to reject H0 based on the sample result.
Type I error: Reject H0 when it is true.
Type II error: Do not reject H0 when it is false.

13. Types I and Type II Errors in Example 1
Example 1 - continued
Type I error: Believe that m > 170 when the real value of m is 170 (reject H0 in favor of H1 when H0 is true).
Type II error: Believe that m £ 170 when the real value of m > 170 (do not reject H0 when it is false).

14. Controlling the probability of conducting a type I error
Recall:
H0: m £ 170 H1: m > 170, Since the alternative hypothesis has the form of m > m0, H0 is rejected if is sufficiently large!
Our job is to determine a critical value for the sample mean.
H0 is rejected if the sample mean exceeds that critical value.
Critical value
m = 170 H0

15. Controlling the probability of conducting a type I error
Recall:
H0: m £ 170 H1: m > 170,
So how do we determine this critical value?
We turn to a type I error and limit the probability it occurs.
Note. May exceed a critical value (leading to the rejection of H0)
but the population mean may still be 170. We don’t want the
probability of this event exceeds some acceptable value (a).
Critical value
m = 170 H0

16. Approaches to Testing
There are two approaches to test whether the sample mean supports the alternative hypothesis (H1)
The rejection region method is mandatory for manual testing (but can be used when testing is supported by a statistical software)
The p-value method which is mostly used when a statistical software is available.
Both involve an upper limit we set on the probability of conducting a type I error.

17. The Rejection Region Method
The null hypothesis is rejected in favor of the alternative hypothesis if a test statistic falls in the rejection region.

18. The Rejection Region Method of a Right Hand Tail Test
Example 1 – solution continued
Recall: H0: m £ H1: m > 170.
Define a critical value for that is just large enough to reject the null hypothesis.
Reject the null hypothesis if

19. Determining the Critical Value for the Rejection Region of a Right Hand Tail Test
Allow the probability of committing a type I error be a (also called the significance level).
Find a critical value of the sample mean that is just large enough to guarantee that the actual probability of committing a type I error does not exceed a.

20. Determining the Critical Value for a Right Hand Tail Test
Example 1 – solution continued a
m = 170
P(commit a type I error) = P(reject H0 when H0 is true)
From the central limit theorem:

21. Determining the Critical Value for a Right Hand Tail Test
Example 1 – solution continued a
m = 170
and

22. Determining the Critical Value for a Right Hand Tail Test
Example 11.1 – solution continued
Simple algebra a

23. Determining the Critical Value for a Right Hand Tail Test
Conclusion
Since the sample mean (178) is greater than the critical value of , there is sufficient evidence to infer that the mean monthly balance is greater than $170 at 5% significance level.

24. Determining the Critical Value for a Right Hand Tail Test
Interpretation
The null hypothesis is rejected in favor of the alternative hypothesis because the sample mean falls in the rejection region. Still we may be erroneous when rejecting the null hypothesis, since m could be 170, but the chance we make such a mistake is not greater than 5% (the significance level).

25. The standardized test statistic
Instead of using the statistic , we can use the standardized value z.
If the alternative hypothesis is: H1: m > m0, then the rejection region is
H0: m = m0

26. The standardized test statistic
Example 1 - continued
We redo this example using the standardized test statistic.
Recall: H0: m = 170
H1: m > 170
Test statistic:
Rejection region: z > z.05 =

27. The standardized test statistic
Example continued
Conclusion
Since Z = > 1.645, reject the null hypothesis in favor of the alternative hypothesis.

28. The P-value Method
Ask the question: How probable is it to obtain a sample mean at least as extreme as 178, if the population mean is 170 (H0 is true)?

29. P-value method
The probability of observing a test statistic at least as extreme as 178, given that m = 170 is…
The p-value

30. Interpreting the p-value
Because the probability that the sample mean will assume a value of more than 178 when m = 170 is so small (.0069), there are reasons to believe that m > 170.
Note how the event
is rare under H0
when but...
…it becomes more
probable under H1,
when

31. Interpreting the p-value
We can conclude that the smaller the p-value the more statistical evidence exists to support the alternative hypothesis.

32. P-value – Summary
The p-value of a test is the probability of observing a test statistic at least as extreme as the one computed, given that the null hypothesis is true.
The p-value provides information about the amount of statistical evidence that supports the alternative hypothesis.

33. Interpreting the p-value
Describing the p-value
If the p-value is less than 1%, there is overwhelming evidence that supports the alternative hypothesis.
If the p-value is between 1% and 5%, there is a strong evidence that supports the alternative hypothesis.
If the p-value is between 5% and 10% there is a weak evidence that supports the alternative hypothesis.
If the p-value exceeds 10%, there is no evidence that supports the alternative hypothesis.

34. The p-value and the rejection region methods
The p-value can be used when making decisions based on rejection region methods as follows:
Compare the p-value to a. Reject the null hypothesis only if the p value < a; Otherwise, do not reject the null hypothesis.
Note: < 0.05! a
= 0.05
The p-value =

35. Left Hand Tail Test H0: m = m 0 H1: m < m 0 Reject H0 if falls here
Critical
value

36. An Example for a Left Hand Tail Test
The SSA envelop plan example.
The chief financial officer in FedEx believes that including a stamped self-addressed (SSA) envelop in the monthly invoice sent to customers will decrease the amount of time it take for customers to pay their monthly bills.
Currently, customers return their payments in 22 days on the average, with a standard deviation of 6 days.

37. An Example for a Left Hand Tail Test
The SSA envelop example – continued
A random sample of 220 customers was selected and SSA envelops were included with their invoice packs.
The time it took customers to pay their bill was recorded (see SSA)
Can the CFO conclude that the plan will be successful at 10% significance level?

38. An Example for a Left Hand Tail Test
The SSA envelop example – Solution
The parameter tested is the ‘population mean of the payment time’ (m).
Since the CFO wants to prove that the plan will be successful, we test whether H1: m < 22
Accordingly, The null hypothesis is: H0: m = 22

39. An Example for a Left Hand Tail Test
The SSA envelop example – Solution continued
The rejection region: It makes sense to believe that m < 22 if the sample mean is sufficiently smaller than 22.
Thus, reject the null hypothesis if
m=22
Rejection Region

40. The Standardized Rejection Region for a Left Hand Tail Test
Note that a is small (certainly less than 50%). So the critical Z value must be negative. Click.
-za a
The standardized rejection region is:

41. An Example for a Left Hand Tail Test
The SSA envelop example – Solution continued
The standardized approach: From the data we find that the sample mean = 21.44
This is the sample mean
-za
Za = Z.10 = so, -Z.10 =
Conclusion: Since < –1.285 reject the null hypothesis.

42. The p – value approach for a Left Hand Tail Test
The SSA envelop example – Solution continue
The p value = P(Z<-1.384) = and a = 0.1
Since < .1 (p value<a) reject the null hypothesis.
p value

43. An Example for a Two Tail Test
H0: m = m0
H1: m ¹ m0
Critical
value
Reject H0 if falls here
Reject H0 if falls here

44. An Example for a Two Tail Test
AT&T has been challenged by competitors whose rates arguably resulted in lower bills.
A statistician believes the monthly mean and standard deviation of the long-distance bills for all AT&T residential customers are $17.85 and $3.87 respectively.

45. An Example for a Two Tail Test
Example 2 - continued
A random sample of 25 customers is selected and customers’ bills recalculated using a leading competitor’s rates.
Assuming the standard deviation is indeed 3.87, can we infer that there is a difference between AT&T’s bills and the competitor’s bills (on the average)?

46. An Example for a Two Tail Test The Rejection Region approach
Solution
Is the mean different than 17.85?
(see ATT)
H0: m = 17.85
Define a two tail rejection region of the form…
17.85

47. An Example for a Two Tail Test The Rejection Region approach
Solution - continued
17.85
a/2 = 0.025
a/2 = 0.025
Even under H0 (m =17.85), can fall far above or far below 17.85, in which case we erroneously state that
We do not want this erroneous rejection of H0 occurs too frequently, say not more than a = 5% of the time.

48. An Example for a Two Tail Test The Rejection Region approach
Solution - continued
From the sample we have:
19.13
17.85

49. An Example for a Two Tail Test The Rejection Region approach
Solution - continued
From the sample we have:
19.13
17.85
Since falls between the two critical values, do not reject the null hypothesis

50. An Example for a Two Tail Test Standardized approach
Solution - continued
Do not reject the null hypothesis
a/2 = 0.025
-za/2 = -1.96
za/2 = 1.96
Rejection region

51. An Example for a Two Tail Test P – Value approach
The two areas combined form the p value
a/2 = 0.025
a/2 = 0.025
-1.65
1.65
-za/2 = -1.96
za/2 = 1.96
The p-value = P(Z< -1.65)+ P(Z >1.65)
= 2 P(Z >1.65) > .05

52. Conclusion: There is insufficient evidence to infer that there is a difference between the bills of AT&T and the competitor, at 5% significance level.

53. 11.3 Calculating the Probability of a Type II Error
To properly interpret the results of a hypothesis test, we need to
specify an appropriate significance level or judge the p-value of a test;
understand the relationship between Type I and Type II errors.
How do we compute a type II error probability?

54. Calculating the Probability of a Type II Error
To calculate the probability of a type II error we need to…
Express the rejection region directly, in terms of the parameter hypothesized (not standardized).
Specify the alternative value under H1
H0: m = m0 H1: m = m1

55. Calculating the Probability of a Type II Error
Let us revisit example 1
The null hypothesis was H0: m = 170
Let the alternative value be m = 180 (rather than just m>170)
H0: m = 170
H1: m = 180
Specify the alternative value under H1.
m= 170
m=180

56. Calculating the Probability of a Type II Error
Express the rejection region directly, not in standardized terms
Let us revisit example 1
The rejection region was with a = .05.
H0: m = 170
H1: m = 180
a=.05
m= 170
175.34
m=180

57. Calculating the Probability of a Type II Error
A type II error occurs when a false H0 is not rejected.
H0 is false when
H0 is not rejected when
So, the probability a type II errors occurs is
H0: m = 170
H1: m = 180
m= 170
m=180
175.34

58. Calculating the Probability of a Type II Error
To summarize:
when m = 180)
H1: m = 180
m=180
True

59. Judging the Test
A hypothesis test is effectively defined by the significance level a and by the sample size n.
The probability of a type II error b can be controlled by
changing a, and/or
changing the sample size.

60. Effects on b of changing a
Increasing the significance level a, decreases the value of b, and vice versa. b1
a2 > a1
b2 <
m0= 170
ma =180

61. Judging the Test Increasing the sample size n reduces b
So, by increasing the sample size decreases, and grows smaller. L x

62. Judging the Test
Graphical demonstration: Note what happens when n increases:
Small n b
H0 :m= 170
H1: m=180
Larger n a
xL moves to the left, thus,
b grows smaller.
H0 :m= 170
H1: m=180 b

63. Judging the Test Increasing the sample size reduces b
In example 1, suppose ‘n’ increases from 400 to 1000.
The probability of conducting a type I error a remains 5%, but the probability of conducting a type II error b drops dramatically.

64. Judging the Test Power of a test
The power of a test is defined as 1 - b.
It represents the probability to reject the null hypothesis when it is false.

65. Optional: Determining the Sample Size for a Hypothesis test about the Population Mean (known s)
It has been shown that b and ‘n’ are inversely related (increasing ‘n’ decreases b).
So, for a desired value of b we can determine the required sample size.
The formula to determine ‘n’ is:
For a two tailed test Za/2 replaces Za.

66. Optional: Determining the Sample Size for a one tail test – Example
Example 8: Determine the sample size needed to test H0: m = 100 against H1: m = 130, if the significance level is 2.5% and the desired probability of a type II error is 8%. The population standard deviation is known to be 30.
Solution: Za = Z.025 = 1.96; Zb = Z.08 = The selected sample size is therefore n = 12.