1. Psych 5500/6500 Null Hypothesis Testing: General Concepts
and Introduction to
The t Test for a Single Group Mean
Fall, 2008

2. Introduction
Time to move on to the heart of experimental science, formulating hypotheses to test theories, gathering data, and then deciding which hypothesis is supported by the data.The general approach is called 'null hypothesis testing'.

3. t Test for a Single Group Mean
We will introduce null hypothesis testing by seeing how it is carried out in what is called the t test for a single group mean.
This test is rarely used in statistics, but it makes a very nice lead into null hypothesis testing from our earlier work on the sampling distribution of the mean.
As the semester progresses we will return to null hypothesis testing again and again, each time will simply be a variation on the same theme.

4. t Test for a Single Group Mean
The t test for a single group mean involves making hypotheses about the true value of the mean of the population from which we are sampling. Its limitation is that you need to have some theoretical reason for hypothesizing specific values about the population mean...

5. Example
We have an IQ test that has a μ=100 for the general population of the United States. We want to test a theory which predicts that the population of people in the country of Elbonia should have a different μ than that of the U.S.
The theory simply predicts that the mean IQ of Elbonians is different, it doesn’t predict specifically whether it should be greater or less than that of the U.S. (we will cover that type of hypothesis testing later).

6. General Approach
We will draw a sample of Elbonians and find the mean IQ of that sample. From that we will decide whether or not the mean of the population of Elbonians is different than the mean of the population of the U.S. (i.e. whether μElbonians100)

7. The Challenge
The challenge we face is this, let’s say that the mean of the sample of Elbonians is 106, does that prove the Elbonians have higher IQ’s (i.e. that μElbonians  μAmericans )? Or, might it simply be that Elbonians are the same as people in the USA (i.e. μElbonians = μAmericans = 100) but that we just happened to get a high sample mean due to chance (i.e. your sample has random bias). This is exactly the sort of thing that null hypothesis testing was set up to handle.

8. Two Hypotheses
The first step in null hypothesis testing is to divide reality into two mutually exclusive and exhaustive hypotheses (i.e. one, and only one, of these will be true):
null hypothesis (H0): the hypothesis that you set up in hopes of rejecting. Often, but not always, it is the hypothesis of 'no difference'.
alternative hypothesis (HA): the hypothesis you hope to prove. Often, but not always, it is the hypothesis of 'difference'.

9. H0: μElbonia = 100 HA: μElbonia  100
The exact form of the null and alternative hypotheses depend upon the statistical procedure you are using. In our example (t test for a single group mean):
H0: μElbonia = 100
HA: μElbonia  100
Note:
Hypotheses are always about populations,
Together H0 and HA cover all possibilities.

10. Two Possible Results
The results are always stated in terms of the null hypothesis (this is ‘null hypothesis testing’ after all). There are two and only two possible results of an experiment (in this class always give one of the following two as the ‘result of the experiment’):
‘Do not reject H0’
‘Reject H0’
Later we will take a look at why it is more accurate to say ‘do not reject H0’ than ‘accept H0’’.

11. Implications Regarding HA
Although our decision is stated in terms of H0, once we decide about H0 that implies something about HA (as the two hypotheses are mutually exclusive and exhaustive)
‘reject H0’ implies ‘accept HA’.
‘do not reject H0’ implies ‘do not accept HA’.

12. Four Possibilities Reality Decision 1) Ho true 2) Ho false
Reality
1) Ho true
2) Ho false
Decision
1) Reject H0
Type 1 error
p = ‘alpha’
Correct decision
p = ‘power’
2) Do not reject H0
Correct
decision
Type 2 error
p = ‘beta’

13. Type 1 Error and alpha
Type 1 error: rejecting H0 when in reality H0 was true.
alpha (α): the probability that you will make a type 1 error. This is a conditional probability:
Correct interpretation of alpha:
α=p(you will decide to reject H0 | H0 is actually true).
Incorrect interpretations of alpha:
p(H0 is true | you decide to reject H0)
p(making a type one error | you decide to reject H0)
p(making a type one error | you are doing null hypothesis testing)
By definition a type 1 error can only occur when H0 is true.

14. Type 2 Error and beta
Type 2 error: not rejecting H0 when in reality H0 was false.
beta (β): the probability you will make a type 2 error. This is also a conditional probability:
β = p(deciding to not reject H0 | H0 is actually false)
by definition a type 2 error can only occur when H0 is false.

15. power power = p(deciding to reject H0 | H0 is actually false)
Power involves making a correct decision, to reject H0 when in reality H0 was false.
We rarely know the actual values of power and beta, but we do know the following:
power + beta = (so we know that when power goes up beta goes down and vice versa). We will devote a whole lecture just on power later in the semester.

16. Basic Approach in Null Hypothesis Testing
Determine the probability of getting the data you did if H0 were true. In this case, what is the probability that we would have gotten a sample mean (106) so far away from what H0 predicted (100) if H0 were actually true? i.e. p(sample mean of 6 or more away from 100 | H0 is actually true)
If that probability is very small, then reject H0.
Example of IQ scores

17. Coin Example H0: the coin is fair, i.e. p(Heads)=0.5
HA: the coins is biased, i.e. p(Heads)0.5
1st coin flip = Heads, p(one head|H0)=0.5
2nd coin flip = Heads, p(two heads|H0)=.25
3rd coin flip = Heads, p(three heads|H0)=.125
4th coin flip = Heads, p(four heads|H0)=.0625
5th coin flip = Heads, p(five heads|H0)=.03125
6th coin flip = Heads, p(six heads|H0)=.01563
7th coin flip = Heads, p(seven heads|H0)=.0078

18. Significance Level
At what point do we say that the data are so unlikely given H0 that we want to reject H0 in favor of HA?
Where we draw the line is called our ‘significance level’, it is how improbable the results have to be to reject H0. In psychology we usually use p=.05, or if we wish to be conservative and not reject H0 without more proof, we use p=.01
Note: H0 could still be true, we can’t rule that out, we can only say the results are improbable if H0 were true, so we will go with HA instead.

19. Significance Level and alpha
Once you set your significance level, you have also set the value of alpha (‘α’).
alpha = your significance level.
Thus the value of alpha--the probability of making a type one error given H0 is true--is under your direct control based upon the significance level you choose (an a priori decision).
a priori decision

20. In Our Example
1) Our hypotheses were:
H0: μElbonia = 100
HA: μElbonia  100
We set our significance level at .05.
The mean of our sample of Elbonians was 106.
If the probability of getting a sample mean at least six away from 100 if H0 were true is, say, p=.04, then we would decide to ‘reject H0’, as the probability of getting such a sample mean if H0 were true is less than .05 (our significance level). Our sample mean was improbable if H0 were true.
If, however, the probability of getting a sample mean at least six away from 100 if H0 were true is, say, p=.17, then we would decide to ‘not reject H0’, as the sample mean is not all that unlikely if H0 were true (i.e. p > significance level of .05).
You can, however, influence the value of beta and power through how you design your experiment...much more on this later.

21. beta and power
Usually you can not directly set the value of beta (‘β’) or power (but at least you do know that beta+power=1.00)