1. From Confidence to Hypothesis-Testing @UWE_KAR
Practice & Communication of Science
From Confidence to Hypothesis-Testing
@UWE_KAR

2. Where We Are/Where We Are Going
Many things we measure are normally dist
When we sample an ND population, we get…
the standard deviation of the sample
sample mean  estimate of the population mean
plus a measure of the mean’s distribution (the SEM)
 a 95%CI of the mean
calculation uses the t-distribution (via t-table)
95%CI = mean ± (t(N-1),0.05 * SEM)
range around the estimated mean in which 95/100 further estimates of the pop mean would lie
So far, so descriptive…
we are using the data to estimate the pop mean
but what if we already know the pop mean?

3. Comparing to Something Known
If we already know the pop mean then another possibility opens up. Eg…
a survey of daily travel time to UWE (min) gave…
26,33,65,28,34,55,25,44,50,36,26,37,43,62,35,38, 45,32,28,34
mean is 38.8 ± 11.7 min, n = 20
national average for commuting is 45.8 min
we can ask the following question; are our times…
significantly different to the national average?
ie does the difference of 7 min mean something?
this is hypothesis-testing, where…
not significantly different is the Null Hypothesis
significantly different is the Alternative Hypothesis

4. Testing Differences
We can extend our use of the t-dist to help us decide which to accept and which to reject
UWE (38.8 ± 11.7 min, n=20) vs Nation (45.8)
SEM = 11.7/√20 = 2.62
DoF = N-1 = 19
95%CI = sample mean ± (t(20-1),0.05 * SEM)
= 38.8 ± (t(20-1),0.05 * 2.62) = 38.8 ± (2.093 * 2.62)
= 38.8 ± 5.48= (33.32, min)
Confident that 95/100 UWE surveys would yield a mean between and min
does not include the National average, so we conclude we are significantly different to the rest! 

5. Testing Differences In practice usually calculate slightly differently
calculate the diff between sample and pop means
calculate t as diff / SEM (this ‘standardises’ it)
compare t to ‘critical t’ in t-table
if t > critical-t then reject Null Hypo
UWE (38.8 ± 11.7 min, n=20) vs Nation (45.8)
diff = 38.8 – 45.8 = -7 min
SEM diff = 11.7/√20 = 2.62
t = -7/2.62 = = 2.67 (we ignore sign)
critical t(20-1),0.05 = 2.093
2.67>2.093 (SEMs to start of 2.5% ‘tail’)
so in ‘tail’, so p<0.05, so reject the Null Hypo

6. Testing Differences
Or stick the numbers into a stats package like Minitab and ask it to do a 1-sample t-test
One-Sample T: C1
Test of mu = 45.8 vs not = 45.8
Variable N Mean StDev SE Mean % CI T P
UWE time (33.33, 44.27)
Note actual p-value calculated..
ie 1.5% chance that a difference in travel time this big would be seen if travel to UWE was no different to the national picture
‘no different’ is the Null Hypo
so reject the Null Hypo in favour of alternative
travel to UWE is significantly quicker than national average

7. A Special Case of the 1-sample t-test
Say we surveyed the same people at UWE before and after some road improvements
are interested in seeing the effect of improvements
for each person, we have two pieces of data;
before and after travel times
if we put the data into two columns, then adjacent cells ‘pair up’ – the data are said to be paired
now calculate difference between each data pair
end up with third column containing the differences
it will have a mean, an SD and an ‘n’
The Null Hypo says that the mean of our third column is not significantly different to 0
we are doing a paired t-test

8. The Paired t-test
Before travel time (min)…26,33,65,28,34, 55,25,44,50,36,26,37,43,62,35,38,45,32,28,34
38.8 ± 11.7 min, n = 20
After travel time (min)…28,30,62,29,31, 54,22,41,52,33,25,38,43,60,31,37,42,31,29,34
37.6 ± 11.5 min, n = 20
At first sight this does not look promising…
two means are close, and huge overlap of SDs
Differences (min)…-2,3,3,-1,3,1,3,3,-2,3,1, ,0,2,4,1,3,1,-1,0
1.2 ± min, n = 20
so, SEM = 1.908/√20 = min

9. The Paired t-test t = mean diff / SEM
from the table, critical t(20-1),0.05 = 2.093
2.81>2.093 (SEMs to start of 2.5% ‘tail’)
our Null Hypo says that expected mean diff is 0
so 0 min (the Null Hypo) is in ‘tail’
so p<0.05
so reject the Null Hypo
the shortening of journey time by an average of 1.2 min is significant!
The paired t-test is very powerful as it compensates for between subject variation

10. The Paired t-test
In practice put paired data into Minitab and ask it to do a paired t-test…
N Mean StDev SE Mean
Before
After
Difference
95% CI for mean difference: (0.307, 2.093)
T-Test of mean difference = 0 (vs not = 0): T-Value = P-Value = 0.011
Note actual p-value calculated..
1.1% chance that a difference in travel time this big would be seen if road improvements had no effect
‘no different’ is the Null Hypo
so reject the Null Hypo in favour of alternative
travel to UWE has been significantly improved

11. The 2-sample t-test
The paired t-test has two columns of data, but the test is actually done on a single column, the differences between pairs of data
but what if we have two data samples that don’t ‘pair up’ (they are independent of each other)?
eg data from males and also from females
This is where the 2-sample t-test comes in…
the 2-sample t-test is an extension of what we just looked at
aka unpaired t-test
but we will approach it in a way that also incorporates the four general steps that underpin hypothesis-testing

12. The Basis of the 2-sample t-Test
Say we are looking at growth of schoolchildren, and we measured heights…
the values of heights will vary and their distribution might look pretty ‘normal’
but science is all about trying to explain variation, so one part of our ‘explanation’ of variation in height might be that some of it is down to gender

13. The Basis of the 2-sample t-Test
Ie, when it comes to height, male and female schoolchildren belong to different populations…

14. The Basis of the 2-sample t-Test
The Null Hypothesis says…
No, the two sets of data (male and female) are drawn from the same population (just plain schoolchildren)
How can we decide?
We know that repeated sampling from a single population yields different means, just by chance
So how far ‘apart’ must our male & female means be before we conclude it is not just chance?
We need some sort of ‘measure’ of separation
called the ‘test statistic’
and a probability level, p, we are happy with
generally use the 95% (5% or 0.05) level

15. The Basis of the 2-sample t-Test
The ‘test statistic’ is a measure of ‘separation’ of our two putative populations, male & female
for the t-test it is the t-value
Depends on diff in means and variability…
big difference in means implies…
a strong ‘signal’ that the two populations differ
diff would be zero if samples the same
big variability implies…
a lot of ‘noise’ masking the diff in means ‘signal’
The t-value is like a signal-to-noise ratio!

16. The Basis of the 2-sample t-Test
The bigger our ‘signal-to-noise ratio’, t, the less likely we are dealing with two samples from the same population
ie we can reject the Null Hypo and…
accept the Alternative Hypo that male and female schoolchildren differ significantly in their heights
How big t needs to be depends on…
degrees of freedom (N-2 in this case)
p-value we are working at (alpha, usually 0.05)
We look up the critical value of t in the t-table, just like we did when calculating a Confidence Interval

17. 2-sample t-Test – Worked Example
Say we are looking at lung function in schoolchildren, say the FVC…
50 boys and 55 girls (n doesn’t have to be same)
Male data…
2.159, 2.065, 1.518, 2.227, 2.09, 2.451, 1.871, 2.571, 2.532, 2.545, 2.538, 2.795, 2.102, 1.804, 2.432, 2.704, 2.258, 2.282, 1.663, 2.795, 2.238, 1.953, 2.382, 2.344, 2.967, 2.68, 2.413, 2.444, 1.953, 2.314, 2.15, 2.634, 2.598, 2.09, 2.641, 2.92, 2.727, 2.307, 2.76, 2.439, 2.259, 2.111, 2.58, 2.602, 2.461, 3.128, 2.241, 2.602, 3.177, 2.419
𝑥 = 2.399, s = L, n = 50
Female data…
1.913, 2.18, 1.56, 1.586, 1.712, 2.038, 1.791, 1.869, 2.296, 1.897, 1.846, 2.246, 2.318, 1.934, 1.92, 1.958, 2.521, 2.04, 2.19, 1.886, 1.734, 2.148, 2.198, 2.351, 2.193, 1.772, 2.38, 1.776, 2.505, 2.438, 2.317, 2.857, 2.604, 2.275, 1.727, 2.185, 2, 2.428, 2.304, 1.775, 2.537, 1.904, 2.519, 2.611, 2.425, 2.302, 2.366, 1.999, 3.111, 1.923, 2.978, 2.673, 2.311, 2.428, 2.407
𝑥 = 2.185, s = L, n = 55

18. 2-sample t-Test – Worked Example
Like any statistical test there are four stages…
Formulate the Null Hypothesis
Generate a test statistic
Use the test statistic to work out a probability
Interpret the probability
1: Formulate the Null Hypothesis
there is a probability of 5% or more that our observed differences in FVC between male and female schoolchildren arose by chance and they both belong to a single underlying population called schoolchildren
ie gender has no significant influence on FVC in schoolchildren

19. 2-sample t-Test – Worked Example
2: Generate the test statistic, t
previously, t calculated from diff in means/SEM
easy to get the simple difference in male and female means
but we have two sets of data contributing to the SEM of the difference
if the variances (square of SD) are similar…
SEM diff = √(SDA2/nA + SDB2/nB)
for our data
t = (2.399 – 2.385) √((0.3482/50) + (0.3452/55))
t = 3.16

20. 2-sample t-Test – Worked Example
3: Use test statistic to look up the probability
Row; DoF is N-2 (103)
Column;
level of ‘confidence’
α = 0.05 (5%)
critical t = 1.96
our t, 3.16 > 1.96
ie a mean diff of 0 (Null Hypo) lies more than 1.96 SEM along the distribution
so p<0.05

21. 2-sample t-Test – Worked Example
4: Interpret the probability
p < 0.05
but our t-value (3.16) > critical t (2.58) at α = 0.01
so actually < 0.01
this means that the chances of ‘randomly’ picking two samples from the same population that are as far apart as we saw, with the variability in each sample we saw, is < 1%
ie < 1% chance the Null Hypo is true
so we conclude that these samples represent two different populations
ie we accept the Alt Hypo that males/females differ

22. 2-sample t-Test – Worked Example
In practice put data into Minitab and ask it to do a 2-sample t-test…
Two-sample T for Male vs Female
N Mean StDev SE Mean
Male
Female
Difference = mu (Male) - mu (Female)
Estimate for difference:
95% CI for difference: (0.0798, )
T-Test of difference = 0 (vs not =): T-Value = P-Value = DF = 103 (Both use Pooled StDev = )
Note actual p-value calculated..
0.2% chance that a difference in FVC this big would be seen if gender had no effect
so reject the Null Hypo in favour of alternative
gender significantly affects FVC

23. Summary Hypothesis-testing tests the Null Hypothesis
Like any statistical test there are four stages…
Formulate the Null Hypothesis
Generate a test statistic (in this case, t)
Use the test statistic to work out a probability
Interpret the probability
For t-test, t = ‘diff’/SEM (‘signal’/’noise’)
1-sample t-test
test a sample against an expected mean
paired t-test tests ‘before-after’ data (against 0)
2-sample t-test (unpaired t-test)
test two independent samples to see if means differ