1. T-tests continued

2. Outline Review the One-sample case Independent samples Paired samples
Basics
Unequal N
Example
Issues
Paired samples
Examples
Comparing formulas
Conclusion

3. Two samples Previously compared sample mean to a known population mean
Now want to compare two samples
Null hypothesis: the mean of the population of scores from which one set of data is drawn is equal to the mean of the population of the second data set
H0: m1=m2 or m1 - m2 = 0

4. Independent samples Consider the original case
Now want to consider not just 1 mean but the difference between 2 means
The ‘nil’ hypothesis, as before, states there will be no difference
H0: m1 - m2 = 0

5. Which leads to...
Now statistic of interest is not a single mean, but the difference between means:
Mean of the ‘sampling distribution of the differences between means’ is:

6. Variability Standard error of the difference between means
Since there are two independent variables, variance of the difference between means equals sum of their variances

7. Same problem, same solution
Usually we do not know population variance (standard deviation)
Again use sample to estimate it
Result is distributed as t (rather than z)

8. Formula
All of which leads to:

9. But... If we are dealing with a ‘nil’ null hypothesis:
So the formula reduces to:
Across the 2 samples we have (n1-1) and (n2-1) degrees of freedom1
df = (n1-1) + (n2-1) = n1 + n2 - 2

10. Unequal sample sizes
Assumption: independent samples t test requires samples come from populations with equal variances
Two estimates of variance (one from each sample)
Generate an overall estimate that reflects the fact that bigger samples offer better estimates
Oftentimes the sample sizes will be unequal

11. Weighted average
Which gives us:
Final result is:

12. Example
New drug MemoPlus is supposed to aid memory. 5 people take the drug and we compare them to a control group of 10 people who take a placebo.
Look at their free recall of 10 items on a word list.
Control: mean = 4.5, SD = 2.0
Drug: mean = 6.0, SD = 3.0

13. Start with variance Using Get:
[note pooled variance nearer to group’s with larger n]

14. Calculate t Enter the values: Critical value approach Specific p

15. Conclusion?
Our result does not give us a strong case for rejecting the null. MemoPlus, like Ginko Biloba1, does nothing special for memory.
1. Solomon et al. (2002) JAMA.

16. More issues with the t-test
In the two-sample case we have an additional assumption (along with normal distribution of sample scores and independent observations)
We assume that there are equal variances in the groups
Recall our homoscedasticity discussion, it is the exact same assumption1
Often this assumption is untenable, and the results, like other violations result in using calculated probabilities that are inaccurate
Can use a correction, e.g. Welch’s t
1. Note also that our normality assumption still technically deals with the residuals in terms of the general linear model, more on that later.

17. More issues with the t-test
It is one thing to say that they are unequal, but what might that mean?
Consider a control and treatment group, treatment group variance is significantly greater
While we can do a correction, the unequal variances may suggest that those in the treatment group vary widely in how they respond to the treatment (e.g. half benefit, ¼ unchanged, ¼ got worse)
Another reason for heterogeneity of variance may be related to an unreliable measure being used
No version of the t-test takes either into consideration
Other techniques, assuming enough information has been gathered, may be more appropriate (e.g. a hierarchical approach), and more reliable measures may be attainable
1. Note that, if those in the treatment are truly more variable, a more reliable measure would actually detect this more so (i.e. more reliability would lead to a less powerful test). We will consider this more later.

18. T-test for dependent samples
Paired sample t-test
Comparing two distributions that are likely to be correlated in some way
Gist: some sort of meaningful relationship between our two samples.
When are two distributions likely to be correlated?
Natural Pairs:
Comparing the scores of two subjects who are related naturally
E.g. twins.
Matched Pairs:
Pairs two subjects on some characteristic that is likely to be important to the variable that you are studying. Members of the pair are randomly assigned to the two conditions.
E.g. married couples
Repeated Measures:
Same person at two different times.

19. Difference scores
While we technically have two samples of data, the test regards the single sample of difference scores from one pair to another
Transform the paired scores into a single set of scores
Find the difference for each case
Now are we have a single set of scores and a known population mean, a familiar situation
Use the one sample t test concept
If the null hypothesis is true then on average there should be no difference
Thus mean difference is 0 ( mD = 0 )

20. Paired t test Where = Mean of the difference scores
sD = Standard deviation of the difference scores
n = Number of difference scores (# pairs)
df = n-1

21. Example
We want to know if dog owners affection for their dog changes after they take a course in dog psychology
We measure their responses on the the Dog Affection Inventory both before and after the course

22. Data t (9)= 3.71, p = .005 2 Conclusion ?
Note that which sample is taken from which is arbitrary, just as in the independent samples t-test. Common practice is to report positive t statistics, with tabled or in text reports very clear on which mean is greater and which lesser.
t.test(before,after, paired=T)

23. ts and Zs

24. The good and the bad regarding t-tests
If assumptions are met, t-test is fine
When assumptions aren’t met, t-test may still be robust with regard to type I error in some situations
With equal n and normal populations HoV violations won’t increase type I much
With non-normal distributions with equal variances, type I error rate is maintained also
The bad
Even small departures from the assumptions result in power taking a noticeable hit (type II error is not maintained)
t-statistic, CIs will be biased