2 Outline Review the One-sample case Independent samples Paired samples BasicsUnequal NExampleIssuesPaired samplesExamplesComparing formulasConclusion
3 Two samples Previously compared sample mean to a known population mean Now want to compare two samplesNull hypothesis: the mean of the population of scores from which one set of data is drawn is equal to the mean of the population of the second data setH0: m1=m2 or m1 - m2 = 0
4 Independent samples Consider the original case Now want to consider not just 1 mean but the difference between 2 meansThe ‘nil’ hypothesis, as before, states there will be no differenceH0: m1 - m2 = 0
5 Which leads to...Now statistic of interest is not a single mean, but the difference between means:Mean of the ‘sampling distribution of the differences between means’ is:
6 Variability Standard error of the difference between means Since there are two independent variables, variance of the difference between means equals sum of their variances
7 Same problem, same solution Usually we do not know population variance (standard deviation)Again use sample to estimate itResult is distributed as t (rather than z)
9 But... If we are dealing with a ‘nil’ null hypothesis: So the formula reduces to:Across the 2 samples we have (n1-1) and (n2-1) degrees of freedom1df = (n1-1) + (n2-1) = n1 + n2 - 2
10 Unequal sample sizesAssumption: independent samples t test requires samples come from populations with equal variancesTwo estimates of variance (one from each sample)Generate an overall estimate that reflects the fact that bigger samples offer better estimatesOftentimes the sample sizes will be unequal
11 Weighted averageWhich gives us:Final result is:
12 ExampleNew drug MemoPlus is supposed to aid memory. 5 people take the drug and we compare them to a control group of 10 people who take a placebo.Look at their free recall of 10 items on a word list.Control: mean = 4.5, SD = 2.0Drug: mean = 6.0, SD = 3.0
13 Start with variance Using Get: [note pooled variance nearer to group’s with larger n]
14 Calculate t Enter the values: Critical value approach Specific p
15 Conclusion?Our result does not give us a strong case for rejecting the null. MemoPlus, like Ginko Biloba1, does nothing special for memory.1. Solomon et al. (2002) JAMA.
16 More issues with the t-test In the two-sample case we have an additional assumption (along with normal distribution of sample scores and independent observations)We assume that there are equal variances in the groupsRecall our homoscedasticity discussion, it is the exact same assumption1Often this assumption is untenable, and the results, like other violations result in using calculated probabilities that are inaccurateCan use a correction, e.g. Welch’s t1. Note also that our normality assumption still technically deals with the residuals in terms of the general linear model, more on that later.
17 More issues with the t-test It is one thing to say that they are unequal, but what might that mean?Consider a control and treatment group, treatment group variance is significantly greaterWhile we can do a correction, the unequal variances may suggest that those in the treatment group vary widely in how they respond to the treatment (e.g. half benefit, ¼ unchanged, ¼ got worse)Another reason for heterogeneity of variance may be related to an unreliable measure being usedNo version of the t-test takes either into considerationOther techniques, assuming enough information has been gathered, may be more appropriate (e.g. a hierarchical approach), and more reliable measures may be attainable1. Note that, if those in the treatment are truly more variable, a more reliable measure would actually detect this more so (i.e. more reliability would lead to a less powerful test). We will consider this more later.
18 T-test for dependent samples Paired sample t-testComparing two distributions that are likely to be correlated in some wayGist: some sort of meaningful relationship between our two samples.When are two distributions likely to be correlated?Natural Pairs:Comparing the scores of two subjects who are related naturallyE.g. twins.Matched Pairs:Pairs two subjects on some characteristic that is likely to be important to the variable that you are studying. Members of the pair are randomly assigned to the two conditions.E.g. married couplesRepeated Measures:Same person at two different times.
19 Difference scoresWhile we technically have two samples of data, the test regards the single sample of difference scores from one pair to anotherTransform the paired scores into a single set of scoresFind the difference for each caseNow are we have a single set of scores and a known population mean, a familiar situationUse the one sample t test conceptIf the null hypothesis is true then on average there should be no differenceThus mean difference is 0 ( mD = 0 )
20 Paired t test Where = Mean of the difference scores sD = Standard deviation of the difference scoresn = Number of difference scores (# pairs)df = n-1
21 ExampleWe want to know if dog owners affection for their dog changes after they take a course in dog psychologyWe measure their responses on the the Dog Affection Inventory both before and after the course
22 Data t (9)= 3.71, p = .005 2 Conclusion ? Note that which sample is taken from which is arbitrary, just as in the independent samples t-test. Common practice is to report positive t statistics, with tabled or in text reports very clear on which mean is greater and which lesser.t.test(before,after, paired=T)
24 The good and the bad regarding t-tests If assumptions are met, t-test is fineWhen assumptions aren’t met, t-test may still be robust with regard to type I error in some situationsWith equal n and normal populations HoV violations won’t increase type I muchWith non-normal distributions with equal variances, type I error rate is maintained alsoThe badEven small departures from the assumptions result in power taking a noticeable hit (type II error is not maintained)t-statistic, CIs will be biased