Independent Samples and Paired Samples t-tests PSY440 June 24, 2008

Statistical analysis follows design The one-sample t-test can be used when: –1 sample –One score per subject –Population mean (  ) is known –but standard deviation (  ) is NOT known

Testing Hypotheses –Step 1: State your hypotheses –Step 2: Set your decision criteria –Step 3: Collect your data –Step 4: Compute your test statistics Compute your estimated standard error Compute your t-statistic Compute your degrees of freedom –Step 5: Make a decision about your null hypothesis Hypothesis testing: a five step program

Performing your statistical test What are we doing when we test the hypotheses? –Consider a variation of our memory experiment example Population of memory patients MemoryTest  is known Memory treatment Memory patients Memory Test X Compare these two means Conclusions: the memory treatment sample are the same as those in the population of memory patients. they aren’t the same as those in the population of memory patients H0:H0: HA:HA:

Performing your statistical test What are we doing when we test the hypotheses? Real world (‘truth’) H 0 : is false (is a treatment effect) Two populations XAXA they aren’t the same as those in the population of memory patients H 0 : is true (no treatment effect) One population XAXA the memory treatment sample are the same as those in the population of memory patients.

Performing your statistical test What are we doing when we test the hypotheses? –Computing a test statistic: Generic test Could be difference between a sample and a population, or between different samples Based on standard error or an estimate of the standard error

Performing your statistical test Test statistic One sample z One sample t identical

Performing your statistical test Test statistic Diff. Expected by chance Standard error One sample z One sample t different don’t know this, so need to estimate it

Performing your statistical test Test statistic Diff. Expected by chance Standard error Estimated standard error One sample z One sample t different don’t know this, so need to estimate it Degrees of freedom

One sample t-test The t-statistic distribution (a transformation of the distribution of sample means transformed) –Varies in shape according to the degrees of freedom New table: the t-tablet-table

One sample t-test If test statistic is here Fail to reject H 0 Distribution of the t-statistic If test statistic is here Reject H 0 –To reject the H 0, you want a computed test statistics that is large The alpha level gives us the decision criterion New table: the t-tablet-table The t-statistic distribution (a transformation of the distribution of sample means transformed)

One sample t-test New table: the t-table One tailed - or - Two-tailed  levels Critical values of t t crit Degrees of freedom df

One sample t-test What is the t crit for a two-tailed hypothesis test with a sample size of n = 6 and an  -level of 0.05? Distribution of the t-statistic  = 0.05 Two-tailed n = 6 df = n - 1 = 5 t crit =

One sample t-test Distribution of the t-statistic  = 0.05 One-tailed n = 6 df = n - 1 = 5 t crit = What is the t crit for a one-tailed hypothesis test with a sample size of n = 6 and an  -level of 0.05?

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment. How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors. Step 1: State your hypotheses H0:H0: the memory treatment sample are the same as those in the population of memory patients. HA:HA: they aren’t the same as those in the population of memory patients  Treatment >  pop = 60  Treatment <  pop = 60

One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment. Step 2: Set your decision criteria H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60  = 0.05 One -tailed How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors. An example: One sample t-test

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment. Step 2: Set your decision criteria H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60  = 0.05 One -tailed How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors.

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment.  = 0.05 One -tailed Step 3: Collect your data H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60 How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors.

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment.  = 0.05 One -tailed Step 4: Compute your test statistics = -2.5 H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60 How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors.

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment.  = 0.05 One -tailed Step 4: Compute your test statistics t = -2.5 H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60 How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors.

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment.  = 0.05 One -tailed Step 5: Make a decision about your null hypothesis H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60 How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors. t crit =

One sample t-test An example: One sample t-test Memory experiment example: We give a n = 16 memory patients a memory improvement treatment.  = 0.05 One -tailed Step 5: Make a decision about your null hypothesis H 0 :  Treatment >  pop = 60 H A :  Treatment <  pop = 60 How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal,  = 60? After the treatment they have an average score of = 55, s = 8 memory errors = t crit t obs = Reject H 0

t Test for Dependent Means Unknown population mean and variance Two situations –One sample, two scores for each person Repeated measures design –Two samples, but individuals in the samples are related Same procedure as t test for single sample, except –Use difference scores –Assume that the population mean is 0

Statistical analysis follows design The related-samples t- test can be used when: –1 sample –Two scores per subject

Statistical analysis follows design The related-samples t- test can be used when: –1 sample –Two scores per subject –2 samples –Scores are related - OR -

Performing your statistical test Diff. Expected by chance Estimated standard error of the differences Test statistic What are all of these “D’s” referring to? Mean of the differences Number of difference scores Difference scores –For each person, subtract one score from the other –Carry out hypothesis testing with the difference scores Population of difference scores with a mean of 0 –Population 2 has a mean of 0

Performing your statistical test What are all of these “D’s” referring to? Difference scores Person Pre-test Post-test (Pre-test) - (Post-test) H0:H0: There is no difference between pre-test and post- test HA:HA: There is a difference between pre-test and post- test  D = 0  D ≠ 0

Performing your statistical test What are all of these “D’s” referring to? Difference scores = 5.5 Person Pre-test Post-test (Pre-test) - (Post-test)

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test Difference scores D = 5.5

Performing your statistical test What are all of these “D’s” referring to? Person Difference scores Pre-test Post-test D = = = = = = SS D D - D(D - D) 2

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) D - D Difference scores

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D D - D Difference scores

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D 1.45 = s D ? Think back to the null hypotheses D - D Difference scores

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D 1.45 = s D H 0 : Memory performance at the post-test are equal to memory performance at the pre-test D - D Difference scores

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D 1.45 = s D This is our t obs D - D Difference scores

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D 1.45 = s D t obs  = 0.05 Two-tailed t crit D - D Difference scores

+3.18 = t crit - Reject H 0 Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D 1.45 = s D t obs  = 0.05 Two-tailed t crit D - D Difference scores t obs =3.8

Performing your statistical test What are all of these “D’s” referring to? Person Pre-test Post-test D = = SS D (D - D) = s D 1.45 = s D t obs  = 0.05 Two-tailed t crit T obs > t crit so we reject the H D - D Difference scores

Statistical analysis follows design The related-samples t- test can be used when: –1 sample –Two scores per subject

Statistical analysis follows design The related-samples t- test can be used when: –1 sample –Two scores per subject –2 samples –Scores are related - OR -

Performing your statistical test Test statistic Diff. Expected by chance One sample z One sample tRelated samples t

Independent samples What are we doing when we test the hypotheses? –Consider a new variation of our memory experiment example Memory treatment Memory patients Memory Test the memory treatment sample are the same as those in the population of memory patients. they aren’t the same as those in the population of memory patients H0:H0: HA:HA: Memory placebo Memory Test Compare these two means

Statistical analysis follows design The independent samples t-test can be used when: –2 samples –Samples are independent

Performing your statistical test Estimate of the standard error based on the variability of both samples

Performing your statistical test Test statistic One-sample t Independent-samples t Sample means

Performing your statistical test Test statistic One-sample t Independent-samples t Population means from the hypotheses

Performing your statistical test Test statistic One-sample t Independent-samples t Population means from the hypotheses H0:H0: Memory performance by the treatment group is equal to memory performance by the no treatment group. So:

Performing your statistical test Test statistic One-sample t Estimated standard error (difference expected by chance) estimate is based on one sample We have two samples, so the estimate is based on two samples

Performing your statistical test “pooled variance” We combine the variance from the two samples Number of subjects in group A Number of subjects in group B

variance Performing your statistical test “pooled variance” We combine the variance from the two samples Recall “weighted means,” need to use “weighted variances” here Variance (s 2 ) * degrees of freedom (df)

Performing your statistical test Independent-samples t Compute your estimated standard error Compute your t-statistic Compute your degrees of freedom This is the one you use to look up your t crit

Performing your statistical test Person Exp. group Control group Need to compute the mean and variability for each sample Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = 0.05.

Performing your statistical test Person Exp. group Control group Need to compute the mean and variability for each sample Control group = 50 (45-50) 2 + (55-50) 2 + (40-50) 2 + (60-50) 2 = 250 SS = A Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = 0.05.

Performing your statistical test Exp. group ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 = 155 SS = B Person Exp. group Control group Need to compute the mean and variability for each sample Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = = 44.5

Performing your statistical test Person Exp. group Control group Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = = 0.95

Performing your statistical test T obs = 0.95 T crit = ±2.447  = 0.05 Two-tailed Person Exp. group Control group Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = = 0.95

Performing your statistical test T obs = 0.95  = 0.05 Two-tailed T crit = ±2.447 Person Exp. group Control group Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = = t crit - Fail to Reject H 0 t obs =0.95 = 0.95

Performing your statistical test T obs = 0.95  = 0.05 Two-tailed T crit = ±2.447 T obs T crit Compare < Fail to reject the H 0 Person Exp. group Control group Dr. Mnemonic develops a new treatment for patients with a memory disorder. He isn ’ t certain what impact, if any, it will have. To test it he randomly assigns 8 patients to one of two samples. He then gives one sample the new treatment but not the other. Following the treatment period he gives both groups a memory test. The data are presented below. Use  = = 0.95

Assumptions Each of the population distributions follows a normal curve The two populations have the same variance If the variance is not equal, but the sample sizes are equal or very close, you can still use a t-test If the variance is not equal and the samples are very different in size, use the corrected degrees of freedom provided after Levene’s test (see spss output)

Using spss to conduct t-tests One-sample t-test: Analyze =>Compare Means =>One sample t-test. Select the variable you want to analyze, and type in the expected mean based on your null hypothesis. Paired or related samples t-test: Analyze =>Compare Means =>Paired samples t-test. Select the variables you want to compare and drag them into the “pair 1” boxes labeled “variable 1” and “variable 2” Independent samples t-test: Analyze =>Compare Means =>Independent samples t-test. Specify test variable and grouping variable, and click on define groups to specify how grouping variable will identify groups.

Using excel to compute t-tests =t-test(array1,array2,tails,type) Select the arrays that you want to compare, specify number of tails (1 or 2) and type of t-test (1=dependent, 2=independent w/equal variance assumed, 3=independent w/unequal variance assumed). Returns the p-value associated with the t- test.

t-tests and the General Linear Model Think of grouping variable as x and “test variable” as y in a regression analysis. Does knowing what group a person is in help you predict their score on y? If you code the grouping variable as a binary numeric variable (e.g., group 1=0 and group 2=1), and run a regression analysis, you will get similar results as you would get in an independent samples t-test! (try it and see for yourself)

Conceptual Preview of ANOVA Thinking in terms of the GLM, the t-test is telling you how big the variance or difference between the two groups is, compared to the variance in your y variable (between vs. within group variance). In terms of regression, how much can you reduce “error” (or random variability) by looking at scores within groups rather than scores for the entire sample?

Effect Size for the t Test for Independent Means Estimated effect size after a completed study

Statistical Tests Summary DesignStatistical test (Estimated) Standard error One sample,  known One sample,  unknown Two related samples,  unknown Two independent samples,  unknown