Testing Differences between Means, continued Statistics for Political Science Levin and Fox Chapter Seven

Testing Differences between Means To test the significance of a mean difference we need to find the standard deviation for any obtained mean difference. However, we rarely know the standard deviation of the distribution of mean differences since we rarely have population data. Fortunately, it can be estimated based on two samples that we draw from the same population.

Step 2b: Translate our sample mean difference into units of standard deviation. 3 Z = ( 1 – 2 ) - 0 Where = mean of the first sample = mean of the second sample 0 = zero, the value of the mean of the sampling distribution of differences between means (we assume that µ 1 - µ 2 = 0) = standard error of the mean (standard deviation of the distribution of the difference between means) We can reduce this equation down to the following: Remember this formula required the standard deviation of the distribution of mean differences.

Result: (assuming equals 2) 4 Z = ( 45 – 40) Thus, a difference of 5 between the means of the two samples (women and men) falls 2.5 standard deviations from a mean of zero. 2 Z =+ 2.5 Child Rearing: Comparing Males and Females

Standard Error of the Difference between Means Here is how the standard error of the difference between means can be calculated. The formula for combines the information from the two samples.

Where A large difference between Xbar 1 and Xbar 2 can result if (1) one mean is very small, (2) one mean is very large, or (3) one mean is moderately small and the other is moderately large.

Variance: Weeks on Unemployment: X (weeks) N=6 Deviation: (raw score from the mean) (raw score from the mean, squared) Variance: = 4 8-5=3 6-5=1 4-5=-1 2-5=-3 1-5= = = = = = = 16 (weeks squared) ΣX=30 χ= 30=5 6 Step 1: Calculate the Mean Step 3: Calculate Sum of square Dev Step 2: Calculate Deviation Step 4: Calculate the Mean of squared dev.

Testing the Difference between Means Let’s say that we have the following information about two samples, one of liberals and one of conservatives, on the progressive scale: LiberalsConservatives N 1 = 25N 2 = 35 = 60 = 49 S 1 = 12S 2 = 14 We can use this information to calculate the estimate of the standard error of the difference between means:

We start with our formula: The standard error of the difference between means is We can now use our result to translate the difference between sample means to a t ratio.

We can now use our standard error results to change difference between sample mean into a t ratio: t = 60 – t = t = 3.13 REMEMBER: We use t instead of z because we do not know the true population standard deviation.

We aren’t finished yet! Turn to Table C. 1) Because we are estimating for both σ 1 and σ 2 from s 1 and s 2, we use a wider t distribution, with degrees of freedom N 1 + N 2 – 2. 2) For each standard deviation that we estimate, we lose 1 degree of freedom from the total number of cases. N = 60 Df ( ) = 58 In Table C, use a critical value of 40 since 58 is not given. We see that our t-value of 3.13 exceeds all the standard critical points except for the.001 level. Therefore, based on what we established BEFORE our study, we reject the null hypothesis at the.10,.05, or.01 level. df

Comparing the Same Sample Measured Twice Some research employs a panel design or before and after test (testing the same sample at two points in time). In these types of studies, the same sample is tested twice. It is not two samples from the same population, it is a measuring the same group of people twice. CRITICAL POINTS TO NOTE: 1. The same sample measured twice uses the. 1. The same sample measured twice uses the t-test of difference between means. 2. Different samples from the same population selected at two points in time use the. 2. Different samples from the same population selected at two points in time use the t-test of difference between means for independent groups.

Example Problem of Test of Difference Between Means for Same Sample Measured Twice Null Hypothesis (µ 1 = µ 2 ): The degree of neighborliness does not differ before and after relocation. Research Hypothesis (µ 1 ≠ µ 2 ): The degree of neighborliness differs before and after relocation. Where µ 1 is the mean score of neighborliness at time 1 Where µ 2 is the mean score of neighborliness at time 2 RespondentBefore (X 1 ) After (X 2 ) Difference (D = X 1 – X 2 ) Difference 2 (D 2 ) Johnson2111 Robinson121 Brown3124 Thomas3124 Smith121 Holmes4139 ∑ X 1 = 14∑ X 2 = 8∑ D 2 = 20

The formula for obtaining the standard deviation for the distribution of before- after difference scores s D = standard deviation of the distribution of before-after difference scores D = after-move raw score subtraction from before-move raw score N = number of cases or respondents in sample From this, we get the formula for the standard error of the difference between the means:

14 6 = = = = = 1.53 =.68 t = 60 – t = 3.13 Step 1: Find mean for each point in time Step 2: Find the SD for the diff between the times Step 3: Find the SE for the diff between the times Step 4: Translate the mean diff into a t score

Comparing the Same Sample Measured Twice df = (n – 1) = 6 – 1 = 6 – 1 = 5 = 5 Step 5: Calculate the degrees of freedom Step 6: compare the obtained t ratio with t ratio in Table C Obtained t = 1.47 Table t = df = 5 α =.05 To order reject the null hypothesis at the.05 significance with five degrees of freedom we must obtain a calculated t ratio of Because our t ratio is only 1.47 – we retain the null hypothesis. df

Two Sample Test of Proportions Where P1 and P2 are respective sample proportions. The standard error of the difference in proportions is: Where P* is the combined sample proportion

Requirements when considering the appropriateness of the t- ratio as a test of significance. (For Testing the Difference between Means): 1. The t ratio is used to make comparisons between two means. 2. The assumption is that we are working with interval level data. 3. We used a random sampling process. 4. The sample characteristic is normally distributed. 5. The t ratio for independent samples assumes that the population variances are equal.

So how do you interpreting the results and state them for inclusion in your research? “ “Since the observed value of t (state the test statistic) exceeds the critical value (state the critical value), the null hypothesis is rejected in favor of the directional alternative hypothesis. The probability that the observed difference (state the difference between means) would have occurred by chance, if in fact the null hypothesis is true, is less than.05.”