EXPERIMENTAL DESIGN Random assignment Who gets assigned to what? How does it work What are limits to its efficacy?

RANDOM ASSIGNMENT Equal probability of assignment to each condition (treatment, control, etc.) –or fixed, known probability if other design conditions are included Use of random number table, computer- generated random number to make assignments

WHO GETS ASSIGNED Primary units (such as students, patients, or clients) assigned individually without additional personal information used Assignment within personal or demographic categories- gender, psychological diagnosis, etc. Multiple levels of assignment- pools used for selection

How Randomization Works Distributes various causal conditions, variables equally across assignment conditions Generates random differences in initial conditions, pretest scores whose variance can be estimated in probability Creates individual variation (“error”) that is independent of treatment

Limits of Efficacy Randomization does not last forever- groups begin to change over time in unknown ways History is uncontrolled Maturation is uncontrolled over long periods of time Testing effects are not controlled Mortality effects are not controlled

TWO GROUP MEANS TESTS

Two independent groups experiments Randomization distributions. 6 scores (persons, things) can be randomly split into 2 groups 20 ways:                     1 2 3

Two independent groups experiments Differences between groups can be arranged as follows: look familiar?

t-distribution Gossett discovered it similar to normal, flatter tails different for each sample size, based on N-2 for two groups (degrees of freedom) randomization distribution of differences is approximated by t-distribution

t-distribution assumptions NORMALITY –(W test in SPSS) HOMOGENEITY OF VARIANCES IN BOTH GROUPS’ POPULATIONS –Levene’s test in SPSS INDEPENDENCE OF ERRORS –logical evaluation –Durbin-Watson test in serial data

Null hypothesis for test of means for two independent groups H 0 :  0 -  1 =0 H 1 :  0 -  1  0. fix a significance level, . Then we select a sample statistic. In this case we choose the sample mean for each group, and the test statistic is the sample difference d = y 0 – y 1.

Null hypothesis for test of means for two independent groups t = d / s d __________________________________________ = (y 0 – y 1 )/  {{ [(n 0 –1)s (n 1 – 1)s 2 1 ] / (n 0 + n 1 –2)} { 1/n 0 + 1/n 1 } The variance of a difference of two scores is: s 2 (y 1 -y 2 ) = s s r 12 s 1 s 2

Standard deviation of differences s 2 (y 1 -y 2 ) = s s r 12 s 1 s 2 Example, s 2 1 = 100, s 2 2 = 144, r 12 =.7 s 2 (y 1 -y 2 ) = (.7)(10)(12) = = 76 s (y 1 -y 2 ) = 8.72

Standard deviation of differences s 2 (y 1 -y 2 ) = s s r 12 s 1 s 2 Example, s 2 1 = 100, s 2 2 = 144, r 12 =0 n 1 = 24, n 2 =16 s 2 (y 1 -y 2 ) = = 244 s (y 1 -y 2 ) = 15.62

0 SD=15.62 t-distribution, df= = 38

Standard error of mean difference score standard error of the sample difference. It consists of the square root of the average variance of the two samples,  d 2 =[(n 0 –1)s (n 1 – 1)s 2 1 ] / (n 0 + n 1 –2) divided by the sample sizes ( 1/n 0 + 1/n 1 )  d 2 =  d 2 / ( 1/n 0 + 1/n 1 ) Same concept as seen in sampling distribution of single mean

Example Willson (1997) studied two groups of college freshman engineering students, one group having participated in an experimental curriculum while the other was a random sample of the standard curriculum. One outcome of interest was performance on the Mechanics Baseline Test, a physics measure (Hestenes & Swackhammer, 1992). The data for the two groups is shown below. A significance level of.01 was selected for the hypothesis that the experimental group performed better than the standard curriculum group (a directional test): GroupMeanSDSample size Exper Std Cur __________________________________________ t = (47 – 37) /  [(74 y 15 2 ) + (49 y 16 2 ) / ( – 2)][1/75 + 1/50] _______________________________ = (10) /  [( ) / (123)][1/75 + 1/50] = The t-statistic is compared with the tabled value for a t-statistic with 123 degrees of freedom at the.01 significance level, The observed probability of occurrence is =.02691, greater than the intended level of significance. The conclusion was that the experimental curriculum group, while performing better than the standard, did not significantly outperform them.

1416 not

experimentwise error probability of a Type I error in any of the tests, called the experimentwise error Rough approximation: < k  Example, if we run 3 t-tests at p=.05, experimentwise error rate <.15 limit by setting experimentwise error to some value, like.05, then  =.05/k Called Bonferroni correction (when calculated exactly) = 1 - (1-  ) k

Confidence interval around d d  t   {{[(n 0 –1)s (n 1 – 1)s 2 1 ] / (n 0 + n 1 –2)} { 1/n 0 + 1/n 1 } Thus, for the example, using the.01 significance level the confidence interval is  (5.136) = (-2.11, 22.11). Thus, the population mean difference is somewhere between about -2 and 22 This includes 0 (zero) so we do not reject the null hypothesis.

Wilcoxon rank sum test for two independent groups. While the t-distribution is the randomization distribution of standardized differences of sample means for large sample sizes, for small samples it is not the best procedure for all unknown distributions. If we do not know that the population is normally distributed, a better alternative is the Wilcoxon rank sum test.

Wilcoxon rank sum test for two independent groups. Heart Rejected?Survival days Yes624, 46, 64, 1350, 280, 10, 1024, 39, 730, 136 No15, 3, 127, 23, 1, 44, 551, 12, 48, 26 Ranks for data aboveSum Yes17, 10, 12, 20, 15, 3, 19, 8, 18, No5, 2, 13, 6, 1, 9, 16, 4, 11, 774 Test StatisticsRANKDAY Mann-Whitney U Wilcoxon W Z Asymp. Sig. (2-tailed).019 Exact Sig. [2*(1-tailed Sig.)].019

Confidence interval for S Confidence interval for S. While S (or U) may not be an obvious statistic to think about, both have the same standard deviation ____________ s S =  n 1 n 2 (n + 1)/12 so that for the asymptotic normality condition (with n 1 and n 2 at least 8 each), for alpha =.05. S  1.96 s S gives a 95% confidence interval. For the data above s S = 13.23, and the 95% confidence interval is 74  = (48.07, 99.93).

Correlation representation of the two independent groups experiment r 2 pb t 2 =  (1 – r 2 pb )/ (N-2) t 2 r 2 pb =  t 2 + N - 2 N=n 1 + n 2

Correlation representation of the two independent groups experiment t r pb =  t 2 + N - 2 1/2

x y e r pb Path model representation of two group experiment

Test of point biserial=0 H 0 :  pb = 0 H 1 :  pb  0 is equivalent to t-test for difference for two means.

Fig. 6.4: Scatterplot of ranks of days of survival for persons who experienced tissue rejection (1) or not (0) 0 1 NO YES REJECTION

Dependent groups experiments d = y 1 – y 0 for each pair. Now the hypotheses about the new scores becomes H 0 :  = 0 H 1 :   0 The sample statistic is simply the sample difference. The standard error of the difference can be computed from the standard deviation of the difference scores divided by n, the number of pairs

Dependent groups experiments _________________ s d =  [s s 2 1 –2r 12 s 0 s 1 ]/n Then the t-statistic is _ t = d / s d

Dependent groups experiments In a study of the change in grade point average for a group of college engineering freshmen, Willson (1997) recorded the following data over two semesters for a physics course: Variable N Mean Std Dev PHYS PHYS Correlation Analysis: r 12 =.5517 To test the hypothesis that the grade average changed after the second semester from the first, for a significance level of.01, the dependent samples t- statistic is ________________________________________ t = [2.648 – 2.233]/ [ – 2 (.5517) x x 1.201]/128 =.415 /.1001 = This is greater than the tabled t-value  t(128) = Therefore, it was concluded the students averaged higher the second semester than the first.

Nonparametric test of difference in dependent samples. sign test. A count of the positive (or negative) difference scores is compared with a binomial sign table. This sign test is identical to deciding if a coin is fair by flipping it n times and counting the number of heads. Within a standard error of.5n 1/2 the number should be equal to n/2.As n becomes large, the distribution of the number of positive difference scores divided by the standard error is normal. An alternative to the sign test is the Wilcoxon signed rank test or symmetry test

Summary of two group experimental tests of hypothesis Table is a compilation of last two chapters: –sample size –one or two groups –normal distribution or not –known or unknown population variance(s)