Design Supplemental

Design Completely randomized Randomized block design Repeated measures Latin-Squares Greco-Latin Squares Repeated measures Pre & post-test designs Mixed designs (Between subjects factors and repeated measures) Split-plot Random effects models

Completely randomized The typical situation with only between-subjects factors (IVs) Participants are randomly assigned to treatment groups Don’t trust yourself to do this, peeps is bad at randomizing Example in R x=c(1,2,3,4)  groups 1-4 y=c(.25,.25,.25,.25)  long run probability of occurrence sample(x, 80, replace=TRUE, prob = y)

Randomized block design Suppose an effect is present that may influence results, but for which you may not be interested in Nuisance variable We can extend the randomization process to control for a variety of factors Randomized Block Design An extension of the completely randomized design in which a single extraneous variable that might affect test units’ response to the treatment has been identified and the effects of this variable are isolated by blocking out its effects. Examples: Researchers collecting the data Probably a relationship between who is collecting and outcome Experimenter/participant expectations Age Create blocks of similarly aged individuals

Randomized block design In such a situation we want to randomly assign to treatment groups within each blocking level Example: Treatment A Experimenter 1: condition 1 condition 2 condition 3 Experimenter 2: condition 1 condition 2 condition 3 Experimenter 3: condition 1 condition 2 condition 3

Randomized Block Design Independent Variables Control: no music Experimental treatment slow music Experimental treatment: fast music Mornings and afternoons Evening hours Blocking variable

Randomized block design Partitioning the sums of squares F-statistic is now MSb/t/MSresidual This will be a more powerful test, although less so as SSresdiual approaches SSw/in i.e. less effect due to blocking variable

Randomized block design Conditions: Correlation between blocking variable and DV Treatment and blocking levels > 2 Random assignment within each blocking unit Goal is to see homogeneous groups w/in each blocking level and treatment levels will look similarly with respect to one another Methods of implementation Subject matching (age example) Repeated measures (everyone gets all conditions, subjects serve as their own block) Other matched pair random assignment (e.g. twins, husband-wife) Latin-squares and Graeco-Latin squares extend the blocking to 2 and 3 nuisance variables respectively Consult Winer or Kirk texts

Latin Square Design A balanced, two-way classification scheme that attempts to control or block out the effect of two or more extraneous factors by restricting randomization with respect to the row and column effects Randomly assign treatment levels according to levels of the nuisance variable such that each level of the treatment variable is assigned to the levels of the nuisance variables Can help with sequence effects with RM variables This special sort of balancing means that the systematic variation between rows, or similarity between columns, does not affect the comparison of treatments.

Previous therapy (none, yes but no drugs, yes w/ drugs) Latin Square Example controlling for previous therapy experience and therapist (A is the treatment, e.g. type of therapy) 1 2 3 1 A1 A2 A3 2 A2 A3 A1 3 A3 A1 A2 Previous therapy (none, yes but no drugs, yes w/ drugs) Example: 3 groups receive Therapy A1 Group1- therapist1, no previous Group2- therapist2, drugs Group3- therapist3, no drugs Therapist

LS design has limited utility in behavioral sciences Not too many cases with factors all with the same number of levels Kirk has suggested that with less than 5 levels it’s not very practical due to few degrees of freedom for error term (except for when several levels of treatment are assigned to each cell e.g. in a mixed design situation) Nevertheless, we see we have various ways to control for a number of effects that might be present in our data but not of interest to the study

Repeated measures design Subjects participate in multiple levels of a factor Simplest is the paired t-test situation, however ‘repeated measures design’ often implies 3 or more applications of some treatment Example: pre and post-test

Mixed design Between subjects factors with repeated measures Simple but common example is control-treatment, pre-post Subjects are randomly assigned to control or treatment groups and tested twice Pre Post Control Treatment

Mixed design This example may appear a straightforward method for dealing with data However we could analyze it in two additional ways ANCOVA Test for differences at time 2 controlling (adjusting) for differences seen at time 1 A simple t-test on the gain scores from pre to post Same result as mixed but w/ different (simpler) output One method may speak more to the exact nature of your research question (e.g. only interested in time 2 or the rate of improvement from time 1 to 2)

Mixed design Consider also the possibility of an effect in posttest scores (or effect of treatment) due solely to exposure to the pretest E.g. with no prior training GRE scores would naturally be assumed to increase the second time around more often than not Solomon four-group design Split-plot factorial controls for a nuisance variable in mixed designs We’ll take a more in depth look at repeated measures and mixed designs later

Random effects models In most ANOVA design we are dealing with fixed effects Random effects If the levels included in our analysis represent a random sample of all the possible levels, we have a random-effects ANOVA. With fixed effects the same levels of the effect would be used again if the experiment was repeated. With random effects different levels of the effect would be used. The conclusion of the random-effect ANOVA applies to all the levels (not only those studied).

Random effects models Questions to ask: Were the individual levels of the factor selected due to a particular interest, or were they chosen completely at random? Will the conclusions be specific to the chosen levels, or will they be applied to a larger population? If the experiment were repeated, would the same levels be studied again, or would new levels be drawn from the larger population of possible levels?

Random effects models Some examples: Persuasiveness of commercials Effect of worker on machine output Vehicle emissions impact on the environment

Random effects models Determination of fixed vs. random effects affects the nature of the tests involved, how the F-statistic will be calculated (sometimes), and the conclusions to be drawn Some designs will incorporate both fixed and random effects Mixed effects designs Don’t confuse with previous mixed design factorial discussed previously