Random Effects & Repeated Measures Alternatives to Fixed Effects Analyses

Questions  What is the difference between fixed- and random-effects in terms of treatments?  How are F tests with random effects different than with fixed effects?  Describe a concrete example of a randomized block design. You should have 1 factor as the blocking factor and one other factor as the factor of main interest.

Questions (2)  How is a repeated measures design different from a totally between subjects design in the collection of the data?  How does the significance testing change from the totally between to a design to one in which one or more factors are repeated measures (just the general idea, you don’t need to show actual F ratios or computations)?  Describe one argument for using repeated measures designs and one argument against using such designs (or describe when you would and would not want to use repeated measures).

Fixed Effects Designs All treatment conditions of interest are included in the study All in cell get identical stimulus (treatment, IV combination) Interest is in specific means Expected mean squares are (relatively) simple; F tests are all based on common error term.

Random Effects Designs Treatment conditions are sampled; not all conditions of interest are included. Replications of the experiment would get different treatments Interest in the variance produced by an IV rather than means Expected mean squares relatively complex; the denominator for F changes depending on the effect being tested.

Fixed vs. Random RandomFixed ExamplesConditions Examples Persuasiveness of commercials Treatment Sampled All of interestSex of participant Experimenter effect Replication different Replication sameDrug dosage Impact of team members Variance due to IV Means due to IVTraining program effectiveness

Single Factor Random The expected mean squares and F-test for the single random factor are the same as those for the single factor fixed-effects design.

Experimenter effects (Hays Table )

Sum of Source DF Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

Random Effects Significance Tests (A & B random/within) SourceE(MS)Fdf AJ-1, (J-1)(K-1) BK-1, (J-1)(K-1) AxB(J-1)(K-1), JK(n-1) Error

Why the Funky MS? Treatment effects for A, B, & AxB are the same for fixed & random in the population of treatments. In fixed, we have the population, in random, we just have a sample. Therefore, in a given (random) study, the interaction effects need not sum to zero. The AxB effects appear in the main effects.

Applications of Random Effects Reliability and Generalizability –How many judges do I need to get a reliability of.8? –How well does this score generalize to a particular universe of scores? –Intraclass correlations (ICCs) Estimated variance components –Meta-analysis Control (Randomized Blocks and Repeated Measures)

Review  What is the difference between fixed- and random-effects in terms of treatments?  How are F tests with random effects different than with fixed effects?

Randomized Blocks Designs A block is a matched group of participants who are similar or identical on a nuisance variable Suppose we want to study effect of a workbook on scores on a test in research methods. A major source of nuisance variance is cognitive ability We can block students on cognitive ability.

Randomized Blocks (2) Say 3 blocks (slow, average, fast learners) Within each block, randomly assign to workbook or control. Resulting design looks like ordinary factorial (3x2), but people are not assigned to blocks. The block factor is sampled, i.e., random. The F test for workbook is more powerful because we subtract nuisance variance. Unless blocks are truly categorical, a better design is analysis of covariance, described after we introduce regression.

Randomized Blocks (3) SourceE(MS)Fdf A workbook (fixed) J-1, (J-1)(K-1) B learner (random) If desired, use MSe AxB(J-1)(K-1), JK(n-1) ErrorLook up designs

Review  Describe a concrete example of a randomized block design. You should have 1 factor as the blocking factor and one other factor as the factor of main interest.  Describe a study in which Depression is a blocking factor.

Repeated Measures Designs In a repeated measures design, participants appear in more than one cell. –Painfree study –Sports instruction Commonly used in psychology

Pros & Cons of RM ProCon Individuals serve as own control – improved power Carry over effects May be cheaper to runParticipant sees design - demand characteristics Scarce participants

RM – Participant ‘Factor’ SourcedfMSE(MS)F Between Subjects K-1No test Within Subjects TreatmentsJ-1 Subjects x Treatments (J-1)(K-1)No test TotalJK-1

Drugs on Reaction Time Order of drug random. All Ss, all drugs. Interest is drug. PersonDrug 1Drug 2Drug 3 Drug 4Mean Mean Drug is fixed; person is random. ‘1 Factor’ repeated measures design. Notice 1 person per cell. We can get 3 SS: row, column, and residual (interaction plus error).

Total SS PersonDrug 1Drug 2Drug 3 Drug 4Mean Mean

Drug SS PersonDrugMD*DPersonDrugMD*D Total698.20

Person SS PersonDrugMD*DPersonDrugMD*D Total680.8

Summary Total = ; Drugs = 698.2, People= Residual = Total –(Drugs+People) = ( ) =112.8 SourceSSdfMSF Between People680.84Nuisance variance Within people (by summing) Drugs Residual Total Fcrit(.05)=3.95

SAS Run the same problem using SAS.

2 Factor, 1 Repeated SubjectB1B2B3B4M A A M DV=errors in control setting dials; IV(A) is dial calibration - between; IV(B) is dial shape - within. Observation is randomized over dial shape.

data d1; input i1-i4; cards; data d2; set d1; array z i1-i4; do over z; if _N_ le 3 then a =1; if _N_ gt 3 then a =2; sub = _N_; b = _I_; y=z; output; end; proc print; proc glm; class a b sub; model y = a b a*b sub(a) sub*b; test h=a e=sub(a)/htype=1 etype=1; test h=b a*b e=sub*b/htype=1 etype=1; run;

Summary SourceSSdfMSF Between people A(calibration) Subjects within groups Within people B (dial shape) AB BxSub within group Note that different factors are tested with different error terms.

SAS & Post Hoc Tests Run the same problem using SAS. The SAS default is to use what ever is residual as denominator of F test. You can use this to your advantage or else over-ride it to produce specific F tests of your desire. If you use the default error term, be sure you know what it is. Post hoc tests with repeated measures are tricky. You have to use the proper error term for each test. The error term changes depending on what you are testing. Be sure to look up the right error term.

Assumptions of RM Orthogonal ANOVA assumes homogeneity of error variance within cells. IVs are independent. With repeated measures, we introduce covariance (correlation) across cells. For example, the correlation of scores across subjects 1-3 for the first two calibrations is.89. Repeated measures designs make assumptions about the homogeneity of covariance matrices across conditions for the F test to work properly. If the assumptions are not met, you have problems and may need to make adjustments. You can avoid these assumptions by using multivariate techniques (MANOVA) to analyze your data. I suggest you do so. If you use ANOVA, you need to look up your design to get the right F tests and check on the assumptions.

Review  How is a repeated measures design different from a totally between subjects design in the collection of the data?  How does the significance testing change from the totally between to a design to one in which one or more factors are repeated measures (just the general idea, you don’t need to show actual F ratios or computations)?  Describe one argument for using repeated measures designs and one argument against using such designs (or describe when you would and would not want to use repeated measures).