Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Repeated-measures data in educational research trials – how should it be analysed? Ben Styles Senior Statistician National Foundation for Educational.

Similar presentations


Presentation on theme: "1 Repeated-measures data in educational research trials – how should it be analysed? Ben Styles Senior Statistician National Foundation for Educational."— Presentation transcript:

1 1 Repeated-measures data in educational research trials – how should it be analysed? Ben Styles Senior Statistician National Foundation for Educational Research

2 2

3 3 Two sweeps example Cluster randomised trial of reading materials Baseline reading test, 10 week intervention, follow-up reading test Two parallel versions of the Suffolk Reading Scale

4 4 Using baseline data as a covariate in a multi- level (pupil, school) regression model Different analysis, different results OutcomeBackgroundCoefficientSEp Post-test scoreConstant *** Intervention NS Pre-test score ***

5 Different analysis, different results Using time as a level in a repeated measures multi-level (time, pupil, school) regression model 5 OutcomeBackgroundCoefficientSEp Total scoreConstant *** Time *** Intervention NS Time*intervention *

6 Interaction 6

7 Six sweeps example Mentoring scheme for struggling readers Pupil-level randomisation Questionnaire administered once at baseline and then every four months for the next two years 7

8 Six sweeps example Using time as a level in a repeated-measures multi-level (time, pupil, school) regression model 8 OutcomeBackgroundCoefficientSEp Aspirations for the futureConstant *** Time ** Intervention NS Time*intervention *

9 Reading Two-waves studies cannot describe individual trajectories of change and they confound true change with measurement error (Singer and Willett, 2002) ANCOVA is valid even with pre-test measurement error (Senn, 2004) Unconditional change models described in text books have three or more time-points The ANCOVA will almost always provide a more powerful test of the hypothesis of interest than will the repeated measures ANOVA approach (Dugard and Todman, 1995) 9

10 Change model assumption violation (2 sweeps) 10

11 Change model assumption OK (six sweeps) 11 Correlations r1r2r3r4r5r6 r1Pearson Correlation Sig. (2-tailed) N r2Pearson Correlation Sig. (2-tailed) N r3Pearson Correlation Sig. (2-tailed) N r4Pearson Correlation Sig. (2-tailed) N r5Pearson Correlation Sig. (2-tailed) N r6Pearson Correlation Sig. (2-tailed) N

12 Measurement error problematic 12

13 Measurement error problematic 13

14 Measurement error not a problem 14

15 A better repeated measures model 15

16 A (slightly) better conditional model 16

17 Conclusion No consensus but it is probably safer to use a conditional model for a pre-test post-test design Designs with three or more sweeps will benefit from a repeated measures multi- level model Care with level 1 residual autocorrelation Try a few models and check assumptions Don’t get hung up on significance 17

18 Questions and advice 18

19 Acknowlegements Pearson Business in the Community and Queen’s University, Belfast Tom Benton Dougal Hutchison NFER 19


Download ppt "1 Repeated-measures data in educational research trials – how should it be analysed? Ben Styles Senior Statistician National Foundation for Educational."

Similar presentations


Ads by Google