Presentation on theme: "NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI Jill Pentimonti Adrea Truckenmiller Jessica Logan Sara Hart Discussion:"— Presentation transcript:
NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI Jill Pentimonti Adrea Truckenmiller Jessica Logan Sara Hart Discussion: Grandpa Schatschneider Presented Feb 6, 2014 Pacific Coast Research Conference, San Diego
Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart & Grandpa Florida State University
Expanding our search for moderators of intervention A little about me – Behavioral genetics background – PCRC participant Even with modest effect sizes, individual differences in intervention response Bioecological model (Bronfenbrenner & Ceci, 1994) – Provides framework for differentiating students based on non-intervention related traits
Integrative Data Analysis (IDA) Item-level pooled data (Curran & Hussong, 2009) Capitalizes on cumulative knowledge – Longer developmental time span – Increased statistical power – Increased absolute numbers in tails Controls for heterogeneity – Sampling, age/grade, cohort, geographical, design, measurement
Project KIDS Expanded definition of moderators of response to intervention – Cognitive, psychosocial, environmental, genetic risk IDA across 9 completed intervention projects – Approximately 5600 kids Data entry of item level data common across at least 2 projects – ~30 different assessments Questionnaire data collection
Proof of Concept Behavior problems and achievement are associated More behavior problems are typically seen in LD populations Is adequate vs inadequate response status differentiated by behavior problems?
Method Participants – 2005-2006 ISI intervention project (Connor et al., 2007) RCTish : 22 treatment, 25 contrast teachers, 3 pilot 821 first graders A2i recommendations vs standard practice – 2007-2008 ISI intervention through FL LDRC (Al Otaiba et al., 2011) RCT: 23 treatment, 21 contrast teachers 556 kindergarteners A2i recommendations vs enhanced standard practice
Method Measures – WJ Tests of Achievement Letter-Word Identification (LWID) Pre- and post-intervention testing periods – Social Skills Rating Scale: Behavioral Problems subscale Teacher completed during intervention year 05/06 1 ISI Mean (SD) 07/08 K ISI LDRC Mean (SD) WJ LWID Fall24.28 (7.97)11.96 (5.53) WJ LWID Spring36.54 (7.39)21.64 (7.10) SSRS.53 (.44).48 (.44)
Results: Calibration LWID Randomly selected 1 time point/child/project to form “calibration sample” for LWID IRT with decision to include only items > 5% endorsement rate Reduced item sample from 75 36 – Items 8 to 44
Results: Response 648 treatment children Unresponsive Cut off Fall SS = 95 Spring SS= 104 Mean Fall SS = 86 Spring SS = 96
Results: Response 648 treatment children Unresponsive Cut off Fall SS = 95 Spring SS= 104 Mean Fall SS = 86 Spring SS = 96 Responsive Mean Fall SS = 99 Spring SS = 111
Results Logistic regression – SSRS behavior problems significant predictor of response status (OR = 1.45, CI = 1.12-1.88) average behavior problems = 19% probability of being “unresponsive” greater than average behavior problems(+ 1SD) = 29% probability of being “unresponsive” Less than average behavior problems (-1SD) = 12% probability of being “unresponsive”
Conclusions Response status is differentiated by behavior problems – Mo’ behavior problems, mo’ (reading) problems! The questionnaire data we will be adding will be real test of bioecological model on response to intervention
Overall IDA conclusions IDA is a “cheap” way to get more power, more n at tails, and show more generalizable effects Given how similar many of our projects are, consider doing item-level data entry – Easy potential to combine data – Can you do factor analysis and IRT? You can do IDA! These data are more useful together than apart – IRT within and between samples? – Treatment effectiveness across samples? – Characteristics of lowest responders?
Acknowledgements Stephanie Al Otaiba Carol Connor Chris Schatschneider Great staff & grad students, and a small army of data enterers NICHD grant HD072286