Presentation on theme: "The Examination of Equivalence and Equating First-Grade DIBELS ORF Chung-Hau Fan The University of Iowa."— Presentation transcript:
The Examination of Equivalence and Equating First-Grade DIBELS ORF Chung-Hau Fan The University of Iowa
Research Purposes 1.Examine DIBELS 1 st grade ORF probes equivalence 2.Establish equivalent scaling for raw scores to facilitate comparison of non-equivalent passages.
Sample & Procedure N = 49, first graders from two Midwestern schools. All first graders within school invited; no selection criteria other than consent. 20 progress monitoring passages were given in a random order across 4 days at the end of the school year.
Data Analysis (1) Confirmatory factor analysis (CFA, Bollen, 1989) was used for examining probe equivalence. The general congeneric model and the parallel measurements models were tested.
Data Analysis (2) Linear equating methods (Kolen & Brennan, 2004) were used to equate WCPM scores across passages. The CFA was re-run using the equated scores to evaluate the extent to which the transformation provided equivalent measurement. Data for two students were graphed for visual analysis.
Results (1) 1.The largest average difference between probes was found for probe #16 (70) and probe #18 (90); a difference of 20 WCPM. 2.The average standard error of measurement (SEM) of the 20 probes was 5.1, ranging from 4.5 to 5.8.
Results (2) Model fit indices for the measurement models on the raw and rescaled data Measurement Models Raw scoresRescaled CongenericParallelCongenericParallel χ2χ2 301.377431.410301.377354.182 df170208170208 p-value< 0.0001 CFI0.950.910.950.94 RMSEA0.130.150.130.12 The fit indices: Comparative Fit Index (CFI) and Root Mean Square Error of Approximation (RMSEA).
Results (3) The row scores column suggested the common factor model fit the data better (CFI=.95; RMSEA=.13) than the parallel model (CFI=.91; RMSEA=.15). After equating, all the probes shared the same mean (83.7) and SD (31.4). The re-analysis of the parallelism indicated better fit (CFI increased to.94). Criteria for an excellent model fit were CFI.95 and RMSEA.06, while an acceptable model fit was defined as CFI.90 and RMSEA.08 (Hu & Bentler, 1999).
Results (4) The χ2 difference test result indicated a non-significant finding at the α = 0.05 level: Δχ2 (38) = 52.81, p =.06, suggesting there was no significant difference between the two models fit to the rescaled data.
Conclusions 1.The equivalence assumption was somewhat supported, depending on how strict the cutoff criteria were set (excellent or acceptable). 2.No finding of the significant reduction of fit from the congeneric to the parallel model on the row data found in Bett et al. (2009) with first-grade CBM-R materials. 3.The linear equating procedure appeared to have made a contribution towards making the set of passages equivalent even though it was not originally designed to reduce variability in scores.