1. Shudong Wang, NWEA Liru Zhang, Delaware DOE G. Gage Kingsbury, NWEA
Predicting Student Success in Grade 8 Using Longitudinal Data on Summative and Formative Assessments
Shudong Wang, NWEA
Liru Zhang, Delaware DOE
G. Gage Kingsbury, NWEA

2. Purpose of Study
The study is an investigation of using longitudinal data to estimate student success in reading and mathematics. Both summative and formative test scores are used to predict student performance at grade 8. The research questions are: 1. Is formative assessment a valid predictor to estimate student success on state summative assessment? 2. To what extent does the formative assessment contribute to estimate student success at grade 8? 3. Is there any different in patterns of prediction between reading and mathematics?

3. Evidence of Predictive and Concurrent Validity
Predictive validity is the extent to which a score on a scale or test predicts scores on some criterion measure
Concurrent validity is the degree to which results from a test agree with the results from other measures of the same or similar constructs.
Predictive validity shares similarities with concurrent validity in that both are generally measured as correlations between a test and some criterion measure.

4. Methods of Study – Data
The longitudinal data was collected from a state summative assessments and a district-wide, computerized adaptive formative assessment in reading and mathematics.
Students must have a valid score on the state assessment in at grade 6, 2007 at grade 7, and 2008 at grade 8;
Students must have a valid score on the formative assessment in the same year at the same grade; and
Students who were tested under non-standard accommodation(s) were excluded.

5. Methods of Study – Data (Continued)
The summative assessment data was collected from a statewide assessments. The variables are called
State 2008; State 2007; State 2006
The formative assessment data was collected from 5 large school districts of the state. The variables are called:
Formative 2008; Formative 2007; Formative 2006
The reading data set contains 1,170 students; the mathematics data set contains 1,454 students.

6. Methods of Study – Data Analysis
1. Multiple linear regression (MLR) was used for the analysis:
Dependent variable - State 2008
Two sets of independent variables:
- State summative assessments - State 2007, State District-wide formative assessments – Formative 2008, Formative 2007, Formative 2006
Stepwise method
2. MLR with selected independent variables from each set as well as combined variables from both sets

7. Descriptive Statistics
Test
Reading
Mathematics N
Mean SD
State 2006
1170
491.0
28.1
1422
503.7
37.7
State 2007
509.0
33.6
511.5
45.8
State 2008
533.1
31.1
528.3
45.0
Formative 2006
219.9
10.3
227.9
12.8
Formative 2007
222.5
10.2
233.4
14.9
Formative 2008
225.8
10.8
238.3
15.6

8. Correlation Matrix - Reading
Test
State 06
State 07
State 08
Formative 06
Formative 07
Formative 08
State 2006
1.000
.726
.688
.714
.660
.650
State 2007
.770
.711
.741
.722
State 2008
.677
.690
.693
.743
.725
.747

9. Correlation Matrix - Mathematics
State 06
State 07
State 08
Formative 06
Formative 07
Formative 08
State 2006
1.000
.869
.859
.856
.841
.815
State 2007
.882
.828
.873
.845
State 2008
.823
.858
.865
.848
.820

10. Multiple Linear Regression by Stepwise
Analysis One

11. Model Summary – Reading
Dependent variable – State 2008 reading score
Independent variables:
Model 1: State 2007
Model 2: State 2007, Formative 2008
Model 3: State 2007, Formative 2008, State 2006
Model 4: State 2007, Formative 2008, State Formative 2007
Model 5: State 2007, Formative 2008, State Formative 2007, Formative 2006

12. Model Summary – Reading
R Square
Adjusted R Square
Std. Error of the Estimate
Change Statistics
R Square Change
F Change
df1
df2
Sig. F Change 1
.770
.593
.592
19.873
1168
.000 2
.795
.632
.631
18.901
.039
1167 3
.807
.652
.651
18.394
.020
66.283
1166 4
.811
.658
.656
18.247
.006
19.814
1165 5
.812
.660
18.201
.002
6.920
1164
.009

13. Model Summary – Mathematics
Dependent variable – State 2008 mathematics score
Independent variables:
Model 1: State 2007
Model 2: State 2007, Formative 2008
Model 3: State 2007, Formative 2008, State 2006
Model 4: State 2007, Formative 2008, State Formative 2007

14. Model Summary – Mathematics
R Square
Adjusted R Square
Std. Error of the Estimate
Change Statistics
R Square Change
F Change
df1
df2
Sig. F Change 1
.883
.779
21.123
1420
.000 2
.911
.830
.829
18.567
.050
1419 3
.919
.845
17.712
.015
1418 4
.921
.849
.848
17.505
.004
34.793
1417

15. Coefficients – Reading (selected models)
Reading Model
Unstd. Coef.
Std. Coef. t
Sig. B SE
Beta 1
(Constant)
8.831
19.217
.000
State 07
.714
.017
.770
41.237 2
81.467
11.542
7.058
.522
.024
.563
21.934
Formative 08
.823
.074
.286
11.146 5
25.048
12.992
1.928
.054
.367
.028
.396
13.106
.458
.084
.159
5.477
State 06
.189
.030
.170
6.206
Formative 07
.326
.093
.107
3.520
Formative 06
.239
.091
.079
2.631
.009

16. Coefficients – Mathematics (selected models)
Math Model
Unstd. Coef.
Std. Coef. t
Sig. B SE
Beta 1
(Constant)
84.641
6.288
13.460
.000
State 07
.867
.012
.883
70.831 2
7.691
-3.228
.001
.519
.020
.528
25.767
Formative 08
1.207
.059
.420
20.467 4
8.248
-8.329
.301
.025
.306
11.950
.848
.063
.295
13.382
State 06
.277
.027
.232
10.264
Formative 07
.436
.074
.145
5.899

17. Coefficient Correlations – Reading
Model
Partial
Part 1
State 07
.770 3
.418
.271 2
.540
.389
Formative 08
.252
.154
.310
.198
State 06
.232
.141 5
.359
.224 4
.367
.231
.158
.094
.182
.108
.179
.106
.210
.126
Formative 07
.103
.060
.129
.076
Formative 06
.077
.045

18. Coefficient Correlations – Mathematics
Zero-Order
Partial
Part 1
State 07
.883 2
.565
.282
Formative 08
.866
.477
.224 3
.371
.157
.407
.175
State 06
.860
.301
.124 4
.303
.123
.335
.138
.263
.106
Formative 07
.859
.155
.061

19. Residual Summary – Reading
Minimum
Maximum
Mean SD
Predicted Value
1170
443.02
638.60
533.10
25.284
Residual
64.291
.000
18.162
Std. Predicted V.
-3.563
4.173
1.000
Std. Residual
-3.440
3.532
.998

20. Residual Summary – MathematicsN
Minimum
Maximum
Mean SD
Predicted Value
1422
418.11
676.03
528.27
41.420
Residual
93.682
.000
17.480
Std. Predicted V.
-2.660
3.567
1.000
Std. Residual
-3.372
5.352
.999

21. Multiple Linear Regression by Designed Entry
Analysis Two

22. Model Summary Reading Variables R Adj. R Adj R2 SEE F Sig.
State
0.793
0.628
0.627
19.001
0.000
Formative
0.733
0.537
0.536
21.210
State/Formative
06 and 07
0.807
0.651
0.650
18.426
Math Variables
0.902
0.814
19.386
0.877
0.769
21.501
0.912
0.832
0.831
18.465

23. Coefficients – Reading (1)
Reading Vars. B SE
Beta t
Sig
Partitial Corr.
(Constant)
9.926
11.563
0.000
State 2006
0.302
0.029
0.273
10.525
0.294
State 2007
0.53
0.024
0.572
22.023
0.393
4.839
14.383
0.336
0.737
Formative 06
1.110
0.090
0.366
12.298
0.339
Formative 07
1.277
0.091
0.418
14.061
0.381

24. Coefficients – Reading (2)
Reading Vars. B SE
Beta t
Sig
Partitial Corr
(Constant)
41.125
12.812
3.210
0.001
State 2006
0.202
0.031
0.182
6.571
0.000
0.189
State 2007
0.403
0.028
0.435
14.63
0.394
Formative 06
0.362
0.089
0.120
4.067
0.118
Formative 07
0.485
0.159
5.458
0.158

25. Coefficients –Mathematics (1)
Math Vars. B SE
Beta t
Sig
Partitial Corr
(Constant)
22.576
6.910
3.267
0.001
State 2006
0.451
0.028
0.378
16.335
0.000
0.396
State 2007
0.545
0.023
0.554
23.963
0.537
10.152
-14.45
Formative 06
1.198
0.084
0.340
14.319
0.352
Formative 07
1.721
0.072
0.570
23.969
0.533

26. Coefficients –Mathematics (2)
Math Vars. B SE
Beta t
Sig
Partitial Corr
(Constant)
9.751
-5.748
0.000
State 2006
0.292
0.030
0.245
9.597
0.247
State 2007
0.376
0.026
0.383
14.603
0.362
Formative 06
0.352
0.084
0.100
4.170
0.110
Formative 07
0.705
0.077
0.234
9.151
0.236

27. Summary of Findings
The results of the study indicate that student scores on the formative assessment can be a valid predictor, by itself or combined with student scores on the state summative assessment, to estimate student success at grade 8 in reading and mathematics.
The analysis results suggest that formative assessment scores make significant contribution to improve the accuracy of predicting student success in both reading and mathematics.
It is not surprise that the latest test score is often the strongest predictor in a longitudinal study. The data seem to imply that mathematics is more sensitive to the year of the assessment score in mathematics than in reading, which perhaps is due to the natural of the content area.
The limitations of using matched student records in longitudinal analyses must be taken into consideration in case of significant changes of student populations over time (e.g., mobility).