1. Copyright © 2010, SAS Institute Inc. All rights reserved. EVAAS Concepts: NCEs and Standard Errors Sandy Horn January 2013 SAS ® EVAAS ® for K-12

2. 2 Copyright © 2010, SAS Institute Inc. All rights reserved. Similarities between the EVAAS Growth Standard Model and the Predictive Model For both models: Progress is measured relative to the average progress for a grade/subject in North Carolina in the year tested. Average Progress/Growth Standard is 0.0. Standard errors determine whether a district, school, or teacher’s growth measure is different from average. (More on this, shortly)

3. 3 Copyright © 2010, SAS Institute Inc. All rights reserved. Differences between the EVAAS Growth Standard Model and the Predictive Model Growth Standard Model  Only used with tests administered in sequential years.  Does not predict where students will score, so doesn’t need three prior test scores. Predictive Model  Can be used with tests, whether administered sequentially or not.  Requires at least three prior test scores to predict where students will score, relative to other NC students who take the same test. (Prior scores don’t have to be in the subject being analyzed.)

4. 4 Copyright © 2010, SAS Institute Inc. All rights reserved. Differences between the EVAAS Growth Standard Model and the Predictive Model Growth Standard Model  Measures the difference between a cohort’s position in the state distribution in a grade/subject at the end of one year and their position in that distribution at the end of the next year.  Reported in NCEs. Predictive Model  Measures the difference between where students would be expected to score, assuming the average progress, statewide, and where they did score, when tested.  Reported in Scale Score Points.

5. 5 Copyright © 2010, SAS Institute Inc. All rights reserved. Growth Standard Value Added Report

6. 6 Copyright © 2010, SAS Institute Inc. All rights reserved. Normal Curve Equivalent Units - NCEs

7. 7 Copyright © 2010, SAS Institute Inc. All rights reserved. NCEs and the Growth Standard 30 40 50 60 70 5 th Grade: NCE 37 6 th Grade: NCE 37 The State Growth Standard (0.0) is achieved when students do not lose ground from year to year, relative to other students, across the state, who take the same test. It signifies one year’s growth. 6 th grade NCE 37 ― 5 th grade NCE 37 = 0 = State Growth Standard 30 40 50 60 70

8. 8 Copyright © 2010, SAS Institute Inc. All rights reserved. Growth Standard Value Added Report

9. 9 Copyright © 2010, SAS Institute Inc. All rights reserved. What are Standard Errors, and Why aren’t they all the same? Standard Errors let us know how confident we can be that a sample approximates a population. In other words, how sure can we be that our students (a sample) made progress similar to or different from the progress that is average for all the students in North Carolina (the population)? If we have a lot of students and/or a lot of data points (test scores) and/or somewhat complete data for our sample and so on, we can be pretty precise in estimating the progress of that group of students. The standard error would be small. If we have few students and/or few data points and/or very spotty data, we can’t be as certain of the estimated progress of that group of students. The Standard Error would be large.

10. 10 Copyright © 2010, SAS Institute Inc. All rights reserved. Standard Errors are Protection against Misclassification Small negatives and small positives may not be significantly different from Average. The standard error lets you know whether the difference is significant or not. Unless there is strong evidence to the contrary, standard errors ensure that an estimate is considered “Average.” They protect districts, schools, and teachers from being misclassified as different from Average unless there is a significant difference.

11. 11 Copyright © 2010, SAS Institute Inc. All rights reserved. About those Standard Errors… How much confidence do we have that the estimate is different from 0.0? 0.0 Growth Standard State Average -3se -2se -1se +1se +2se +3se 80% 90% 95%+ 99% Meets Expected Growth Exceeds Expected Growth Does Not Meet Expected Growth

12. 12 Copyright © 2010, SAS Institute Inc. All rights reserved. About those Standard Errors… How much confidence do we have that the estimate is different from 0.0? 6 th grade NCE 38.8 ― 5 th grade NCE 37 = 1.8 NCEs Is it different from the State Growth Standard? 30 40 50 60 70 5 th Grade: NCE 37 6 th Grade: NCE 38.8 30 40 50 60 70 +1.8 NCEs above 0.0 Growth Measure: 1.8 NCEs/points 0.0

13. 13 Copyright © 2010, SAS Institute Inc. All rights reserved. About those Standard Errors… How much confidence do we have that the estimate is different from 0.0? +1.8 NCEs above 0.0 School 1: Growth Measure: 1.8 NCEs SE: 1.1 NCEs +/- 2 SEs = 2.2 NCEs +2.2 NCEs -2.2 NCEs Growth Std/State Avg (0.0) School 1 Meets Expected Growth because its Growth Measure is less than 2 SEs above or below the Growth Standard. School 2: Growth Measure: 1.8 NCEs SE: 0.8 NCEs +/- 2 SEs = 1.6 NCEs +1.6 NCEs -1.6 NCEs School 2 Exceeds Expected Growth because its Growth Measure is 2 SEs or more above the Growth Standard. It works just the same for the Predictive Model. Just change NCE to Scale Score and Growth Standard to State Average.

14. 14 Copyright © 2010, SAS Institute Inc. All rights reserved. Growth Standard Value Added Report