1. EVAAS Overview

3. SAS EVAAS Analyses What is EVAAS? LOOKING BACK
Writing
ACT
End of Course
End of Grade
LOOKING AHEAD
Planning for Students’ Needs:
Student Projections to Future Tests
LOOKING BACK
Evaluating Schooling Effectiveness:
Value Added & Diagnostic Reports
Copyright © 2010, SAS Institute Inc. All rights reserved.

4. How can EVAAS help me? Improve the Education Program
EVAAS: Looking Back
Past Program Effectiveness
Local Knowledge & Expertise
EVAAS: Looking Ahead
Incoming Student Needs
EVAAS helps by allowing educators to:
Analyze past program performance for trends
Make informed projections for current and incoming students

5. Education Value Added Assessment System
Answers the question of how effective a schooling experience is
Produces reports that
Predict student success
Show the effects of schooling at particular schools
Reveal patterns in subgroup performance
EVAAS extracts data AFTER DPI collects data through the secure shell. DPI runs processes and checks for validity. Once DPI has completed their processes with the data, they present to the SBE. At this point, data is sent to EVAAS.

6. Changes in Reporting for 2012-13
Above
Exceeds Expected Growth
Not Detectably Different
Meets Expected Growth
Changes in reporting for
Color coding and descriptors
Above (Green) – students in the district made significantly more progress in this subject than students in the average district in NC. Progress was at least two standard errors above average.
NDD (Yellow) – Not Detectably Different from students in the average district. Less than two standard errors above average and no more than two standard errors below it.
Below (Light Red) – students in the district made significantly less progress in this subject than students in the average district in NC. Progress was more than two standard errors below average.
Color coding and descriptors
Exceeds Expected Growth (Blue): Estimated mean NCE gain is above the growth standard by at least 2 standard errors.
Meets Expected Growth (Green): Estimated mean NCE gain is below the growth standard by at most 2 standard errors but less than 2 standard error above it.
Does Not Meet Expected Growth (Red): Estimated mean NCE gain is below the growth standard by more than 2 standard errors.
The descriptors in EVAAS now match the Standard 6 Ratings.
Below
Does Not Meet Expected Growth

7. District Value Added Report
Use to evaluate the overall effectiveness of a district on student progress
Compares each district to the average district in the state for each subject tested in the given year
Indicates how a district influences student progress in the tested subjects
We will look at three kinds of reports - value-added, diagnostic, performance diagnostic - at the district level and review how to read them b/c this is the same way you will read your school data. The reports have elements in common and once you can interpret the district reports, you’ll be able to read your school reports easily.

8. Value-Added Reporting
Use this report to evaluate the overall effectiveness of a school on student progress. The School Value Added Report compares each school to the average school in the state. Comparisons are made for each subject tested in the given year and indicate how a school influences student progress in those subjects.
* Facilitator preference for the next few slides until break – use only power point or use live site to model – more questions pop up when using the live site

9. Value-Added Reporting
Scores from the EOG tests are converted to State NCEs (Normal Curve Equivalent scores) for the purpose of these analyses. NCE scores have the advantage of being on an equal-interval scale, which allows for a comparison of students' academic attainment level across grades. NCE scores remain the same from year to year for students who make exactly one year of progress after one year of instruction, even though their raw scores would be different. Their NCE gain would be zero.

10. Value-Added Reporting
The NCE Base is by definition set at 50.0, and it represents the average attainment level of students in the grade and subject, statewide.
If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state.
Student achievement levels appear at the bottom of the report in the Estimated School Mean NCE Scores section. The NCE Base is by definition set at 50.0, and it represents the average attainment level of students in the grade and subject, statewide. Compare the estimated grade/year mean for a school to the NCE Base. If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state.

11. District Diagnostic Reports
Use to identify patterns or trends of progress among students expected to score at different achievement levels
Caution: subgroup means come from “a liberal statistical process” that is “less conservative than estimates of a district’s influence on student progress in the District Value Added Report”

12. Diagnostic Report
Use this report to identify patterns or trends of progress among students expected to score at different achievement levels. This report is intended for diagnostic purposes only and should not be used for accountability.
Explain that on this report the students are grouped into quintiles. Students are assigned to groups on a statewide basis. The assignment pattern shows schools how their students are distributed compared to other students in the same grade across the state.
on the performance as it compares to similar students through out the state.

13. District Performance Diagnostic Reports
Use to identify patterns or trends or progress among students predicted to score at different performance levels as determined by their scores on NC tests
Students assigned to Projected Performance Levels based on their predicted scores
Shows the number (Nr) and percentage of students in the district that fall into each Projected Performance Level
Click on the underlined number in the Mean or Nr of Students row for a subgroup to see the names of the students assigned to the subgroup
Click on the % of Students for the current year or for Previous Cohort(s) to see the data in Pie Chart format.
Mean Differences The Mean of the difference between the students’ observed test performance and their predicted performance appears for each Projected Performance Level, along with the Standard Error associated with the Mean. The Standard Error allows you to establish a confidence band around the Mean. A large negative mean indicates that students within a group made less progress than expected.
A large positive mean indicates that students within a group made more progress than expected.
A mean of approximately 0.0 indicates that a group has progressed at an average rate in the given subject.
When the means among groups vary markedly, districts may want to explore ways to improve the instruction for students making less progress.

14. District Performance Diagnostic Reports
The Reference Line in the table indicates the gain necessary for students in each Prior-Achievement Subgroup to make expected progress, and it reflects the growth standard. When Gain is reported in NCEs, as it is here, the growth standard is 0.0.
The Gain is a measure of the relative progress of the school's students in each Prior-Achievement Subgroup compared to the Growth Standard
Standard errors appear beneath the Gain for each Prior-Achievement Subgroup. The standard error allows the user to establish a confidence band around the estimate. The smaller the number of students, the larger the standard error.
A student becomes a member of a Prior-Achievement Subgroup based on the average of his or her current and previous year NCE scores. A single student score contains measurement error. Using the average of two years allows a more appropriate assignment.
The ro. of Students row shows the number of students in a subgroup. Some subgroups may contain more students than others because students are assigned to groups on a statewide basis. The assignment pattern shows schools how their students are distributed compared to other students in the same grade across the state.

15. Interpreting the Pie Chart
Green
Yellow
Light Red
The Pie Chart shows the percent of students in each subgroup and compares their progress to the Growth Standard.
Yellow: students in this group progressed at a rate similar to that of students in the average district in the state.
Light Red: students in the group made more than one standard error less progress in this subject than students in the average district in the state.
Green: the progress of students in this group was more than one standard error above that of students in the average district in the state.