EVAAS Overview

SAS EVAAS Analyses What is EVAAS? LOOKING BACK Writing ACT End of Course End of Grade LOOKING AHEAD Planning for Students’ Needs: Student Projections to Future Tests LOOKING BACK Evaluating Schooling Effectiveness: Value Added & Diagnostic Reports Copyright © 2010, SAS Institute Inc. All rights reserved.

How can EVAAS help me? Improve the Education Program EVAAS: Looking Back Past Program Effectiveness Local Knowledge & Expertise EVAAS: Looking Ahead Incoming Student Needs EVAAS helps by allowing educators to: Analyze past program performance for trends Make informed projections for current and incoming students

Education Value Added Assessment System Answers the question of how effective a schooling experience is Produces reports that Predict student success Show the effects of schooling at particular schools Reveal patterns in subgroup performance EVAAS extracts data AFTER DPI collects data through the secure shell. DPI runs processes and checks for validity. Once DPI has completed their processes with the data, they present to the SBE. At this point, data is sent to EVAAS.

Changes in Reporting for 2012-13 2011-12 2012-13 Above Exceeds Expected Growth Not Detectably Different Meets Expected Growth Changes in reporting for 2012-13 2011-12 Color coding and descriptors Above (Green) – students in the district made significantly more progress in this subject than students in the average district in NC. Progress was at least two standard errors above average. NDD (Yellow) – Not Detectably Different from students in the average district. Less than two standard errors above average and no more than two standard errors below it. Below (Light Red) – students in the district made significantly less progress in this subject than students in the average district in NC. Progress was more than two standard errors below average. 2012-13 Color coding and descriptors Exceeds Expected Growth (Blue): Estimated mean NCE gain is above the growth standard by at least 2 standard errors. Meets Expected Growth (Green): Estimated mean NCE gain is below the growth standard by at most 2 standard errors but less than 2 standard error above it. Does Not Meet Expected Growth (Red): Estimated mean NCE gain is below the growth standard by more than 2 standard errors. The descriptors in EVAAS now match the Standard 6 Ratings. Below Does Not Meet Expected Growth

District Value Added Report Use to evaluate the overall effectiveness of a district on student progress Compares each district to the average district in the state for each subject tested in the given year Indicates how a district influences student progress in the tested subjects We will look at three kinds of reports - value-added, diagnostic, performance diagnostic - at the district level and review how to read them b/c this is the same way you will read your school data. The reports have elements in common and once you can interpret the district reports, you’ll be able to read your school reports easily.

Value-Added Reporting Use this report to evaluate the overall effectiveness of a school on student progress. The School Value Added Report compares each school to the average school in the state. Comparisons are made for each subject tested in the given year and indicate how a school influences student progress in those subjects. * Facilitator preference for the next few slides until break – use only power point or use live site to model – more questions pop up when using the live site

Value-Added Reporting Scores from the EOG tests are converted to State NCEs (Normal Curve Equivalent scores) for the purpose of these analyses. NCE scores have the advantage of being on an equal-interval scale, which allows for a comparison of students' academic attainment level across grades. NCE scores remain the same from year to year for students who make exactly one year of progress after one year of instruction, even though their raw scores would be different. Their NCE gain would be zero.

Value-Added Reporting The NCE Base is by definition set at 50.0, and it represents the average attainment level of students in the grade and subject, statewide. If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state. Student achievement levels appear at the bottom of the report in the Estimated School Mean NCE Scores section. The NCE Base is by definition set at 50.0, and it represents the average attainment level of students in the grade and subject, statewide. Compare the estimated grade/year mean for a school to the NCE Base. If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state.

District Diagnostic Reports Use to identify patterns or trends of progress among students expected to score at different achievement levels Caution: subgroup means come from “a liberal statistical process” that is “less conservative than estimates of a district’s influence on student progress in the District Value Added Report”

Diagnostic Report Use this report to identify patterns or trends of progress among students expected to score at different achievement levels. This report is intended for diagnostic purposes only and should not be used for accountability. Explain that on this report the students are grouped into quintiles. Students are assigned to groups on a statewide basis. The assignment pattern shows schools how their students are distributed compared to other students in the same grade across the state. on the performance as it compares to similar students through out the state.

District Performance Diagnostic Reports Use to identify patterns or trends or progress among students predicted to score at different performance levels as determined by their scores on NC tests Students assigned to Projected Performance Levels based on their predicted scores Shows the number (Nr) and percentage of students in the district that fall into each Projected Performance Level Click on the underlined number in the Mean or Nr of Students row for a subgroup to see the names of the students assigned to the subgroup Click on the % of Students for the current year or for Previous Cohort(s) to see the data in Pie Chart format. Mean Differences The Mean of the difference between the students’ observed test performance and their predicted performance appears for each Projected Performance Level, along with the Standard Error associated with the Mean. The Standard Error allows you to establish a confidence band around the Mean. A large negative mean indicates that students within a group made less progress than expected. A large positive mean indicates that students within a group made more progress than expected. A mean of approximately 0.0 indicates that a group has progressed at an average rate in the given subject. When the means among groups vary markedly, districts may want to explore ways to improve the instruction for students making less progress.

District Performance Diagnostic Reports The Reference Line in the table indicates the gain necessary for students in each Prior-Achievement Subgroup to make expected progress, and it reflects the growth standard. When Gain is reported in NCEs, as it is here, the growth standard is 0.0. The Gain is a measure of the relative progress of the school's students in each Prior-Achievement Subgroup compared to the Growth Standard Standard errors appear beneath the Gain for each Prior-Achievement Subgroup. The standard error allows the user to establish a confidence band around the estimate. The smaller the number of students, the larger the standard error. A student becomes a member of a Prior-Achievement Subgroup based on the average of his or her current and previous year NCE scores. A single student score contains measurement error. Using the average of two years allows a more appropriate assignment. The ro. of Students row shows the number of students in a subgroup. Some subgroups may contain more students than others because students are assigned to groups on a statewide basis. The assignment pattern shows schools how their students are distributed compared to other students in the same grade across the state.

Interpreting the Pie Chart Green Yellow Light Red The Pie Chart shows the percent of students in each subgroup and compares their progress to the Growth Standard. Yellow: students in this group progressed at a rate similar to that of students in the average district in the state. Light Red: students in the group made more than one standard error less progress in this subject than students in the average district in the state. Green: the progress of students in this group was more than one standard error above that of students in the average district in the state.