Using EVAAS to Improve Student Performance Donna Albaugh Rachel McBroom Heather Stewart Region 4 PD Leads NCDPI

Outcomes Overview of EVAAS Growth Model Interpreting Value-Added Reports Interpreting Diagnostic Report Helping teachers use their individual data Note: Slides marked with * are from SAS.

Underlying EVAAS Philosophy All students deserve opportunities to make appropriate academic progress every year. There is no “one size fits all” way of educating students who enter a class at different levels of academic achievement. Adjustments to instruction should be based on the academic attainment of students, not on socio-economic factors. Given reliable information on past effectiveness, educators can make appropriate adjustments to improve student opportunities. "What teachers know and can do is the most important influence on what students learn." (National Commission on Teaching and America's Future, 1996) One of the most important things educators can know is with whom they are effective and where they need to develop new skills.. * SAS

Achievement and Poverty * SAS

Academic Growth and Poverty No one is doomed to failure. * SAS

Benefits for Principals Gain a consolidated view of student progress and teacher effectiveness, as well as the impact of instruction and performance. Bring clarity to strategic planning and function as a catalyst for conversations that must take place to ensure that all students reach their potential. Understand and leverage the strengths of effective teachers. Use the valuable resource of effective teaching to benefit as many students as possible.

Changes in Reporting Above Not Detectably Different Below Exceeds Expected Growth Meets Expected Growth Does Not Meet Expected Growth

Value-Added Reporting Predictions Projections

North Carolina uses two different models in EVAAS 1.The Univariate Response Model (URM) is used for EOGs in grades 5 and 8 science, EOCs in high school, and CTE Post-Assessments; URM will be used for the Common Exams (predicts) 2.The Multivariate Response Model (MRM) is used for EOGs in grades 3 – 8 mathematics and English Language Arts (projects)

Predict/Project The URM uses a student’s prior test scores to predict where a student will be positioned in the statewide distribution of students who take a certain assessment The MRM uses a student’s prior test scores to project where students will be positioned in the statewide distribution of students who take a certain assessment In both cases, the basic methodology is the same: using a student’s prior test scores to consider their performance on an assessment he or she will take at the end of the school year

Predictive VA Model – Univariate Response Model (Predicts) For reports where testing is not sequential (gr. 5 and 8 science, EOCs in high school, CTE post-assessments) Common Exams Students must have three previous test scores for a predictive model Effect is difference between predicted and observed scores

Value-Added Reports – Multivariate Model (Projections)

Value-Added Reporting

The NCE Base is by definition set at 50.0, and it represents the average attainment level of students in the grade and subject, statewide. If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state.

District Diagnostic Reports Use to identify patterns or trends of progress among students expected to score at different achievement levels

Diagnostic Report

Diagnostic Reports – the whiskers

Diagnostic Reports Looking for Patterns

Student Pattern Report

Student Pattern Report – Key Questions Different experience? Different strategies? Different needs? Number of hours?

Student Pattern Report – Key Questions Different experience? Different strategies? Different needs? Number of hours? Rerun the report with new criteria. YES!

Student Pattern Report – Next Steps 16 Students who attended for 40+ hours All 31 Students in the Program

Less Informed Conclusion: We need to change the selection criteria for this program. More Informed Conclusion: We need to adjust the recommended hours for participants.

Custom Student Report HANDOUT

Using Data to Drive Crucial Conversations What’s a Crucial Conversation? And Who Cares? A crucial conversation is a discussion between two or more people where stakes are high, opinions vary, and emotions run strong. When we face crucial conversations, we can do one of three things: We can avoid them, we can face them and handle them poorly, or we can face them and handle them well. Ironically, the more crucial the conversation, the less likely we are to handle it well.

Frequently Asked Questions How is the school composite determined? In accordance with current State Board of Education policy, the school composite only includes assessments that are part of the State Testing Program. Included: EOCs and EOGs Not Included: CTE Post-Assessments and Common Exams Many educators feel strongly that the CTE Post-Assessments and Common Exams should be included – the State Board of Education will consider this recommendation

Frequently Asked Questions How will the Common Exams be entered into EVAAS? The data will be processed and transferred in the same way as data from the State Testing Program: districts scan their answer sheets; DPI does quality checks and audits; DPI moves the data to SAS through a secure FTP A major difference for all assessments (EOCs, EOGs, CTE Post-Assessments, and Common Exams) is that student- teacher links will come from the EVAAS roster verification process, not files pulled from NCWISE

Frequently Asked Questions Why is there no growth data for individual students? All growth models depend on having a large enough sample of student test scores to produce valid and reliable measures of growth The ABC Growth Model is not valid or reliable when used to measure growth at the student-level; the error around any estimate is larger than the entire scale for an assessment EVAAS will only produce growth values in which we can all have confidence, hence, there will be no growth values for individual students

Frequently Asked Questions How can we have growth measures for assessments that have never been given before? Even if an assessment has never been given before, a first- time administration will produce a mean score, and 50% of students will do better than the mean and 50% of students will do worse than the mean Using percentiles and NCEs, EVAAS is always considering a student’s position in the statewide distribution, not the scale or raw score on the assessment

Frequently Asked Questions How can we have growth measures for assessments that have never been given before? For assessments that are completely new (like the Common Exams), SAS performs an analysis of data from first semester to determine the prediction model. After the second semester administration, all of the data are used in the model to produce value-added scores

Frequently Asked Questions Why are there large numbers of teachers “not meeting expected growth” in some content areas and not others? There are larger numbers of students taking some state assessments compared to others. Additionally, evidence from across the country shows that ELA teachers are less likely to have either an extremely positive or extremely negative impact than teachers of other content areas The distributions currently in EVAAS are based on one-year of data; they tend to normalize as we add more data to the system

Questions?