1. Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and Accountability West Virginia Department of Education

2. Assessment Continuum in WV Promoting a balanced assessment system against WV State Standards Formative Diagnostics Interim Diagnostics Interim Benchmarks Summative Assessments

3. Formative, Interim, and Summative Assessments Summative – WESTEST 2 – Online Writing Assessment (PT) Interim – Benchmarks in Math, ELA, Science, Social Studies Formative – Acuity Item Banks in Math, ELA, Science, Social Studies – West Virginia Writes: Practice Writing Program – Other District Purchased Supports – Teacher Developed Assessments

4. Major Question: Purpose? Purpose of Summative – Evaluative against a set of standards – Least flexible: long-cycle – Broad sense of the educational state of the state or district Purpose of Interim – Instructional, – Evaluative, and/or – Predictive – Medium-cycle Purpose of Formative – Process to adjust teaching and learning – Instructional diagnostics – Typically classroom specific – Very short-cycle

5. Examining the Use of Formative and Interim High Use vs. Effective Use – Emerging case study of a district and school Context – Targeted professional development – Dedicated school leadership and staff – State-level support around the use of formative and interim assessments Early Results:

6. Grade 9 School A and B

7. Grade 10 School A and B

8. Grade 11 School A and B

9. WESTEST 2 Writing: At or Above Proficiency

10. Examining the Use of Formative and Interim Strong support for use of writing short- cycle assessments Currently examining use of other short- cycle (formative) and medium-cycle (interim) assessments on summative assessment results Implications on teacher evaluation systems looking for triangulated evidence of student learning

11. Multiple Use Concerns Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments) Adequate Yearly Progress Improvement and Growth Teacher Evaluation Systems Student Learning Conflicting?

12. Multiple Use Concerns Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments) – Summative Assessment Gaming Cheating “Teaching to the Test” – Interim/Formative Assessment Fabricated student learning gains? Student sorting to highlight teacher “gains”? What does the research say? Not impact/effect, but valid link to teacher input/output? Cognitive vs. Non-Cognitive measures? Scratching the surface

13. The West Virginia Case A transition from nothing to something – Old: 5 years experience  evaluation “only when the principal determines it to be necessary…” – New: Yearly evaluation for all educators, principals, and evaluators using research- based professional standards and evidence of student learning

14. Developing the System Goal: To develop a supportive system around valid data to drive improvement Access: A variety of data to support self- reflective practice: – Summative Growth Data – Interim Benchmarks – Formative Diagnostic

15. Developing the System Context: – Politics: National vs. State Pressure – Policy: Accountability – Pragmatics: Using actionable and valid data Development: Stakeholder driven – Board of Education – SEA Staff – LEA staff – Teacher Unions – Legislators – Community Members

16. CategoryInputOutput 5 Professional Teaching Standards80% 14 Critical Standard Elements – 4 pt. rubric 5.7%/CSE Student Learning Outcomes20% Summative Growth Data Math & ELA Only 5% Other Student Learning Evidence Any Content15% Professionalism (Mandatory)N/A The System System Components

17. The System 20% Student Learning Outcomes – Minimum 5% Student Assessment Data School-wide Growth – Student Growth Percentiles – A host of evidence – Agreed upon with principal input – Multiple measures Acuity (Math, ELA, Science, Social Studies) WV Writes (Writing) Other evidence of student learning Teacher Developed Assessments

18. The System Overall Ratings Category100 Point Scale Distinguished88-100 Accomplished54-88 Emerging33-54 Unsatisfactory <33

19. Implementation 2011-2012 – Concluded 1 st pilot year in a small number of schools – Self-ratings – Accomplished across the board… 2012-2013 – 2 nd pilot year with schools in every district represented 2013-2014 – Full statewide implementation for all schools in the state

20. Implications for a “Young System” Embedded Program Evaluation – WVDE Office of Research small scale implementation study and program evaluation – Formative data has resulted in changes to the system and program – Exploring third-party consulting for enhancements to guidance, system, and process (post-revision)

21. Lessons Learned Politics vs. Pragmatism – Representative inclusion vs. Over/under- representation of certain views – Political support to push a bill to law Capacity and Foresight – In-house development and reinventing the wheel? (…more politics) – Building a system for the present vs. building a system for the future – Revising the re-invention

22. Thank You Juan M. D’Brot (jdbrot@access.k12.wv.us)jdbrot@access.k12.wv.us Executive Director Office of Assessment and Accountability West Virginia Department of Education