1. July 19-21, 2011 Stonewall Resort, Roanoke, West Virginia Educator Evaluation Pilot

2. Overview The “Big Picture” – Where are we headed? – Where have we been? – How will we get there? Two Day Agenda Next Steps

3. VISION: West Virginia will have a comprehensive and equitable evaluation system that clearly articulates, measures, rewards, and develops educator effectiveness

4. Where Have We Been?

5. Historical Perspective 2009 Standards Adopted 2011 Evaluation Task Force Worked 2012 Revised System Piloted

6. Importance of Leadership July 20-21 – Stonewall Resort ALL Pilot schools leadership team members – Principals and all assistant principals – Title I Directors and/or Assistant Superintendents – Teacher Leaders – Central office representative for non SIG pilot schools

7. Clear Expectations Embrace the Opportunities: Be “early adopter” Engage in professional development Work as part of a collaborative team Provide input to policy makers Participate in the research study Communicate, partner and problem-solve

8. Day 1 Agenda Expectations for Training History Evaluation Creation Conceptual Framework Overview Self-Assessment Observation Student “Growth”

9. Day 2 Agenda SMART Goals and Collaboration Student Learning Goals Professional Conduct Research Performance Assessment

10. What are the Next Steps?

11. Teacher Evaluation Training Dates August 6-12 – Regional Training Participants: all teachers, administrators and transformation specialists Fall and Spring Training On site visits Ongoing Technical Assistance

12. Historical Perspective 2009 Standards Adopted 2011 Evaluation Task Force Worked 2012 Revised System Piloted 2013 Begin Statewide Scale Up

13. Thank You!

14. Expectations for Professional Development

15. Learning Target – Understand how the new evaluation system works to be able to implement the processes yourself and to be able to assist others in school Strategies to accomplish the goal – Hands-on activities – Multiple opportunities to master content – Collaboration with tablemates

16. Expectations for Professional Development Expectations for us – Well prepared – Clarity – Consideration of the audience Expectations for you – Active participation – Regular feedback – Focused attention

17. Teacher Evaluation Creation Task Force Members’ Perspective

18. Evaluation Perceptions & Experiences

19. Table Tasks Describe most recent evaluation experience – What was the purpose? What feedback was provided? What was your role in the process? What are the challenges of teacher evaluation (not including time)? What will you need to learn to make this PD beneficial?

20. A New Evaluation System: The Rationale

21. A New Evaluation System: The Rationale Learning Target: Know – the three drivers behind a revised evaluation system for principals and teachers

22. Converging Forces

23. Conceptual Framework Overview

24. Learning Targets: Understand – the nexus between the WVPTS and the Critical Standard Elements; – how levels of teacher performance are defined with rubrics; – the tiered evaluation system with progressions based on experience

25. Levels of Performance DistinguishedAccomplishedEmergingUnsatisfactory

26. Table Tasks Share one activity with distinguished performance, one activity with unsatisfactory performance

27. Levels of Performance DistinguishedAccomplishedEmergingUnsatisfactory Distinguished performance describes professional teaching that engages students to be highly responsible for their own learning. Performing at this level involves contributing to the professional learning of others through teacher leadership. Accomplished performance describes professional teaching that exhibits mastery of the work of teaching while improving practice and serving the professional community. Emerging performance represents teaching that demonstrates knowledge and skills to implement essential elements albeit not always successfully at times. Unsatisfactory performance describes teaching that does not convey sufficient understanding of concepts or the successful implementation of essential elements.

28. Self Assessment – Advance Progression

29. Learning Targets: – Identify individual performance within the established rubrics

30. Table Tasks Share an insight from doing this activity about individual performance Share reaction to critical standard elements What implications does this have for individual professional development

31. Table Tasks Discuss perception of how comfortable teachers with 6+ years of experience will be completing self assessment Discuss how this process could be used in relation to the PD plan for of the school Discuss how this process could be used in collaborative teams.

32. Observation

33. Learning Targets: – Understand that observation is formative and offers a window into instructional performance; – Understand that observation, though limited, is informative about certain key aspects of instruction

34. Some specifics about observation Not the evaluation Initial Progress-4, Intermediate-2, Advanced if requested Class period or minimum of 30 mins One piece of a two part conversation Will be supported by evidence and conversation Elements that contribute to the research

35. Table Tasks What does observation tell you about the critical standard elements? What elements do you still need to know about to make a fair and accurate assessment about performance?

36. Evidence

37. Learning Targets: – Understand that the educator plays an active role in demonstrating performance level by providing evidence; – Identify and classify evidence

38. Table Tasks Share the kinds of evidence that were generated. Were any common among the table? Discuss whether you agree that evidence “presented” was acceptable. Even though none are required, which ones would be considered essential? Which ones should be brought to conference with principal?

39. Evidence For Self Assessment

40. Table Tasks Share the kinds of evidence that were generated. Were any common among the table? Does the evidence convincingly support the rating? Discuss whether you agree that evidence “presented” was acceptable. Even though none are required, which ones would be considered essential?

41. Student Growth 101 Juan D’Brot Executive Director Office of Assessment and Accountability

42. Student Growth 101 Learning Targets: – Understand how the school-wide growth measure is developed

43. The WV Growth Model: Changing Conversations about Education Juan D’Brot Executive Director Office of Assessment and Accountability

44. WV Growth Model 1.What questions? 2.Purpose 3.Status vs. growth 4.What student growth data looks like 5.What school growth data looks like

45. First… A challenge! Stop me for questions…

46. Asking the Right Questions… Until you’ve defined the question, you cannot examine the appropriate data. Why? – Different data answer different questions – Different questions lead to different conversations

47. Fact #1: …increased use of student assessment data as a “significant factor” in (insert favorite accountability topic here…) But what student data? That leads us to start asking questions… But what questions? Whose questions?

48. Whose Questions? Researchers Administrators Teachers Parents Policymakers Business Leaders Community Members Guardians WHY? To answer questions about the student How? By not declaring a verdict. Instead, by changing the conversation…

49. Components of the System should Align with the Purpose of the System The revised teacher evaluation system is intended to do what? – Change conversations – Provide support and direction – Identify areas of best practice and need The revised consideration of student learning (growth) is intended to do what? – Change conversations – Provide direction – Reframe evidence of student learning

50. Bigger Question: How do we align the revision of both systems? …Fact #2

51. Fact #2: Assessment Systems try to be Everything to Everyone Danger: answering too many questions with the same data – Differentiated Instruction – Student Proficiency – Class Performance – AYP Possible Solution: Can we come up with a common question across stakeholders?

52. The WV Growth Model: Our Purpose To provide an answer to a few common questions: 1.“How much did my student grow this year?” 2.“What does this growth mean compared to everyone else?” 3.“Is it enough growth?” What do these three questions have in common? Information about students What do these three questions have in common? Information about students

53. Status vs. Growth Before growth, let’s discuss status – Status: A snapshot measure of a single point in time – “Proficiency” – Growth: Multiple snapshots across many points in time

54. Status vs. Growth A Balanced Approach Low GrowthHigh Growth Quadrant 2 Is believed not to require improvement because declining student growth is not recognized under status model Quadrant 3 Does not require improvement and is a potential site of interest for best practices Quadrant 1 Legitimately requires improvement Quadrant 4 Is believed to require improvement because high rates of growth are not recognized under the current status model. May be a potential site of interest for best practices. High Performing (Status) Low Performing (Status)

55. Questions - Revisited 1.“How much did my student grow this year?” (Time 2 – Time 1) – We can see this today 2.“What does this growth mean compared to everyone else?” (Normative Component) 3.“Is it enough growth?” (Criterion Component - CSOs)

56. How much did my student grow this year? – 1 st Question » How much academic growth do individual students in WV exhibit? » Scale Scores – Tell very different stories » Time 2 (2010 Scale Scores – Mastery = 550) – Time 1 (2009 Scale Scores – Mastery = 500) 2009 Scale Score2010 Scale ScoreNet “Growth” 52557550 55057525 425575125 37540025

57. What does this growth mean compared to everyone else? – Pre-Growth Model » This year and last year (only 2 years worth of comparison) – Post-Growth Model » Contextual consideration of growth » Yields “Student Growth Percentiles” » Think height

58. What is a Student Growth Percentile? Given (3 things): – A student’s prior scale scores – Academically similar students – Current scale score A student’s current scale score represents a percentile of growth – a Student Growth Percentile Think of it as the probability of a student’s current achievement based on their past achievement: – Pr(Current Achievement|Past Achievement) Growth percentiles describe the probability/rarity of a student’s current achievement based upon their prior achievement.

59. Vertical Axis: Number of Students at a Particular Scale Score

63. What does this student’s growth mean compared to everyone else? – 2 nd Question » What does the academic growth of an individual student in WV really mean? » Scale Score Growth – Tells very different stories » Time 2 – Time 1: Based on other students with similar academic histories. 2009 Scale Score 2010 Scale Score Net “Growth”Growth Percentile 5255755040 th 5505752520 th 42557512599 th 3754002550 th

64. But is it enough growth? – 3 rd Question » Is the academic growth of an individual student in WV enough to make it to Mastery? What about staying at Mastery? » This year’s Growth Percentile isn’t enough » Where would a student be next year if s/he exhibited: High Growth Typical Growth Low Growth

65. Our Growth Options How much growth? – Starts with the student. Can be aggregated to drive school conversations – Very Low Growth - Unsatisfactory – Lower Growth - Emerging – Typical Growth - Accomplished – Higher Growth – Distinguished Is it enough growth? – Can drive student learning conversations – Catching Up – Keeping Up – Falling Behind

66. How much growth? Is it enough growth? Novice Partial Mastery Mastery Above Mastery NovicePartial Mastery Mastery

67. Novice Partial Mastery Mastery Above Mastery Novice Partial Mastery Mastery Above Mastery NovicePartial Mastery Mastery Partial Mastery

68. Novice Partial Mastery Mastery Above Mastery Novice Partial Mastery Mastery Above Mastery NovicePartial Mastery Mastery Partial Mastery

69. From Student to School Growth – For whom can we calculate growth? » Only students in grades 4 – 11, in tested subjects » Students must have at least 2 consecutive scores » We examine RLA and Math – How do we aggregate growth to the school? » Examine students by grade and by content for the state » Identify all of the students in the school » Take the median (middle) growth percentile of all students in a school for each grade and content.

70. Let’s start with 5 th grade StudentMathematics Johnny25 th Suzy35 th Kenny75 th Lori85 th Juan15 th Lisa40 th Amelia27 th Robert40 th Lee45 th Jan62 nd Frank51 st Mary12 th 5 th Grade MEDIAN Growth 40 th

71. Let’s start with 5 th grade StudentMathematicsProficiency Johnny25 th Novice Suzy35 th Novice Kenny75 th Novice Lori85 th Novice Juan15 th Novice Lisa40 th Novice Amelia27 th Distinguished Robert40 th Distinguished Lee45 th Distinguished Jan62 nd Distinguished Frank51 st Distinguished Mary12 th Distinguished 5 th Grade MEDIAN Growth 40 th

72. Distribution of Student Growth Percentiles by Scale Score Grade 5 Mathematics Demographi cs

73. Aggregated Growth at the School Level Remember the 4 quadrants – High vs. Low Achievement – Based on a Scale Score – High vs. Typical vs. Low Growth – Based on Student Growth Percentiles An example of a school level report

74. Moving on to school Look at each grade distribution of SGPs Take the median of a median – 4 th grade – 5 th grade – 6 th grade GradePercentile 4 th 35 th 5 th 40 th 6 th 37 th School Growth for MATH37 th

75. How it fits in with the Evaluation System 5% of the total evaluation weight at the school level School level growth? – Lowest Growth (1-24 th percentile) – Lower Growth (25 th – 34 th percentile) – Typical Growth (35 th – 65 th percentile) – Higher Growth (66 th – 99 th percentile)

76. Percentile Bands within the Evaluation System Lowest Growth Low Growth Typical Growth High Growth Unsatisfactory Emerging Accomplished Distinguished

77. Why the ranges? Empirical consideration of current data Proposed percentile bands for first pilot year Open for revision Aligns with 4 point rubrics

78. Questions?

79. Thank You Juan D’Brot (jdbrot@access.k12.wv.us)jdbrot@access.k12.wv.us Executive Director Office of Assessment and Accountability

80. Student Learning in Context

81. Learning Targets: – Know the definition of student learning; – Identify examples and non-examples; – Evaluate the quality of potential measures

82. Wrap Up & Feedback

83. A New Landscape

84. Some Parting Instructions Feedback Forms Principals and Assistant Principals – Part Two Dine well and rest well! See you tomorrow at 8:00 a.m.