Presentation on theme: "Educator Evaluation Workshop: Gathering Evidence, Conducting Observations & Providing Feedback MSSAA Summer Institute July 26, 2012 Massachusetts Department."— Presentation transcript:
1 Educator Evaluation Workshop: Gathering Evidence, Conducting Observations & Providing Feedback MSSAA Summer InstituteJuly 26, 2012Massachusetts Department of Elementary and Secondary Education1
2 Agenda The Role of Evidence in the 5-Step Cycle Artifacts of Practice Three types of evidenceRoles & responsibilitiesArtifacts of PracticeObservations & FeedbackTips & StrategiesResourcesMassachusetts Department of Elementary and Secondary Education
3 Intended OutcomesAt the end of this session, participants will be able to:Define “evidence of practice” and understand the role of artifacts, observations, and feedback in the 5-Step CycleUnderstand the value of frequent, unannounced observations with targeted feedbackIdentify tools and processes for gathering and organizing evidence that will make evidence collection and feedback more doable in their schools.Massachusetts Department of Elementary and Secondary Education
4 Every educator is an active participant in the evaluation process ContinuousLearningEvery educator & evaluator collects evidence and assesses progressCollaboration and Continuous Learning are the focusMassachusetts Department of Elementary and Secondary EducationMassachusetts Department of Elementary and Secondary Education4
5 “I was evaluated today…” sound familiar? Evaluation Observations
6 Multiple sources of evidence inform the Performance Rating Three categories of evidence must be collected for each educator:Multiple measures of student learning, growth and achievementJudgments based on observations and artifacts of professional practiceAdditional evidence relevant to standardsstudent/staff feedback ( )Massachusetts Department of Elementary and Secondary Education
7 Multiple sources of evidence inform the performance rating This graphic explains the summative performance rating in new educator evaluation framework.The left column includes the three categories of evidence used in evaluation: Products of Practice (including observations, unit plans, schedules and the like), Multiple Measures of Student Learning (ranging from classroom assessments to MCAS Growth Percentile Scores and MEPA when available), and Other Evidence including, eventually, student feedback.(A note here: MEPA – the Massachusetts English Proficiency Assessment for English Language Learners – is being replaced by a better assessment being developed by a national consortium of states called WIDA (pronounced “WEE-DUH.”))The middle column represents the translation of the evidence into an assessment of performance on each of four standards in addition to an assessment of progress on goals.The right column is the single summative performance rating.Starting from the left, using a rubric, an evaluator lines up evidence from the three categories on the left to determine a rating on each of the four Standards (for teachers, the four standards are Curriculum, Planning and Assessment, Teaching all Students, Family Outreach & Engagement, and Professional Culture) and an assessment of progress on both student learning and professional practice goals.Massachusetts Department of Elementary and Secondary EducationMassachusetts Department of Elementary and Secondary Education
8 What does this look like? Products of Practice related to StandardsMultiple Measures of Student LearningOther Evidence related to StandardsArtifactsTeacher-developed unit assessmentsGrade level meeting notesParent/teacher communication logPLC meeting notesObservationsNotes/feedback from short, frequent observations (inside/outside classrooms)Notes and feedback from announced observationsStudent work (quizzes, homework, presentations, etc.)PortfoliosPerformance assessments (including arts, vocational, health & wellness)Interim assessmentsState or district assessmentsStudent and staff feedback (2013/2014 school year)
9 Implementation Responsibility Educator Responsibilities:Documenting action steps completed.Collecting, organizing and submitting evidence to demonstrate progress toward professional practice and student learning goals.Evaluator Responsibilities:Observing practice on a regular basis and providing targeted feedback on performanceMaking resources and supports available.Identification of common artifacts/evidence.
11 It starts with the Educator Plan… Student Learning Goal: In order to ensure mathematical literacy in each of the three content areas for 8th grade geometry (8.G), I will incorporate at least one essay question into each unit assessment that requires elaboration of mathematical reasoning so that 80% or more of my ELL students demonstrate proficiency on essay questions on the end of the year 8th grade geometry assessment.Student Learning Goal(s) Planned ActivityActionSupports/Resources from School/DistrictTimeline/Benchmark or FrequencyBy October 1, I will assess ELL student comprehension and knowledge with formative assessmentsBy October 15th, I will share this data with my department team and instructional coach and solicit feedback on instructional strategies related to teaching mathematical literacy to ELL students.By October 30th, I will develop writing objectives for each unit and integrate them into unit assessments.From November through May, after each unit, I will disaggregate assessment data for ELL students, focusing on mathematical literacy. I will track their progress and adjust instruction as necessary.Formative geometry assessmentMonthly department team meetingsMonthly one-on-one data analysis with instructional coach and ELL specialistUnit assessmentsOct. 1: review formative assessment results for my ELL studentsOct. 15: share formative assessment results with department team and instructional coach and identify at least three instructional strategies related to building mathematical literacy with ELL students.Evidence: meeting notes, 3 strategies2. October 30th: developing writing objectives for each unitEvidence: written objectives, essay questions3. November—June: Administer unit assessments in three content areas and analyze student performance on essay questionsEvidence: student data from essay questions in at least three unit assessments
12 Importance of Strategically Collecting Artifacts Artifacts should be a sample that demonstrates educator performance and impactAligned with educator goals, the Model System Teacher Rubric or school goalsNumber of artifacts to collect varies by educatorArtifacts can provide evidence of more than one Standard-IndicatorAn annotated summary of Grade 5 unit assessment results can include evidence of practice related to I.C. (Analysis), IV.A.1 (Reflective Practice), and IV.C (Collaboration).
13 Lessons from Early Adopters: Collecting Evidence Quality not quantityGuidelines and exemplars will helpPrioritize based on focus areasMassachusetts Department of Elementary and Secondary Education
15 ObservationsThe regulations require a minimum of one unannounced observation.The Model System recommends short, frequent unannounced observations for all educators, as well as at least one announced observation for non-PTS educators and struggling educators.Observations are one more source of evidence, to support and live alongside artifacts of practice.They serve a formative AND summative purpose, as do artifacts. They’re the vehicle through which evaluators can provide targeted, ongoing feedback to educators and maintain a dialogue around teaching and learning.Massachusetts Department of Elementary and Secondary Education
16 Why short, frequent observations? More opportunities to see patterns of practiceFlexibility in schedulingPromotes ongoing conversation around teaching and learningFacilitates observations beyond the classroomIs 5-15 minutes enough?
17 Observation and Feedback School-level Administrator Rubric (I-D-2): Typically makes at least two unannounced visits to classrooms each day and provides targeted, constructive feedback to all educators. Acknowledges effective practice and provides redirection and support for those whose practice is less than proficient. Superintendent Rubric (I-D-2): Typically makes at least three unannounced visits to each school to observe principal practice every year and provides targeted, constructive feedback to all administrators. Acknowledges effective practice and provides redirection and support for those whose practice is less than proficient.We need to go into more detail on Observation here since it is an important source of evidence about educator performance. Here are two more excerpts from the model rubrics for principal (school-level administrator) and superintendent.Going forward, observations will look quite different from the observations that are done in many schools today. In the model system, principals will be making at least two unannounced observations to classrooms every day, and superintendents are expected to make at least three unannounced visits to each school over the course of the year. The school-level classroom observations described here, in particular, may sound undoable, especially if you’re thinking about it in terms of the “classic” observation that requires scheduling an announced, 45-minute visit to a classroom and setting up a pre- and post-conference.HANDOUT: Strategies & Suggestions for ObservationsTake a moment to read the section on frequent, unannounced observations. Think about how this type of observation differs from what you currently do, and how it will reshape the way we think about observations with regard to educator evaluation.Will this be a big culture shift for you?Massachusetts Department of Elementary and Secondary Education17
18 Principles of high quality observations FrequentFocusedInside/Outside the ClassroomUseful & Timely FeedbackThe rubric is not an evaluation tool, but a guide to help identify trends and patterns of practice over time.
19 Feedback“[O]bservers must learn how to capture classroom events in literal notes, and to talk productively with the teacher about it afterward in a way that is evidence-based and productively points toward actionable improvement.” –John Saphier
20 Principles of Good Feedback Verbal as well as writtenFocused on a few key areasBased on evidenceTied to Standards of effective practiceOffers reinforcement for areas of effective practiceFacilitates self-reflection on areas of practice that need refinement and guides the teacher in thinking beyond the lesson observed“The Proficient performance descriptors represent the expected Standard—the bar we expect all experienced teachers to demonstrate over time.”Instruct school teams that they will now have the opportunity to begin looking at the performance descriptors for a particular Indicator and corresponding elements found in Standards I and II, starting with Proficient.Ask participants to count off from 1–7 until every individual has been assigned a number.Place the numbered tabletop cards at each table, and instruct participants to temporarily “re-sort” into these new “teams,” with the 1’s gathering together, the 2’s gathering together, and so on.Massachusetts Department of Elementary and Secondary EducationMassachusetts Department of Elementary and Secondary Education
23 1. PLANThe more concrete the Educator Plan, the easier it is to identify and collect artifactsIdentify common artifacts all or most educators will be expected to collect (unit assessments, parent-teacher logs, etc.)Share examples of high-quality, valuable evidence during faculty or team meetingsDemonstrate example artifacts that provide evidence of more than one Standard-Indicator
25 2. Communicate Expectations Artifacts should be a sample that demonstrates educator performance and impactSubmitted evidence should be tied to educator goals, Standards or Indicators, or school goalsProvide everyone with a clear idea of how and when to share products of practice? Paper? Online cloud?
26 3. ORGANIZE Calendar observations Adopt a process for organizing artifacts and observation notes by Standard/Indicator and/or goalsPaper-based, -driven, or online repositorySample tools for evidence collection and organization
27 Sample Tools for Evidence Collection and Organization Included in your packetSample Evidence ToolCompleted by . . .Artifact Cover PageEducator or Evaluator(the person who identifies the artifact)Observation Evidence Collection ToolObserver/evaluator
28 Massachusetts Department of Elementary and Secondary Education
29 Next Steps – Suggestions for Principals Read “Strategies and Suggestions for Observations” (p. 39 of the School-Level Planning & Implementation Guide)Identify options for collecting and organizing evidence at your school and establish a protocol for all educatorsWork with your administrative team to set a calendar for observations and evaluations based on the distribution of educators by plan type at your schoolMassachusetts Department of Elementary and Secondary Education
30 Massachusetts Model System for Educator Evaluation ResourcesMassachusetts Model System for Educator EvaluationMassachusetts Department of Elementary and Secondary EducationMassachusetts Department of Elementary and Secondary Education
31 School-Level Planning & Implementation Guide Content OverviewThe Massachusetts Model System for Educator EvaluationStep 1: Self-AssessmentStep 2: Goal Setting and Plan DevelopmentStep 3: Implementation of the PlanStep 4: Formative Assessment and EvaluationStep 5: Summative EvaluationAppendices: Forms for Educator Evaluation, Setting SMART GoalsThis is a quick snapshot of what you will see in the School Guide. The guide itself is structured around the five steps of the evaluation cycle: Self-Assessment, Goal Setting & Plan Development, Implementation, Formative Assessment/Evaluation, and Summative Evaluation. The related forms appear in the Appendix along with other resources including guidance about setting “SMART” goals.HANDOUT: The 5-Step Educator Evaluation Cycle: Train-the-Trainer ModulesOne additional resource will be available to every district and school this spring. ESE will be publishing seven “train-the-trainer” modules with facilitator guides designed to help school-level leadership teams learn about the 5-step cycle. ESE will subsidize approved vendors to offer these modules regionally. The seven sequential modules are as follows:1 – Overview2 – Unpacking Rubrics3 – Self-Assessment and Goal Proposal4 – S.M.A.R.T. Goal and Educator Plan Development5 – Gathering Evidence through Artifacts6 – Gathering Evidence through Observation7 – Rating Educator PerformanceAn eighth module will follow this on Rating Educator Impact on Student Learning based on District-Determined Measures, after ESE publishes guidance on Phase 2 of the Educator Evaluation Framework.Massachusetts Department of Elementary and Secondary EducationMassachusetts Department of Elementary and Secondary Education31
32 ESE Evaluation Resources What’s coming?Summer 2012Guidance on District-Determined MeasuresTraining Modules with facilitator guides, PowerPoint presentations, and participant handoutsList of approved vendorsUpdated website with new Resources sectionNewsletterMassachusetts Department of Elementary and Secondary Education
33 ESE Evaluation Resources What’s coming?Fall/Winter 2012Solicit and review feedback on Model System; updateResearch & develop student and staff feedback instrumentsCollect and disseminate best practicesCollect and vet assessments to build a repository of district measuresInternal collaboration to support cross-initiative alignmentEX: Support for use of rubric for teachers of ELLs aligned to RETELL initiativeMassachusetts Department of Elementary and Secondary Education
34 For More Information and Resources: Visit the ESE educator evaluation website: Contact ESE with questions and suggestions: Presenter: Claire Abbott –34Massachusetts Department of Elementary and Secondary Education