Educator Effectiveness: Instruction and Assessment Linda Barker, Director of Teaching and Learning

Goal for today….. Provide an overview of how quality standards, instruction, assessment fit into a system for improving practices of educators and student learning results for students.

An invitation…

Requirements: SB 191 State law requirements: Conduct performance evaluations for all teachers and principals at least once each school year. Base at least half of each teacher’s and principal’s evaluation on multiple measures of students’ academic growth (CSAP, as appropriate, plus other growth data). Requires all teachers and principals in one of the performance standards, “highly effective”, “effective”, “partially effective”, or “ineffective”.

Requirements cont… State law requirements : Award non-probationary status to teachers with three consecutive years of “effective” performance and remove it for those who are not “effective” for two consecutive years. Consider factors such as student mobility and the numbers of students with disabilities or at risk of failing school. Require mutual consent of teachers and principals to teacher assignments. Factor in teacher effectiveness before seniority when considering district-level layoffs.

The WHY….Purpose of Evaluation According to the rules for administration of a state system to evaluate the effectiveness of licensed personnel, the basic purposes of this system are: 1. To ensure that all licensed personnel are evaluated using multiple, fair, transparent, timely, rigorous and valid methods, 50 percent of which is determined by the academic growth of their students and 50% on practice 1. To ensure that all licensed personnel receive adequate feedback and professional development support to provide them a meaningful opportunity to improve their effectiveness 2. To ensure that all licensed personnel are provided the means to share effective practices with other educators throughout the state

Key Priorities of the System: 1. Data should inform decisions, but human judgment will always be an essential component. 2. The implementation and assessment of the evaluation system must embody continuous improvement. 3. The purpose of the system is to provide meaningful and credible feedback that improves performance. 4. The development and implementation of educator evaluation systems must continue to involve all stakeholders in a collaborative process. 5. Educator evaluations must take place within a larger system that is aligned and supportive.

1. Evaluation System Goals 2. Stakeholder Investment and Communication Plan 3. Selecting Measures 4. System Structure 5. Evaluators6. Data Integrity 7. Using Results & Professional Development 8. System Evaluation Components of an Effective Evaluation System

2. Annual Orientation 3. Self- Assessment 4. Review of Annual Goals and Performance Plan 5. Mid-Year Review 6. Evaluator Assessment 7. End-of-Year Review 8. Final Ratings 9. Goal-Setting and Performance Planning 1. Training 1. Training A Process, not an Event! Evaluation Cycle Principal/Assistant Principals and Teachers May 15End of September. Prior to the beginning of Spring Semester Train: Prior to the beginning of School. Orient: Within the first week of School. End of MayMid-June End of June Review and finalize compiled results of measures of student learning Include any new district or school decisions for measuring Student Learning Confirm measures used to determine Student Learning – Finalize baseline information Review available information from measures to determine if students are on track May 15

Teacher Evaluation Framework

Definition of Teacher Effectiveness Effective Teachers in the state of Colorado have the knowledge, skills, and commitments needed to provide excellent and equitable learning opportunities and growth for all students. They strive to support growth and development, close achievement gaps and to prepare diverse student populations for postsecondary and workforce success. Effective Teachers facilitate mastery of content and skill development, and employ and adjust evidence-based strategies and approaches for students who are not achieving mastery and students who need acceleration. They also develop in students the skills, interests and abilities necessary to be lifelong learners, as well as for democratic and civic participation. Effective Teachers communicate high expectations to students and their families and utilize diverse strategies to engage them in a mutually supportive teaching and learning environment. Because effective Teachers understand that the work of ensuring meaningful learning opportunities for all students cannot happen in isolation, they engage in collaboration, continuous reflection, on-going learning and leadership within the profession.

STATE COUNCIL FOR EDUCATOR EFFECTIVENESS Framework for System to Evaluate Teachers Definition of Teacher Effectiveness I. Know Content 50% Professional Practice Standards50% Student Growth Measures Weighting: How Much Does Each Standard Count Towards Overall Performance? Observations of Other Measures Teaching Aligned with CDE Guidelines State Other Assessments Other Measures Summative for Non-tested Aligned Assessments Areas CDE Guidelines Match of test to teaching assignments Weighting: Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance? Performance Standards IneffectivePartially Effective Effective Highly Effective Quality Standards II. Establish Environment III. Facilitate Learning IV. Reflect on Practice V. Demonstrate Leadership VI. Student Growth Appeals Process

Look for the first unchecked professional practice. Move one column back to identify the rating for the element. Understanding the Scoring “Business” Rule

BEFORE AFTER

Redundancies have been eliminated. Revised Teacher Rubric 16 Example: Data is now represented in Std. IV, Element a.

Rubric Structure and Rating Level Focus The focus of the Basic rating level is the educator whose performance does not meet state performance standards and who is not achieving at expected levels. The focus of Partially Proficient and Proficient levels is what educators do on a day-to-day basis to achieve state performance standards and assure that students are achieving at expected levels. The focus of Accomplished and Exemplary ratings shifts to the outcomes of the educator’s practices, including expectations for staff, students, parents and community members, as a result of practices exhibited under rating levels 2 and 3.

Teacher Evaluations 50% Professional Practice I. Mastery of content II. Establish learning environment III. Facilitate learning IV. Reflect on practice V. Demonstrate leadership I. Mastery of content II. Establish learning environment III. Facilitate learning IV. Reflect on practice V. Demonstrate leadership Measured using multiple measures on multiple occasions, including: (1) observations; and (2) at least one of the following: student perception measures, where appropriate and feasible, peer feedback, feedback from parents or guardians, or review of teacher lesson plans or student work samples. May include additional measures.

It’s All About the Practice

CDE tool for Observable Practices… The Teacher Students: Element a:  Uses lesson plans that reflect:  Daily review and revision.  Instructional objectives appropriate for students.  Explicit connections to specific learning objectives and approved curriculum  Implements lesson plans based on  Student needs  Colorado Academic Standards.  District’s plan of instruction.  Stated learning objectives.  Collaborates with other school staff to vertically and horizontally align, articulate, and deliver the approved curriculum.  Interact with the rigorous and challenging content.  Perform at a level consistent with or above expectations.  Discuss strengths and next steps regarding their learning with their teachers.

Teacher Self-Assessment Ratings Distribution

Teacher Self-Assessment Ratings-Standard 3- plan and deliver effective instruction.

Theory into Practice…are rubrics real? Portscheller, Schechty, Marzano

Student Learning Outcomes Here’s a simple truth that is easy to forget. School is not about grading. School is about… Learning

STATE COUNCIL FOR EDUCATOR EFFECTIVENESS Framework for System to Evaluate Teachers Definition of Teacher Effectiveness I. Know Content 50% Professional Practice Standards50% Student Growth Measures Weighting: How Much Does Each Standard Count Towards Overall Performance? Observations of Other Measures Teaching Aligned with CDE Guidelines State Other Assessments Other Measures Summative for Non-tested Aligned Assessments Areas CDE Guidelines Match of test to teaching assignments Weighting: Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance? Performance Standards IneffectivePartially EffectiveEffectiveHighly Effective Quality Standards II. Establish Environment III. Facilitate Learning IV. Reflect on Practice V. Demonstrate Leadership VI. Student Growth Appeals Process

Teacher Evaluations 50% Student Learning Outcomes VI. Responsibility for student academic growth Evaluated using the following: (1) a measure of individually- attributed growth, (2) a measure of collectively- attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model. Evaluated using the following: (1) a measure of individually- attributed growth, (2) a measure of collectively- attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model. Refers to outcomes on a measure that are attributed to an individual licensed person, e.g. DRA2 growth measures for a 1 st Grade Teacher’s students Refers to outcomes on a measure are attributed to two or more licensed personnel, e.g. 10 th gr. Math TCAP – All Secondary math teachers in school

It’s All About the Questions?

How do measure student learning outcomes in your district/school? Student Learning Outcomes

Do you understand what % your district (1338) has assigned for collective or individual attribution in your evaluation system. Collective vs. Individual Attribution

Example: Categorizing Teachers No Summative/CGM All Other Teachers 70% Colorado Growth Model (CGM) Reading, Writing, Math (Gr. 4-10) 2. State Summative Reading, Writing, Math (Gr. 3 & 1 st Yr. Gr ) Science (Gr. 5, 8, 10) Social Studies (Gr. 4, 7, HS) ACT (HS) WIDA ACCESS (ELL) What are the pros and cons of creating groups of like teachers based on the types of measures that must be included in their body of evidence? Based on what is expected in law, what categories of teachers are possible?

Select and weight multiple measures of student learning to be included in educator evaluations. What can we do as a district to build comparable bodies of evidence in like groups of teachers? Questions to consider : What assessments must we include? How much influence will they have on the evaluation? Student Learning Outcomes Tool(SLOT) Student Learning Outcomes Tool Student Learning Outcomes

Districts, BOCES and schools may decide after completing the Assessment Inventory that a locally created assessment would likely be included as a measure in evaluation. Which ones? Districts and BOCES can use the Assessment Review Tool to show they are taking steps to ensure that the locally created measurements they select are fair, valid, and reliable. Using Local Assessments in Evaluation

Decision Points How will our district identify teacher roles and the types of assessments that exist? Who will participate in the process? What do we need to know in order to make good decisions? What resources (internal and external) do we have/need? How will we know if we have made good decisions? Categorizing, Weighting, and Selecting Measures Reflection

What are Student Learning Targets? Method of measuring student growth, progress or mastery of the standards Long-term academic goal set by educators for students Based on available prior student learning data and information Set at the beginning of the year/course by teachers in collaboration with their supervisor or evaluator Can be shared by teachers of the same course/grade level or individually Setting Student Learning Targets

Student Learning Target Colorado Academic Standards CurriculumAssessments Student Learning Target Infrastructure

NWEA, targets, %, grades, SLOs,

District & School Performance Frameworks Through the Colorado Educational Accountability Act of 2009 (SB09-163)… CDE annually evaluates districts and schools based on student performance outcomes. All districts receive a District Performance Framework (DPF). This determines their accreditation rating. All schools receive a School Performance Framework (SPF). This determines their school plan types. Provide a common framework through which to understand performance and focus improvement efforts.

Accreditation & Plan Types Accreditation designations: Accredited with Distinction (10%) Accredited (50%) Accredited with Improvement Plan (25%) Accredited with Priority Improvement Plan (10%) Accredited with Turnaround Plan (5%) School plan types: Performance Plan (60%) Improvement Plan (25%) Priority Improvement Plan (10%) Turnaround Plan (5%) What’s my school rating?

Unified Improvement Planning Through the Colorado Educational Accountability Act of 2009 (SB09-163)… All schools and districts must annually develop and submit an improvement plan. The plan streamlines federal and state accountability systems and reporting requirements. Plans must include: Data Analysis: trends, performance challenges, root causes Action Planning: targets, interim measures, major improvement strategies, action steps, implementation benchmarks Aligned with ESEA (Titles I, IIA, III), State Graduation Completion Plans and various grant requirements.

Improvement Planning Process Gather and Organize Data Review Current Performance Describe Significant Trends Prioritize Performance Challenges Identify Root Causes Set Performance Targets Identify Interim Measures Identify Major Improvement Strategies Identify Implementation Benchmarks Section III: Data Analysis and Data Narrative Section IV: Target Setting Section IV: Action Planning Ongoing: Progress Monitoring Preparing to Plan

Dr. Harvey Silver’s work on connections…9 years—”formative assessment leads to dramatic gains in achievement, one of the most powerful tools a teacher can use”. Assessment and Instruction/Practice

What ASSESSMENTS of student learning do you currently use in your classroom, caseload practice or school? Standardized Tests Textbook-based Tests Teacher-developed Unit Tests: e.g. quadratic equations, Catcher in the Rye, photosynthesis. Mid-Year & Finals Common across subject (all students in the grade/subject take the same test) Common across schools (all students in the grade/subject across all schools take the same test) Performance Assessment: lab experiment, speaking a second language, research project, serving a volleyball, solving a math problem with written explanation. Writing Tasks: compositions, essays, reports Portfolios: collection of student work over time Exhibitions: oral report, art show, musical performance, DECA competition, science fair, capstone projects. Use of assessment diagnostic interim/benchmark summative Non-cognitive Value Judgment Do we value the assessment(s)? Why? Assessment Inventory

Think about your assessments/purpose/audience: Feedback/Instruction … FOR learning Data/Trends (after T and L) ….OF learning Exit tickets Sponge pads Work folders Writing logs Discussion records Journals Writing samples TCAP SAT ACT Previous year’s grades District tests Final exams Chapter quizzes and tests

All grades, all levels….. Your job is to uncover understanding, not cover content.

Assessments

Science and Social Studies Summative Assessments Colorado is piloting new science and social studies assessments. These assessments will be computer-based and will be designed to measure the CAS. These assessments will be administered in The science assessments will be administered in grades 5, 8, and 12 and …… social studies assessments will be administered in grades 4, 7, and 12 th grade. Practice testing environments, also referred to as ePATs (electronic Practice Assessment Tools), that help to familiarize students with the testing environment can be accessed at:

Partnership for Assessment of Readiness of College and Careers PARCC’s next-generation assessment system will provide students, educators, and the public with the tools needed to identify whether students — from grade 3 through 11 th grade in math and language arts/literacy— are on track for postsecondary success. PARCC in

Summative assessments aligned to College and Career Readiness Standards Computer adaptive Might not be a set “test window” Results will show achievement and growth Question types: 1. Selected Response 2. Constructed Response 3. Extended Response 4. Performance –Based Tasks 5. Technology-Enhanced 6. Technology-Enabled How will the new assessments be different from CSAP/TCAP?

Drag the words from the word box into the correct locations on the graphic to show the life cycle of a butterfly as described in “How Animals Live.” Words: Grade 3 Technology-Enhanced Constructed-Response Item Pupa Adult Egg Larva

Grade 7 Analytical Constructed Response You have read three texts describing Amelia Earhart. All three include the claim that Earhart was a brave, courageous person. The three texts are: “Biography of Amelia Earhart” “Earhart's Final Resting Place Believed Found” “Amelia Earhart’s Life and Disappearance” Consider the argument each author uses to demonstrate Earhart’s bravery. Write an essay that analyzes the strength of the arguments about Earhart’s bravery in at least two of the texts. Remember to use textual evidence to support your ideas. CCSS-use of evidence, summary of text, complex texts, writing to explain/inform, writing coherently, drawing evidence from texts, grammar and conventions, comparisons of authors, relevant info from multiple sources.

Assessments How do these questions differ from the questions we currently ask on TCAP? Interdisciplinary, online, not one choice, multiple possible answers, defend and justify their answers, mental manipulation, inference, transfer….not your mom/dad or even older brother test! What are the implications for classroom instruction?

Still …

Rubrics are currently being piloted for the following Specialized Service Professional groups: SSP Pilot Year Pilot sites will pilot the evaluation system for those SSP groups they have selected SSP Implementation and Validation Study Year All districts will evaluate their SSPs under SB 191 requirements CDE will conduct a validation study on the evaluation rubrics Specialized Service Professionals 57 AudiologistsCounselorsNurses Occupational Therapists Orientation and Mobility Specialists Physical Therapists School PsychologistsSchool Social WorkersSpeech Language Pathologists

District uses State Scoring Framework Matrix to determine Performance Standard aggregate measures Aggregate professional practice scores into a single score on Quality Standards I-V Aggregate student growth measures into a single score on Quality Standard VI data collection procedures Standards I-V: Must occur with enough frequency to create a credible body of evidence Standard VI: Must occur with enough frequency to create a credible body of evidence weights On each Standard I-V districts may weight priority standards more Standard VI must count for at least 50% of total score measures Standards I-V: use observation plus at least one other method Standard VI: select multiple measures appropriate to teaching assignment Districts decide…

Year One Development and Beta Testing CDE ACTIVITIES Develop State Model Systems for teachers and principals Beta-testing of rubrics and tools Develop technical guidelines on Prof Practices and Student Growth Provide differentiated support for districts Populate and launch online Resource Bank Develop state data collection and monitoring system Develop tools for district implementation of system Year Two Pilot and Rollout CDE ACTIVITIES Usability study of rubrics Support pilot districts through resources, training, tools, etc. Convene pilot districts to share lessons learned Analyze pilot district data and make adjustments as needed Train ALL non-pilot districts that are using the state model Make Recommendations on Specialized Service Professionals (SSP)to State Board of Education (SBE) Year Three Pilot and Rollout CDE ACTIVITIES Statewide TA on rollout of teacher/principal systems Develop evaluation system for Specialized Service Professionals (SSP) Support ALL districts through resources, trainings, tools, etc. Convene pilot districts to share lessons learned Analyze state data and make adjustments to the system as needed Validate teacher and principal rubrics Dev. criteria for eval training courses for approval Year Four Full Statewide Implementation CDE ACTIVITIES Finalize statewide implementation of teacher/principal systems Statewide roll out and validation of SSP system Continue support to districts via resources and training Ensure there are evaluator training courses throughout the state Analyze data and make adjustments as needed Make recommendations to SBE this year and all following years for Continuous Improvement Timeline of Implementation

Questions? Resources? statemodelevaluationsystem Tools, webinars, fact sheets, FAQs, videos…… User guide-all the answers! foundation