1. Michele Winship, Ph.D. winshipm@ohea.org

2.  Presentation slides  Ohio Teacher Evaluation System Framework  Battelle for Kids Value-Added Resources  Student Growth Measures Resources  Race to the Top Work Flow Chart 2

3.  Specifies that the statutory requirements regarding teacher evaluations prevail over collective bargaining agreements entered into on or after September 29, 2011.  Allows for third-party evaluators, such as Educational Service Centers, to be contracted by the board to perform evaluations (requires that an evaluator must hold a credential from ODE).  Specifies that the public school teachers who are subject to the requirement in current law to undergo evaluation by their employers are those teachers who are employed under a teacher license and spend at least 50% of their time employed providing student instruction.  Requires the district to annually report teacher name and evaluation rating to ODE. 3

4.  A national push for teacher evaluation reform from policy makers  Recognition through research that current teacher evaluation practices are not effective in helping teachers improve performance and identifying underperforming teachers  A desire to identify levels of teacher performance to reward high performers and remove low performers  RttT mandate to change evaluation practices  State-level policies that change evaluation requirements  Student performance as a significant factor in teacher (and principal) evaluation (adopted in 13 states currently) 4

5. 5

6.  Sec. 3319.112 Sec. 3319.112  (A) Not later than December 31, 2011, the state board of education shall develop a standards- based state framework for the evaluation of teachers. The framework shall establish an evaluation system that does the following:  (1) Provides for multiple evaluation factors, including student academic growth which shall account for fifty per cent of each evaluation;  (2) Is aligned with the standards for teachers adopted under section 3319.61 of the Revised Code;  (3) Requires observation of the teacher being evaluated, including at least two formal observations by the evaluator of at least thirty minutes each and classroom walkthroughs;  (4) Assigns a rating on each evaluation in accordance with division (B) of this section;  (5) Requires each teacher to be provided with a written report of the results of the teacher's evaluation;  (6) Identifies measures of student academic growth for grade levels and subjects for which the value-added progress dimension prescribed by section 3302.021 of the Revised Code does not apply;  (7) Implements a classroom-level, value-added program developed by a nonprofit organization described in division (B) of section 3302.021 of the Revised Code;  (8) Provides for professional development to accelerate and continue teacher growth and provide support to poorly performing teachers;  (9) Provides for the allocation of financial resources to support professional development. (HB 153 as signed by the Governor) 6

7. 7

8. Sec. 3319.111 Sec. 3319.111 [Effective 9/29/2011] Teacher evaluation (A) Not later than July 1, 2013, the board of education that of each school district, in consultation with teachers employed by the board, shall adopt a standards-based teacher evaluation policy that conforms with the framework for evaluation of teachers developed under section 3319.112 of the Revised Code. The policy shall become operative at the expiration of any collective bargaining agreement covering teachers employed by the board that is in effect on the effective date of this section and shall be included in any renewal or extension of such an agreement. (B) When using measures of student academic growth as a component of a teacher's evaluation, those measures shall include the value-added progress dimension prescribed by section 3302.021 of the Revised Code. For teachers of grade levels and subjects for which the value-added progress dimension is not applicable, the board shall administer assessments on the list developed under division (B)(2) of section 3319.112 of the Revised Code. (C)(1) The board shall conduct an evaluation of each teacher employed by the board at least once each school year, except as provided in divisions (C)(2) and (3) of this section. The evaluation shall be completed by the first day of April and the teacher shall receive a written report of the results of the evaluation by the tenth day of April. (2) If the board has entered into a limited contract or extended limited contract with the teacher pursuant to section 3319.11 of the Revised Code, the board shall evaluate the teacher at least twice in any school year in which the board may wish to declare its intention not to re-employ the teacher pursuant to division (B), (C)(3), (D), or (E) of that section. One evaluation shall be conducted and completed not later than the fifteenth day of January and the teacher being evaluated shall receive a written report of the results of this evaluation not later than the twenty-fifth day of January. One evaluation shall be conducted and completed between the tenth day of February and the first day of April and the teacher being evaluated shall receive a written report of the results of this evaluation not later than the tenth day of April. (3) The board may elect, by adoption of a resolution, to evaluate each teacher who received a rating of accomplished on the teacher's most recent evaluation conducted under this section once every two school years. In that case, the biennial evaluation shall be completed by the first day of April of the applicable school year, and the teacher shall receive a written report of the results of the evaluation by the tenth day of April of that school year. 8

9.  HB 153 creates an advantage for RttT districts  RttT districts can use their funds to buy the time and support to re-create their evaluation systems, including the development of an assessment system  RttT districts can use their funds to purchase support for assessment systems (data management, specific testing protocols, testing materials and grading support)  However…these funds will go away. How will the systems be supported financially in the future? 9

10.  We must begin with the belief that the main purpose of teacher evaluation is improved teaching practice and student learning.  Teacher performance is to be measured through multiple sources of evidence, with observation as one source.  Student performance is required to be 50% of the evaluation, BUT student performance is to be measured through multiple sources of data, not just a single standardized test score.  The State Board of Education has adopted a framework; districts must still develop their evaluation system that includes processes, procedures and forms. frameworksystem 10

11. 11

12.  Student academic growth will be measured through multiple measures which must include value-added scores on evaluations for teachers where value-added scores are available.  Value-added scores are ONLY available for tested grades and subjects, math and reading in grades 4 – 8. Some extended reports are available in locals who participate in Battelle for Kids projects.  Even if there are value-added scores, there must be additional student growth measures for all teachers to meet the multiple measures requirement. 12

13.  Districts will create a local student growth measure worth 50% of the evaluation from a combination of the following:  Value Added Data  Vendor Assessment Student Growth Data  LEA-developed Student Growth Measures 13

14. A. Teacher Value-Added 10-50% + LEA Measures 0-40% = Student Growth (50%) B. Approved Vendor Assessments 10-50% + LEA Measures 0-40% = Student Growth (50%) C. No V-A or Approved Vendor Assessments + LEA Measures 50% = Student Growth (50%) 14

15.  Value-Added + Vendor Assessments  Value-Added + LEA Measures  Value-Added + Vendor Assessments + LEA Measures  Vendor Assessments + LEA Measures  Multiple LEA Measures  Currently, districts will be able to configure their own combinations which may look different for different teachers. 15

16.  Generally three years’ of data are needed in order to draw any conclusions.  Until that data is available in all categories, districts could choose how to weight different data.  Example: 6 th grade math teacher with V-A  Year 1: 10% Teacher V-A + 40% LEA Measures  Year 2: 25% Teacher V-A + 25% LEA Measures  Year 3: 40% Teacher V-A + 10% LEA Measures  There is no research to determine the best combination for accuracy and usefulness. 16

17.  Value-Added Modeling (VAM) has become the “gold” standard for measuring educator effectiveness.  One year’s growth in one year’s time is the benchmark = effective.  Teachers who exceed this growth rate have a positive value-added rating (+) = highly effective  Teachers who fail to meet this growth rate have a negative value-added rating (-)  HB 153 supplements districts rated excellent at $17 per student (the performance index bar is being raised; few schools will be be “excellent.” 17

18.  VAM can provide useful information for teachers:  To look at trends in their classes over time;  To look at performance of various sub-groups of students;  To interpret the impact of their curriculum, instruction, programs and practices on student achievement.  However, V-D data is only part of the picture.  “Student test scores alone are not sufficiently reliable and valid indicators of teacher effectiveness to be used in high-stakes personnel decisions, even when the most sophisticated statistical applications such as value-added modeling are employed.” (EPI Briefing Paper--Problems with the Use of Student Test Scores to Evaluate Teachers) 18

19.  Given that students are not randomly assigned to classes, VAM can’t distinguish between teacher effects and the effects based on students’ needs.  VAM do not provide information to help “struggling” teachers.  Lack of properly scaled year-to-year tests makes it difficult to evaluate gains along the continuum.  Mobility of students (especially in high needs schools) impact the data  VAM cannot distinguish among teachers in the middle range of performance. 19

20.  About 69% of teachers can’t be accurately assessed with VAMs*  Teachers in subject areas that are not tested  Teachers in grade levels (lower elementary) where no prior test scores are available  Special education & ELL  VAM estimates vary with the tests used  If a teacher is in the bottom quintile based on one test there is a 43% chance she will be in the bottom quintile on a different test, but a 16% chance she will be in the top two quintiles.  If a teacher is in the top quintile based on one test there is a 43% chance she will be in the top quintile on a different test, but a 13% chance she will be in the bottom two quintiles. 20

21. 21 Accountability Measures and Reports Technical Assistance and Support SAS® Data Processing Maintained by SAS®  Single Limited Access Password Protected Data:  District/LEA and school  Student information  Analytic tools  Teacher-level reports  Limited Use Public Access  Includes BFK SAS® EVAAS® reporting  Enhanced reporting features Developed & supported by BFK  Regional System  Trained VAL’s support districts/LEAs through DVALT training  Support to teacher- teams  Focus on school improvement  Toolkits  Online courses Maintained by ODE  LRC  Valued-Added  AYP Growth Measure  ODE Reports-School and District (LEA) Measures Diagnostic Tools EVAAS® Diagnostic Tools EVAAS® ODE- BFK Partnership

22. o Rollout Schedule o 30% of LEAs Link in Year 1 RttT (reports received fall 2011) o 60% of all RttT LEAs in Year 2 o 100% of all LEAs in Ohio in Years 3 & 4 o Requirements—Accuracy of Reporting o Must conduct linkage o Minimum number of students and time enrolled o Access to Reporting o Online via EVAAS® accounts o Password protected o Grades/Subjects Available o ODE: grades 4-8, math & reading o BFK: grade 3, math & reading; grades 3-8, science & social studies; high school—algebra I & II, geometry, pre-calculus, biology, chemistry, English 9, 10 & 11  Issue—Public Records Requests 22

23. 23

24. 24

25. 25

26. 26

27.  ODE has solicited information from test vendors about the assessments they offer which are appropriate for measuring student growth.  There are many tests that measure “student achievement,” but fewer that are scaled and administered in such a way to determine student growth.  From the information received, ODE has published a list of “approved” assessments from which districts can select to administer. 27

28. 28 Assessment Name / Vendor NameGrade(s) / Subject(s) STAR Early Literacy Renaissance PK-3 ELA STAR Early Math Renaissance G1-3, 9-12 Math STAR Reading Renaissance G1-3, 9-12 Reading Stanford 10 Pearson K-12 Math Science, Social Studies, Solution 1 Stanford Aprenda 3 Pearson K-12 Math Science, Social Studies, Solution 2 Terra Nova 3 CTB K-12, 1-12 ELA, Math, Science, Social Studies Iowa Assessments Riverside K-12 ELA, Math, Science, Social Studies Riverside Interim Assessments Riverside G 2-11 ELA, Math Performance Global Scholar K-12 ELA, Math, Science

29. 29 Assessment Name / Vendor NameGrade(s) / Subject(s) iReady Diagnostic Curriculum Associates K-8 ELA, Math Explore ACT G 8-9 ELA, Math, Science Quality Core ACT G 9-12 End of Course Exams: Algebra I, II, Geometry, Pre-Calculus, Biology, Chemistry, Physics, U S History The ACT ACT G 11-12 ELA, Math, Science PLAN ACT G 10 English, Reading, Math, Science Compass ACT G 10-12 Writing, Reading, Math MAP NWEA G 9-10 Science MAP NWEA G 9-12 Math Reading, Language Usage

30. 30

31.  If local boards of education choose to administer assessments chosen from the Ohio Department of Education’s assessment list for teachers of subjects where value- added scores are not available, they will have to purchase these assessments.  If vendor assessments are used, they must be part of the 50% SGM. 31

32. 1. Rigorous. Are measures “rigorous,” focused on appropriate subject/grade standards? Measuring students’ progress towards college and career readiness? 2. Comparable. Are measures “comparable across classrooms,” ensuring that students are being measured with the same instruments and processes? 3. Growth over time. Do the measures enable student learning growth to be assessed “between two points in time”? 32

33. 4. Standards-based. Are the measures focused on assessing growth on important high-quality grade level and subject standards for students? 5. Improve teaching. Does evidence from using the measures contribute to teachers’ understanding of their students’ needs/progress so that instruction can be planned/adapted in a timely manner to ensure success? 33

34.  Student Learning Objectives (SLOs) are targets of student growth that teachers set at the start of the school year and strive to achieve by the end of the semester or school year.  These targets are based on a thorough review of available data reflecting students' baseline skills and are set and approved after collaboration and consultation with colleagues and administrators.  AISD uses SLOs as part of a performance compensation model. Teachers set and assess two student learning objectives and receive financial incentives for meeting one or both of these goals at the end of the school year. 34

35. 1. Review baseline data 2. Create SLOs 3. Obtain SLO approval, per local process 4. Monitor progress toward attainment of SLO growth targets 5. Revise SLOs, if necessary 6. Review evidence and evaluate progress towards and attainment of SLO growth targets 35

36. 36

37. ScoreDescription Exceptional Attainment (5) Evidence across all Student Learning Objectives indicates exceptional student mastery or progress. All objectives are exceeded. This category is reserved for the educator who has surpassed expectations described in their Student Learning Objectives and/or demonstrated and outstanding impact on student learning. Full Attainment (4) Evidence across all Student Learning Objectives indicates superior student mastery or progress. This category applies to the educator who overall has nearly met the majority of the expectations described in their Student Learning Objectives and/or who has demonstrated a notable impact on student learning. Considerable Attainment (3) Evidence across all Student Learning Objectives indicates significant student mastery or progress. If an objective was not met, evidence indicates that is was nearly met. This category applies to the educator who overall has nearly met the majority of the expectations described in their Student Learning Objectives and/or who has demonstrated a considerable impact on student learning. Partial Attainment (2) Evidence across all Student Learning Objectives indicates some student mastery or progress. Educator may have met or exceeded some objectives and no met other objectives. Educator may have nearly met all objectives. This category applies to the educator who has demonstrated an impact on student learning, but overall has not met the expectations described in their Student Learning Objectives. Minimal or No Attainment (1) Evidence across all Student Learning Objectives indicates little student mastery or progress. This category applies to the educator who has not met the expectations described in their Student Learning Objectives and has not demonstrated a sufficient impact on student learning. This category also applies when evidence of objectives is missing, incomplete, or unreliable or when the educator has not engaged in their in the process of setting and gathering evidence for Student Learning Objectives.

38.  To be meaningful, student performance data should be used by educators to  Identify achievement gaps,  Inform instructional practice, and  Direct professional development.  To effectively use the data, teams of educators should  Be trained in the analysis and interpretation of student performance data,  Have real-time access to the data, and  Meet regularly in teams to analyze the data and plan intervention, instruction and professional development. 38

39. How do we create the conditions for educators to use student performance data effectively? 39

40.  Requiring student performance in teacher evaluations means districts will need to: 1. Map current school-based and district-wide assessments in all grades and subjects 2. Determine where assessment “gaps” exist 3. Create groups of educators to select/develop appropriate assessments for “gaps” 4. Create an assessment timeline for all grades and subjects 5. Collect, analyze and store student performance data 6. Provide time and training for educators to work together with student data to improve their own instruction 40

41.  Use the assessments you have first.  Determine what assessments you need to create a rigorous, comparable and inclusive assessment system that is designed to provide student performance data to be used for educator professional growth and also for inclusion in an evaluation system.  Chart a course of action with a timeline, persons responsible and deliverables. 41

42. 42

43.  Each evaluation will consist of two formal observations of the teacher at least thirty minutes each in duration, as well as periodic classroom walkthroughs.  Teacher performance metrics must also use multiple and variable sources of data, such as lesson plans, samples of student work, classroom assessment results, and portfolios, in addition to data from direct observation in classrooms. 43

44. 44

45. 45

46.  Goal Setting  Self Assessment against Ohio Standards  Analysis of student data  Identifying 2 professional growth goals  Formative Assessment of Teacher Performance—Formal Observation  Pre-observation conference  Observation  Post-observation conference and reflection  Evidence Collaboration and Professionalism (determined locally)  Student Growth 46

47.  The overall teacher performance rating (50%) will be combined with the results of student growth measures (50%) to produce a summative evaluation rating as depicted in the following matrix.  Teachers will be rated in one of four categories:  Accomplished  Proficient  Developing  Ineffective 47

48. 48

49.  Teachers with above expected levels of student growth will develop a professional growth plan and may choose their credentialed evaluator for the evaluation cycle.  Teachers with expected levels of student growth will develop a professional growth plan collaboratively with the credentialed evaluator and will have input on their credentialed evaluator for the evaluation cycle.  Teachers with below expected levels of student growth will develop an improvement plan with their credentialed evaluator. The administration will assign the credentialed evaluator for the evaluation cycle and approve the improvement plan.  This is entirely unrealistic and does not reflect what actually happens in schools. 49

50.  Anyone who evaluates a teacher will have to be trained and credentialed by state trainers (similar to Resident Educator mentor training)  Training is 3 days of face-to-face work viewing teaching videos and learning how to collect evidence and score using the OTES rubric  Credentialing requires passing an online assessment that will involve viewing a video and scoring within an acceptable range. 50

51.  At the local level, the board of education will include in its evaluation policy, procedures for using the evaluation results for retention and promotion decisions and for removal of poorly- performing teachers.  Seniority will not be the basis for teacher retention decisions, except when deciding between teachers who have comparable evaluations.  The local board of education will also provide for the allocation of financial resources to support professional development. 51

52.  With a July 1, 2013 deadline for system completion, evaluation work will need to begin ASAP and may not fit into current bargaining cycle  Effective evaluation reform will require collaboration with administration at a very different level in many locals  Future evaluation language in CBAs will need to include all processes, procedures and tools  Stakes are high; we can’t afford to adopt systems that aren’t designed to support teachers 52

53.  Composition and selection of evaluation team members  Timeline for evaluation work  Compensation for work outside of the school day  Mandatory training for evaluators for observation protocols and ratings  Training for staff about evaluation processes, procedures and tools  No-fault piloting provision to work out problems 53

54. 1. Identify and engage district evaluation team, including teachers from various levels/areas 2. Review and analyze teacher current evaluation polices and rules 3. Conduct ODE Evaluation GAP AnalysisGAP Analysis 4. Review effective evaluation models including the OTES 5. Select/Develop a district evaluation system and tools 6. Map and develop student assessments that will provide student performance data 7. Create training for evaluators and teachers 8. Construct a pilot timeline 9. Have volunteer teachers and evaluators pilot the system 10. Review and revise the system based on pilot data 11. Train all evaluators and teachers 12. Implement the new evaluation system 54

55. 55

56.  Teacher Evaluation Systems materials and resources (login required) http://www.ohea.org/teacher-evaluation- systems http://www.ohea.org/teacher-evaluation- systems  www.lauragoe.com Includes various state and local systems and examples of multiple measures for teacher performance and student growth www.lauragoe.com  Teacher Assessment and Evaluation: The NEA's Framework http://www.nea.org/home/41858.htm http://www.nea.org/home/41858.htm  Getting Teacher Assessment Right: What Policymakers Can Learn from Research -- the source for Dr. Hinchey’s presentation: http://nepc.colorado.edu/publica tion/getting-teacher-assessment-righthttp://nepc.colorado.edu/publica tion/getting-teacher-assessment-right 56

57.  Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality.Approaches to evaluating teacher effectiveness: A research synthesis  Goe, L., Holdheide, L., Miller, T. (2011) A practical guide to designing comprehensive teacher evaluation systems. Washington, DC: National Comprehensive Center for Teacher Quality.A practical guide to designing comprehensive teacher evaluation systems.  Hinchy, P. (2010). Getting Teacher Assessment Right: What Policymakers Can Learn From Research. Boulder, CO: National Education Policy Center.Getting Teacher Assessment Right: What Policymakers Can Learn From Research  Mathers, C., Oliva, M., with Laine, S. W. M. (2008). Improving instruction through effective teacher evaluation: Options for states and districts. Research and Policy Brief. Washington, DC: National Comprehensive Center for Teacher Quality.Improving instruction through effective teacher evaluation: Options for states and districts.  National Education Association. (2009). Teacher evaluation systems: The window for opportunity and reform. Washington, D.C.Teacher evaluation systems: The window for opportunity and reform.  Stronge, J. H, & Tucker, P. D. (2003). Handbook on teacher evaluation: Assessing and improving performance. Larchmont, NY: Eye on Education.Handbook on teacher evaluation: Assessing and improving performance 57

58.  Laura Goe--Webinar for Oregon School Coaches, April 20, 2011: http://www.lauragoe.com/LauraGoe/Oregon- April%202011.pptx http://www.lauragoe.com/LauraGoe/Oregon- April%202011.pptx  EPI Briefing Paper--Problems with the Use of Student Test Scores to Evaluate Teachers: http://www.epi.org/publications/entry/bp278 http://www.epi.org/publications/entry/bp278  Rand Education—Evaluating Value-Added Models for Teacher Accountability: http://www.rand.org/pubs/monographs/2004/R AND_MG158.pdf http://www.rand.org/pubs/monographs/2004/R AND_MG158.pdf 58

59.  Michele Winship  614-227-3001  winshipm@ohea.org winshipm@ohea.org 59