1. Copyright © Tulsa Public Schools 2011 © 2012, Tulsa Public Schools The Tulsa Model TLE Observation and Evaluation System / Process OVERVIEW

2. 2 Why did Oklahoma adopt a new teacher evaluation system? (a) We needed a better method of identifying and describing effective teaching. (b) We needed a better method of identifying and describing ineffective teaching. 2

3. 3 Why Teacher Effectiveness Matters 3

4. 4 Facts… 1. Effective Teachers impact student performance. 2. TPS TLE… was “home grown”, tested and revised by Oklahoma’s best… Oklahoma Teachers. is research-based and externally validated. measures what matters. 4

5. 5 The Tulsa Model Evaluation System Observations Evaluations Conferences Feedback / Support Rubric-Based 5

6. 6 A CAUTION… The Tulsa Model was approved by the OSDE and must be followed as written. Specifically, material or substantive alteration to the Model is not permitted. 6

7. 7 Tulsa Model’s Focus for Accomplishment Create a common framework and language for evaluation. Provide educators with clear expectations about what is being assessed, as well as standards that should be met. Send messages about what is most meaningful. Increase the consistency and objectivity in evaluating professional performances. 7 Switching Gears

8. 8 Tulsa Model’s Focus for Accomplishment Provide educators with information about where they are in relation to where they need to be for success. Identify instructional practices that drive student achievement. Give educators guidance for evaluating and improving their work. 8

9. 9 Statutory Requirements The Tulsa Model has a five (5) tier rating system: Ineffective Needs Improvement Effective Highly Effective Superior 70 S.O. § 6-101.16 (SB 2033) 9

10. 10 Statutory Requirements – Qualitative Characteristics – 50% of total rating* Organizational and classroom management skills Ability to provide effective instruction Focus on continuous improvement and professional growth Interpersonal skills Leadership skills *No final decision on 50% QUANTATIVE MEASURES as of yet. 70 S.O. § 6-101.16 (SB 2033) 10

11. 11 Statutory Requirements Qualitative assessment must be evidence- based and include observable and measureable characteristics that are correlated to student performance. 70 S.O. § 6-101.16 (SB 2033) 11

12. 12 Statutory Requirements Local board policies regarding evaluations must be based on TLE by 2013-2014 school year. Data generated from the TLE will be used to drive educational decisions that include: Career teacher status Mandatory exiting of ineffective teachers 12

13. 13 Statutory Requirements Comprehensive remediation plan / instructional coaching are required for teachers rated 1 - Ineffective or 2 - Needs Improvement. Reduction in Force policies will use teacher effectiveness as the primary basis for releasing teachers. 13

14. 14 What We Have Learned… and a Fact 1. The Tulsa Model is a performance- based evaluation SYSTEM and has an adherence PROCESS. 2. The Tulsa Model is controlled by Statutory provisions. 3. The evaluation data from the Tulsa Model will NOT be used for employment decisions… unless the local Board adopts the relevant model as its official teacher evaluation policy for the district. 14 CURRENT TULSA MODEL TULSA MODEL

15. 15 Tulsa Model Evaluation System Description Rubric Provides the narrative definitions of professional proficiency (effectiveness) for all 5 rankings. A matrix structure provides clarity and specificity for the 20 Indicators that impact student performance. 12345 IneffectiveNeeds Improvement EffectiveHighly EffectiveSuperior NOT in Evidence >>> Efficacy in Effectiveness Achieved 15 Switching Gears

16. 16 TLE Evaluation System Description Observation+ Process The Observation scores reflect the Evaluator's intentional study of the teacher’s classroom performance plus an assessment of other factors specified in the Rubric up to a given point in time. Observations and the Observation Form serve as the precursors (foundations) to the Evaluation. Is guided by the rubric. A minimum of two (2) observations are required before every evaluation. 16

17. 17 TLE Evaluation System Description (cont.) Evaluation Form Web-based or paper-based. Your choice. Conferences Follow every observation and evaluation. Customized Feedback and Support Focusing the most intensive supports for 1 (“Ineffective”) and 2 (“Needs Improvement”). 17

18. 18 The Tulsa Model System - Structure Teachers Classroom Management Instructional Effectiveness Professional Growth & Continuous Improvement Interpersonal Skills Leadership TAKEN FROM SB 2033 EFFECTIVENESS MEASURES 18

19. 19 TLE Evaluation System – Indicators, etc. Teachers (20 Indicators within 5 Domains, and Domains have assigned weights) Classroom Management (6) @ 30% Instructional Effectiveness (10) @ 50% Professional Growth & Continuous Improvement (2) @ 10% Interpersonal Skills (1) @ 5% Leadership (1) @ 5% DEFINING AND ADDING SPECIFICITY TO SB 2033 EFFECTIVENESS MEASURES 19

20. 20 The RUBRIC… Definitions 20 Establishing a Common Language… “Domains” are the Performance Measures taken from SB 2033. “Dimensions” are the TITLES of the specific components that establish the parameters of the Domain, e.g., the branches off the main truck of a tree. “Indicators” are those observable and measurable characteristics and practices that are correlated to student performance success.

21. 21 Teacher Rubric THE FOUNDATION FOR DEFINING EFFECTIVENESS 21 Switching Gears

22. 22 Full Rubric for Indicator 2… 22

23. 23 Rubric, Observation and Evaluation Teachers Counselors Deans Librarians School Development Teachers School Nurses School Psychologists Speech-Language Pathologists 23 ALSO AVAILABLE ON THE PORTAL

24. 24 Rubrics and the Horse’s Tale Which direction is the horse’s head facing? 24 Switching Gears

25. 25 Perception Perception is a unique interpretation of the situation, not an exact recording of it. Perception is a very complex cognitive process that yields a unique picture of a person, event or even the world. 25

26. 26 Perception The process of attaining awareness or understanding of information. 26

27. 27 Perception 27

28. 28 Perception 28

29. 29 An Administrative Goal for the Evaluation Process To move from subjectivity to higher and higher levels of objectivity and insight. How? Information, information, information, and more information. 29

30. 30 Determining an Indicator’s Rating Each Indicator often has several definitional narratives for each level of proficiency. However, Evaluators must enter only one (1) score as to each Indicator (1, 2, 3, 4, 5, NO, NA). Use the “BEST FIT.” 3 30

31. 31 Observation+ Process The Observation scores reflect the principal's intentional study of the teacher’s classroom performance plus an assessment of all inclusive factors specified in the Rubric up to a given point in time. 31 It is the sum total of the instructional value that a Teacher brings to his/her students.

32. 32 Observation+ Process 13 of the 20 Indicators are commonly apparent in a classroom observation: From the Domain of Classroom Management: 32 1Teacher plans for delivery of the lesson relative to short-term and long-term objectives. 2Teacher clearly defines expected behavior. 5Teacher acknowledges student progress and uses assessment practices that are fair and based on identified criteria. 6Teacher optimizes the learning environment through respectful and appropriate interactions with students, conveying high expectations for students and an enthusiasm for the curriculum.

33. 33 Observation+ Process 13 of the 20 Indicators are commonly apparent in a classroom observation: From the Domain of Instructional Effectiveness: 33 7Teacher embeds the components of literacy into all instructional content. 9Teacher uses active learning, questioning techniques and/or guided practices to involve all students. 10Teacher teaches the objectives through a variety of methods. 11Teacher gives directions that are clearly stated and relate to the learning objectives. 12Teacher demonstrates / models the desired skill or process. 13Teacher checks to determine if students are progressing toward stated objectives. 14Teacher changes instruction based on the results of monitoring. 15Teacher summarizes and fits into context what has been taught. 16Effective development and use of modified assessments and curriculum for special education students and other students experiencing difficulties in learning.

34. 34 Teacher Observation Form THE TOOL TO OPERATIONALIZE THE RUBRIC 34 2012 -2013 TLE Observation Form

35. 35 Rubric Converted to Observation Form Observation Form retains the Effectiveness Level of 3, but uses (-) if less than effective or (+) if exceeds effectiveness. 35

36. 36 Teacher Observation Form 20 CELLS Flexible Structure 36

37. 37 Completing the Form… a CONVERSATION On the Observation Form, Tulsa uses a coding system of “-” for Ineffective or Needs Improvement; “3” for Effective; “+” for Highly Effective or Superior; “NO” for Not Observed; “NA” for Not Applicable. Plus comments. Districts can opt to use ALL rankings, e.g., 1, 2, 3, 4, 5, NO, or NA. Plus comments. What advantages and/or disadvantages can you see in using one option or the other on the Observation Form? 37

38. 38 Teacher’s Observation Form Bottom of page 1 Top of page 2 1 2 38

39. 39 Observation+ to the Evaluation Observations and the Observation Form serve as the precursors (foundations) to the Evaluation. Is guided by the Rubric. A minimum of two (2) observations are required before every evaluation. 39 REMINDER

40. 40 Evaluation Form THE OFFICIAL PERFORMANCE DOCUMENTATION 40 Switching Gears

41. 41 Evaluation Form Ranges were needed to define the Rankings. 41

42. 42 Evaluation Form Calculations 42 Domain: C’room ManagementRatingDomain Average Indicator 1 narrative sentence32.33 Indicator 2 narrative sentence2 Indicator 3 narrative sentence1 Indicator 4 narrative sentence4 Indicator 5 narrative sentence2 Indicator 6 narrative sentence2

43. 43 Evaluation Form Calculations 43 DomainAverageDomain Weight by % Classroom Management2.3330% Instructional Effectiveness3.2050% Prof Growth / Cont. Impro.2.5010% Interpersonal Skills4.005% Leadership2.005% The single number that “potentially is high stakes” is the Composite Weighted Average and it is arrived at via an Excel formula. In the above hypothetical case it would be: =((2.33*30)+(3.20*50)+(2.50*10)+(4.00*5)+(2.00*5))/100

44. 44 Evaluation Form Calculations 44 The Composite Weighted Average (CWA) would be 2.89. Next the Ranges determine the Rating Classification: Less than 1.8 = Ineffective Equal to or greater than 1.8, but less than 2.8 = Needs Improvement Equal to or greater than 2.8, but less than 3.8 = Effective Equal to or greater than 3.8, but less than 4.8 = Highly Effective Equal to or greater than 4.8 = Superior In the above hypothetical, the CWA is 2.89; therefore, the Teacher is EFFECTIVE. However, if any Indicator reflects a 1 or a 2, then a PDP must be written.

45. 45 Requirements of the Evaluation Form 1. Any ranking of 1 or 2 on any component (an individual Indicator) of this Evaluation requires a Personal Development Plan (PDP), following the prescribed format and conferencing on same. 2. Any ranking of 4 or 5 on any component (an individual Indicator) requires “summary” narrative comments under Evaluator Comments on the Evaluation Form. 45

46. 46 Observation / Evaluation Process Fidelity If an Evaluator BEGINS the process, then that SAME Evaluator conducts all steps along the way including the summative Evaluation. There is no hand-off between Observations or between the Observations and the Evaluation. If that occurs the entire process must begin again. 46

47. 47 What We Have Learned… 1. The Rubric drives all subsequent actions… Observations, Evaluations, Feedback and Support Opportunities and Conferences. 2. The Tulsa Model Process cannot be interrupted by a change in Evaluator. 47

48. 48 Does not lend itself to “catch-up” The Tulsa Model scheduling requires INTENSE and FOCUSED Time Management. Use Outlook, an electronic scheduler, a whiteboard or a calendar… BUT, begin planning the year by the 1 st week in August. 48 AugSeptOct NovDecJan FebMarApr May Switching Gears

49. 49

50. 50 The Conference Process Observations shall NOT be conducted on the day immediately following any extended break in the instructional calendar (whether scheduled or unexpected). 50

51. 51 Artifact File or Portfolio On a voluntary basis, an Educator may wish to provide additional evidence of effectiveness in the form of a portfolio or artifact file / binder for purposes of INFORMING the Evaluator of additional involvements, contributions and activities in support of improving student performance and achievement. 51

52. 52 Feedback and Support-Focused Rubrics are detailed roadmaps for improvement for all teachers. Observation conferences are a status check prior to formal evaluation. Requires customized teacher training and responses w/re to teachers ranked in bottom two tiers. 52 Switching Gears

53. 53 Feedback and Support-Focused Language from SB 2033… “Annual evaluations that provide feedback to improve student learning and outcomes; Comprehensive remediation plans and instructional coaching for all teachers rated as needs improvement or ineffective;” 53

54. 54 Customized Feedback and Supports for Teacher Improvement 1. “Push Pins” (less formal, yet documented) approaches to remedy area(s) of ineffectiveness or needing improvement 2. Personal Development Plans (PDPs) 54

55. 55 Feedback / Support – Push Pin Approach “Push Pins” (less formal, yet documented) approaches to remedy area(s) of ineffectiveness or needing improvement. Approaches could include: In passing, a brief conversation in hallway, classroom, etc. Requested brief conversation in office. Written note in mailbox. Email to staff member. 55

56. 56 Feedback / Support – Push Pin Approach It is a professional “judgment” call deciding upon the use of a “Push Pin” or a PDP. Example 1: A 60 second verbal tirade to a fellow staff member (in a closed door faculty room) regarding the loss of a planning period because of actions of that staff member. Push Pin or PDP? Example 2: Outdated bulletin boards. Push Pin or PDP? Example 3: Leaving a class of AP students unsupervised for 15 minutes while talking on a cell phone in the hallway. Or, special needs students. Push Pin or PDP? 56

57. 57 Feedback and Support - PDPs A Personal Development Plan… 1. Cites / Summarizes the situation and/or Observation / Evaluation details (referencing Indicator(s)) 2. Provides a List of Expectations 3. Provides an Action Plan using a SMART format (specific, measurable, attainable, resources attached, timeline included) 4. Provides a Follow-Up Progress Reporting (within a maximum of a two month window) NOTE: ALL PDPs require a CONFERENCE. 57

58. 58 Feedback and Support - PDPs A Personal Development Plan… MAY be created based upon any Observation that results in a 1 – Ineffective or 2 – Needs Improvement determination MUST be created based upon an Evaluation that results in a 1 – Ineffective or 2 – Needs Improvement ranking OR, could be generated as a Stand-Alone document that illustrates behavior(s) at Levels 1 or 2 58

59. 59 What We Have Learned… 1. Time management is CRITICAL to the success of the TLE Process. 2. The statutory requirement for “feedback and support” can be achieved via Push Pin and Personal Development Plan (PDP) approaches. 3. A PDP has a prescribed format, with clear, focused, descriptive narrative writing being a requirement. 59

60. 60 Transparency and Understanding Either as paper or electronic copies… ALL staff members MUST have access to the following: Evaluator’s Handbook Rubric Observation Form Evaluation Form Structure for the Personal Development Plan 60

61. 61 For more information: www.tulsaschools.org Office of Teacher and Leader Effectiveness 918-746-6800 Tulsa Framework: http://sde.state.ok.us; left column: Teacher and Leader Effectiveness; Scroll to Frameworks; Tulsa Framework Talia Shaull – shaulta@tulsaschools.org Jana Burk – burkja@tulsaschools.org Katy Ackley – ackleka@tulsaschools.org Gene S. Kleindienst - kleinge@tulsaschools.org 61 Tulsa Model via Portal: http://tulsaschools.org; left column: Teacher and Leader Effectiveness Portal; Scroll to tabs of interest