1. CLP Fidelity of Implementation Rater Training November 18 th, 2011

2. Learning Intention We are learning to… understand the Fidelity of Implementation (FOI) Observation Protocol for the CLP.

3. Success Criteria We will know we are successful when we can… complete the FOI Protocol and input the data online to ensure consistency among all reliability raters.

4. Forecasting Our Learning Fidelity of Implementation Observation Protocol (Reading K-8 & ELA K-12) Overview Reading Observation guide and protocol Practice Video and Test Video ELA Observation guide and protocol Practice Video and Test Scenario.

5. How we will handle questions … Please write down any questions on your note cards. We will answer questions after lunch and at the end of the day.

6. CLP Fidelity of Implementation Fidelity of Implementation (FOI) is the extent to which the actual classroom instruction looks like the intended instruction.

7. Who? Reliability Raters ▫ Regional Coordinators of C & I ▫ DIFI Supervisors ▫ Literacy Specialists ▫ RtI Specialists Raters ▫ Literacy Coaches and ICLs A random sample of 30% of MPS teachers will be observed in DPI and MPS selected schools.

8. What? Observing Reading and ELA instruction in classrooms Collecting information on the Fidelity of Implementation protocols

9. Where? In classrooms during Reading and ELA in a random sample of classrooms The DPI sample included three schools per region during first walk window

10. When? Tentatively: During regularly-scheduled Reading classroom blocks Observation Window #1: Sept. 16 – 30 ▫ Entered by October 6 Observation Window #2: Nov. 28 – Dec. 16 ▫ Entered by December 16 Observation Window #3: Feb. 27-16 ▫ Entered by March 16 Observation Window #4 : May 14-June 1 ▫ Entered by June 1

11. When? Tentatively: During regularly-scheduled ELA classroom blocks Observation Window #1: January 16 – 31 ▫ Entered by January 31 Observation Window #2: April 2-5 & 16-27 ▫ Entered by April 27

12. Fidelity of Implementation Observation Protocol NOT an individual teacher evaluation ▫ Observation only ▫ Data is confidential, but not anonymous District-wide collection of program-specific data Completed on paper and later entered online

13. And most importantly, Why? Ultimately, to help improve literacy teaching and learning in MPS To increase understanding of the CLP in action To help support teachers, students, and others involved in these efforts To provide data to DPI as well as to MPS leaders

14. And then….how? The Fidelity of Implementation Observation Protocol will help MPS: ▫ Conduct observations with inter-rater reliability (yoked walks) ▫ Collect precise data ▫ Assess observations with validity ▫ Report observations with consistency ▫ Measure levels of implementation around essential components of the CLP

15. Rater instructions upon arrival to the school FOI checks are unannounced visits. Arrive to the school early, so you have enough time to enter the classroom a few minutes prior to the scheduled start time to ensure you begin the FOI check at the start of the class period.

16. Rater instructions upon arrival to the classroom As you enter the classroom, please smile and make eye contact with the teacher when you introduce yourself. Explain that you are holding a Fidelity of Implementation check and will observe at various points during the lesson.

17. Transparency about the FOI process Please make a copy of the “Milwaukee Public Schools Fidelity of Implementation General Information” sheet (located in your guide). Fill in the information at the top of the sheet. Please leave this sheet in the teacher’s mailbox at the culmination of your observation.

18. How to Access School/Teacher Information for Raters Reading Questions 1-8 Raters can access their assigned schools and teachers that require FOI checks by logging in to: http://www2.milwaukee.k12.wi.us/rti/reporting or http://www2.milwaukee.k12.wi.us/survey http://www2.milwaukee.k12.wi.us/rti/reporting http://www2.milwaukee.k12.wi.us/survey Be sure to click on Reading OR ELA accordingly Here you will find all the information you will need to plan for your FOI Reading and ELA checks (School, teacher, classroom, daily schedule, grade level).

19. Reading Question 9: Accessing the Instructional Timeline Use the Instructional Timeline to determine whether or not their instruction is on pace.

20. Reading Questions 10, 11, 12, 16 The reading class is scheduled to start at 1:00. When you get to the classroom the students are working on a math lesson. At 1:10 they begin their reading lesson.

21. Answers 10: No 11: 6-10 minutes 12: 1:10 16: 1:15

22. Reading Questions 13, 14 (use abbreviations), 15 During Segment 1, the teacher is working with all students together in a whole group. She begins by asking the students to define the vocabulary words listed on the Focus Wall. Then, the teacher directs students’ attention to the punctuation marks on the board and tells the students how different punctuation marks alert the reader to use different expression when reading. Students practice with sentences that they make up. (this content is not listed on the focus wall or in the pacing guide).

23. Answers 13: Whole Group 14: Vocabulary (Aligned), Explicit Fluency Instruction (Not Aligned), Fluency Practice (Not Aligned) 15: Yes

24. Reading Question: Sheet 3 The teacher remains in whole group instruction during the entire class period. Do you record anything on Sheet 3: Teacher-Led Small Group Observation Segment?

25. Answer Yes

26. Reading Question, Segment 2 & 3 What do you do between Segment 2 (Teacher-Led Observation Segment) and Segment 3 (Whole Group Ending Observation Segment)?

27. Answer Remain in or return to the classroom to observe additional teacher-led small groups if they occur.

28. Reading Questions 18, 25, 26a, 26b You observe 3 different teacher-led small groups, one beginning at 9:30 led by the teacher, another beginning at 9:50 led by the special education teacher who teams in the classroom and a third at 10:05, led by the teacher. In each small group, the teacher follows the exact same lesson plan – using round robin reading and the same questions on the same story.

29. Answers 18: 9:30 25: 9:35 26a: 3 26b: No

30. Reading Question 19 A 2 nd grade class begins at 9:00. You return to the class to observe Segment 2 at 9:25. The class is in the bathroom and remains there until 9:40. What are the next TWO questions you will answer?

31. Answers 19: No Questions 20 and 28

32. Reading Question 24 How do you know which stations are in use? How do you know if stations are aligned?

33. Reading Questions 28 and 34 The class was scheduled to go from 10:00- 11:30. They began CLP instruction at 10:16 and taught CLP-related activities until 11:35. PLEASE ALSO ANSWER QUESTION 35

34. Answers 28: 11:25 34: 11:30 35: No

35. Practice Video Don’t worry about questions 1-4 You can use today’s date and segment times 7 th grade classroom Scheduled time: 8:05-9:05 Not a bilingual classroom, in a traditional school Don’t answer questions 24, 26a or 26b Alignment can also include district-adopted materials

36. Certification of Raters Now that we have completed working through the protocol, let’s see how you do on your own. 80% success rate to be a certified rater.

37. Reading Test: Online Data Entry Go to http://www2.milwaukee.k12.wi.us/rti/reporting to submit data for each observation. http://www2.milwaukee.k12.wi.us/rti/reporting All data must be submitted online by the end of the day of the observation. Make sure to click submit!! Please give your literacy specialist a paper copy of Observation Protocols to keep on file for the academic year.

38. Test Video Don’t worry about questions 1-4 You can use today’s date and segment times K5 classroom Scheduled time: 8:00-9:30 Not a bilingual classroom, in a traditional school Don’t answer questions 24, 26a or 26b Alignment can also include district-adopted materials

39. ELA guide and protocol Many of the ELA questions are similar to the Reading Protocol Briefly review a few differences: ▫Question 8 ▫Question 9 ▫Learning Intentions ▫Common Core State Standards ▫Whole/Small/Whole Rating ▫Learning Experiences

40. ELA Scenarios and Video Scenario A (Complete with a partner and share as a group) Video (Complete on your own and share as a group) Scenario B (rater reliability assessment)

41. ELA Test: Online Data Entry Go to http://www2.milwaukee.k12.wi.us/rti/reporting to submit data for each observation. http://www2.milwaukee.k12.wi.us/rti/reporting All data must be submitted online by the end of the day of the observation. Make sure to click submit!! Please keep your paper copy of Observation Protocols for the academic year.

42. Fidelity of Implementation (FOI)