Presentation on theme: "USING OBSERVATION TO ANALYZE EFFECTIVENESS Marybeth Flachbart, Ed.D"— Presentation transcript:
USING OBSERVATION TO ANALYZE EFFECTIVENESS Marybeth Flachbart, Ed.D
OBJECTIVES 3 1.Review research 2.Clarify types of walk-throughs 3.Identify areas of need through observational data
RESOURCES USED: LEADING FOR INSTRUCTIONAL IMPROVEMENT, FINK, MARKHOLT, 2011 VISIBLE LEARNING, HATTIE, 2009 VISIBLE LEARNING FOR TEACHERS, HATTIE,
LEADING FOR INSTRUCTIONAL IMPROVEMENT STEPHEN FINK, ANNEKE MARKHOLT 5
VISBILE LEARNING: A SYNTHESIS OF OVER 800 META-ANALYSES RELATING TO ACHIEVEMENT 6 John Hattie, Professor of Education and Director of the Visible Learning Labs, University of Auckland
EFFECT SIZE MOVES US BEYOND DID IT WORK 7 Effect size is a way of quantifying the size of the difference between two groups. How well does it work? Generally used in meta-analysis for combining and comparing estimates from different studies.
BASIS OF HATTIE’S CONCLUSION 800+ META-ANALYSIS 50,000 STUDIES, AND 200+ MILLION STUDENTS
HATTIE’S RANGE OF EFFECT SIZE 9 Effect Size Benchmark.40 Enhanced – closer to 1.0
VISBILE LEARNING: A SYNTHESIS OF OVER 800 META-ANALYSES RELATING TO ACHIEVEMENT 10 John Hattie, Professor of Education and Director of the Visible Learning Labs, University of Auckland
11 Raise your hand if you’ve read any or all of these books
12 If you have read any of the books: Which one(s)? What did you think of it? Please provide answers in the comment box
DISPELLING THE WIDGET MYTH 13
THE WIDGET EFFECT, NEW TEACHER PROJECT, 2009 Surveyed 14 large American School Districts 98% of teachers were evaluated as “satisfactory” “Based on such findings, many have characterized classroom observation as a hopelessly flawed approach to assessing teacher effectiveness. “ 14
CAN CLASSROOM OBSERVATIONS IDENTIFY PRACTICES THAT RAISE ACHIEVEMENT? 15 effectiveness/ YES!
17 Defining Classroom Management Room setup Access to materials Transition time Procedures Grouping Climate
21 Which is easier to observe, classroom management or instructional practices? Respond in comment box
GATHERING DATA (FINK, MARKHOLT) 22 Types of walk-throughs Learning walk Goal setting Implementation Supervisory Other
YOUR PRACTICE 23 Which types of walk-through do you do most frequently? Why? 1.Learning walk 2.Goal setting 3.Implementation 4.Supervisory 5.Other
OBSERVATIONAL DATA IS NOT EVALUATIVE! “Stick to the facts m’am, just the facts.”
A Profession in Search of a Practice City and Elmore 2009
26 Developing the discipline of seeing Seeing is a discipline It is like a muscle – it gets stronger with repetition Foundation of Instructional Rounds
Using the Observation Protocol collect data that is: Descriptive not evaluative- just the facts! Specific About instructional core Related to problem of practice (or data needed to be collected) OBSERVING CLASSROOM PRACTICE
“ Teacher introduced the concept of fractions and students applied it in a hands-on activity.” vs. “Prompt for student essays: “What role did symbolism play in foreshadowing the main character’s dilemma?” SPECIFICITY OF EVIDENCE
“Student 1 asked student 2: ‘what are we supposed to write down?’ Student 2 said, ‘I don’t know.’” “The teacher read from a book that was not at the appropriate level for the class.” “Teacher introduced the concept of fractions and students applied it in a hands-on activity.” DESCRIPTION VS. JUDGMENT WITH WITHOUT
“Students followed directions in the text to make circuit boards.” “There was too much time on discussion, not enough time on individual work.” “The students conducted a sophisticated lab experiment.” DESCRIPTION VS. JUDGMENT
32 What is the teacher doing and saying? What are the students doing and saying? What is the task? WHEN OBSERVING, WHAT DO YOU SEE?
33 Possible data points: Percentage of teacher vs. student talk Formative assessment practices Standard clear to students Meta-cognitive strategies Number of questions (types of questions) Who is doing the questioning? LEARNING WALK
LEARNING WALK CONTINUED 34 Amount of time spent reading/writing/researching Cognitive demand of student work Academic Vocabulary use Level of engagement What else have you learned by walking around?
PRACTICE: Observing: What do you see? What is the teacher doing? What are the students doing? 36
WHAT DO YOU WONDER? WHAT QUESTIONS WOULD YOU HAVE FOR THE TEACHER? WHAT QUESTIONS WOULD YOU HAVE FOR THE STUDENTS? 37
GOAL SETTING WALK THROUGH 38
GATHER DATA BEFORE YOU SET THE GOAL!
GOAL: IMPROVE WRITING ACHIEVEMENT BY 10% 40 Define the problem of practice: Is it a lack of instruction in writing? Is the lack of instruction in mechanics, process, or both? Is it a lack of practice opportunities to write across the content areas? Is it a lack of vocabulary? Are the vocabulary deficits in oral or written language?
SUPPOSE YOUR STUDENTS WRITING SKILLS WERE WEAK, AND YOU THEN OBSERVED INSTRUCTION… 41 “Based on my observation of 15 classrooms (visited once a week for three weeks), we need to improve the quality of interaction between faculty and students. We need to model and require academic language in both our oral and written communication.”
42 “If we model and then scaffold our students ability to use academic language, we will see an improvement in our students reading, writing, and speaking proficiency.”
THEORY OF ACTION 43
IMPLEMENTATION WALK 45 When do you want to see this change in practice? How frequently will you gather implementation data? Who will gather the data? How will that refine your theory of action? Grade level, whole school, individual teacher?
WHAT IF YOU SEE VERY LITTLE IMPLEMENTATION?
THOU SHALT… 47 Only schedule professional development for faculty that you can measure in terms of a change in instructional practice Prepare a standard protocol for faculty and share it before observation Give sufficient time to implement and resources for job embedded coaching
CLARIFY: 49 Message from observer: Last Friday we had an in-service day where faculty shared their strategies for teaching writing across content areas. I am anxious to see how you’re incorporating these practices in your classroom. I will be doing walk-throughs beginning next week to see how the students are responding to their new learning. Attached is a form I’ll be use to gather data so that we can discuss student work at our PLCs in November. I look forward to seeing you in action.
INSPECT WHAT YOU EXPECT 50 Ask professional development providers (including staff) for an observation protocol “If you can’t see it in the classroom, it doesn’t exist.” Elmore
Inquiry Cycle: Habits of Thinking for Instructional Leadership Noticing – Describe specific facts related to the Observation Protocol that you recorded. Wondering – Based on what you saw (or didn’t see), what are you curious or would like to know more about? Analysis – Using what (collective) knowledge is known about what learning should look like in the ideal in relation to this context observed, think about this information as it compare with my noticing and wonderings. Theories – Based on multiple observations across all classrooms and preponderance of data, this is one theory (or problem of practice) about student learning is ….. Leadership Implications – Using Theory as a guide this is one implication for professional development and my decisions as a leader. ( Gathering additional information and data to measure effectiveness)
TICKET OUT As a result of this presentation what if anything will you add to your classroom observations? response to: