Preparation

Resources Chappuis, J. & Stiggens, R. (2011) Classroom Assessment for Student Learning: Doing It Right - Using It Well (2nd Edition) Hattie, J. (2008) Visible Learning A Synthesis of Over 800 Meta-Analyses Relating to Achievement, Routledge Hattie, J. (2012) Visible Learning for Teachers, Routledge Hattie, J. & Yates, G. (2014) Visible Learning and the Science of How We Learn, Routledge

Pre-Reading Teachers Make a Difference (pp. 1-3) Source: Professional Learning and Leadership Development, New South Wales Department of Education and Communities [Hattie, J.A.C. (2003, October)] Retrieved from: http://www.decd.sa.gov.au/limestonecoast/files/pages/new%20page/plc/teachers_make_a_difference.pdf Optional Pre-Reading: Based on the needs of the audience, here is a pre-reading that you may want to send electronically to participants prior to training. Contents of this pre-reading (pp. 1-3 of the article) will be referred to in the information presented on Slide #

Concept or Topic Addressed Handouts or Materials Needed Outline of Visible Learning Introduction Training (continued on next slide) Time in Minutes Concept or Topic Addressed PPT Slide Numbers Activities Handouts or Materials Needed 15 - 25 Welcome , Introductions and Overview Mindset Activity Norms/Working Agreements Learning Objectives Session at a Glance Connection to Teacher Standards Interdependent System (optional) 6-17 “Walk About –Talk About” Activity to establish mindset and focus for the rest of the presentation – impact, teacher and student responsibility, etc “Walk and Talk” Activity Directions Hattie quote strips Missouri Teacher Standards Document Interdependent System Poster & Power Point (optional) 45 - 60 Section 1: Key Concepts and Principles Visible Learning Defined John Hattie Meta-analysis and Effect Size Eight Mindframes Jigsaw & Reflection 18-26 Video Clip Mindframes: Jigsaw Activity   Mindframes reflection Video Clip: What Great Schools Do Video Transcript Article: Know Thy Impact: Teaching, Learning and Leading - An interview with John Hattie Jigsaw Worksheet Mindframes reflection sheet   An Overview of the 2-hour training

Outline of Visible Learning Introduction Training (continued) Time in Minutes Concept or Topic Addressed PPT Slide Numbers Activities Handouts or Materials Needed 40 - 45 Section 2: Maximizing Impact Effect Size – The Research Ruler The Barometer of Influence Make an Educated Guess – a “Pop” Quiz Major Messages for Teachers 27-42 Effective Practices Pop Quiz Fist-to-five Prediction activity Effective Practices Pop Quiz Sheet 20 -25 Section 3: Implementation Observable evidence of visible learning 43-47 View video Table-group discussion One of 3 videos (presenter choice): Slide 44 – Precision teaching: Word Sort (Elementary ELA) Slide 45 – Precision Teaching: Inference Game (Elementary ELA) Slide 46 – Clarity of Learning Targets; Samples of Work: Feedback (8th Grade Math) 20 Section 4: Closure Practice Profile Fidelity Checklist – Self Assessment Final Reflection Goal Setting and Action Planning   48-51 Self-Assessment Exercise with Fidelity Checklist Goal-setting & Action Step Activity Implementation Fidelity Checklist Fidelity Checklist Self-Assessment Sheet An Overview of the 2-hour training

Materials Included in the Packet Outline of Visible Learning Overview Training (1 page power point slide #3) PowerPoint Presentation Handout Packet (Contains all of the handouts referenced.) Practice Profile Fidelity Implementation Checklist “Hattie’s 8 Mindframes” video clip http://www.youtube.com/watch?v=6xpcXobZF1k “What Great Schools Do” John Hattie Video Clip http://www.youtube.com/watch?v=9ousr8uKosk

High IMPACT practices for Effective Teaching and Learning Visible Learning High IMPACT practices for Effective Teaching and Learning

Welcome and Introductions Welcome participants, introduce trainers and ask each participant to tell their name and position. If there are a large number of participants or there is not much time, a “That’s Me” activity may be done instead of having each person introduce him/herself.

Do you agree or disagree with the quote? Walk About / Talk About Silently read your quote and formulate your thoughts around the following prompts: Do you agree or disagree with the quote? On a scale of 1-4, how would you rate your feelings regarding the quote: 1 = Strongly Disagree, 2 = Disagree, 3 = Agree, 4 = Strongly Agree Choose 1-2 key words or concepts contained in the quote with regard to student learning. Follow the directions and cues from the presenter and engage in 2 rounds of walking and talking. A warm-up activity to establish mindset. See Walk and Talk Activity directions included in the materials packet.

Today’s Norms/Working Agreements Be Present and Professionally Courteous Limited side conversations, please Technology is in manners mode Be Open Minded Be Willing to Engage in Conversation, Share Ideas, and Ask Questions Look through the Lens of “How Might I Transfer…”

Today’s Learning Objectives Introduce and review John Hattie’s research around influences related to student achievement Gain an awareness of those habits of mind and beliefs (mind frames) that contribute to positive impact on student learning Explore the effect size of various influences on student learning. Understand instructional behaviors and practices that have significant impact student achievement.

How can instructional impact on student learning be measured? Essential Questions How can instructional impact on student learning be measured? Why is it important to track and monitor instructional impact on student learning? What instructional practices and behaviors have significant positive impact on student learning?

Session at A Glance Framing the Learning Setting the stage: Welcome &Introductions Mindset Activity – “Walk About, Talk About” Norms/Objectives/Framing the Learning/Connections Visible Learning :Key Concepts and Principles What is Visible Learning/John Hattie/Meta-Analysis/Effect Size Eight Mindframes /Jigsaw Activity

Session at A Glance Framing the Learning Maximizing Impact Effect Size Activity – Make an Educated Guess Three major messages for teachers Visible Learning: Implementation

Missouri Teacher Standards A Direct Link Missouri Teacher Standards Standard #2 Student Learning, Growth and Development The teacher understands how students learn, develop and differ in their approaches to learning. The teacher provides learning opportunities that are adapted to diverse learners and support the intellectual, social, and personal development of all students.

Missouri Teacher Standards A Direct Link Missouri Teacher Standards Standard #5 Positive Classroom Environment The teacher uses an understanding of individual/group motivation and behavior to create a learning environment that encourages active engagement in learning, positive social interaction, and self-motivation.

Missouri Teacher Standards A Direct Link Missouri Teacher Standards Standard #7 Student Assessment and Data Analysis The teacher understands and uses formative and summative assessment strategies to assess the learner’s progress and uses both classroom and standardized assessment data to plan ongoing instruction. The teacher monitors the performance of each student, and devises instruction to enable students to grow and develop, making adequate academic progress

Missouri Teacher Standards A Direct Link Missouri Teacher Standards Standard #8 Professionalism The teacher is a reflective practitioner who continually assesses the effects of choices and actions on others. The teacher actively seeks out opportunities to grow professionally in order to improve learning for all students.

Interdependent System Presenters may or may not use this slide. Included is a link to a video tutorial that includes this Systems Thinking Tutorial Video: http://youtu.be/dqslelpZj4M The video includes places for facilitators to pause the video and lead participants in discussion. The video also presents an introduction to the eight mind frames that are the subject of a Jigsaw Activity later in this presentation (slides 24 & 25) Missouri’s Interdependent educational system. When you work in one area, you strengthen all Visible learning is the implementation piece of the system. Transition to next slide “What is Visible Learning?” See Interdependent System Poster in Materials and Handouts file. This may be blown up and displayed as a wall poster or may be reproduced for individual participants.

Visible Learning: Key Concepts and Principles This is a title slide only.

What is Visible Learning? “Visible” refers to making student learning visible to teachers, ensuring attributes that make a “visible” difference to student learning. The “learning” aspect refers to how we go about knowing and understanding then doing something about student “learning.” Visible learning as defined by John Hattie. Have participants read the John Hattie quote silently and then turn to their shoulder partner and “say something” about their thinking around the quote. The “something” may be a comment or question, whatever a participant’s first reaction is after reading the quote.

Professor John Hattie Currently the Director of Melbourne Education Research Institute at the University of Melbourne. Honorary Professor of Education at the University of Auckland Has been both Professor and Chair of Educational Research Methodology at the University of North Carolina. Regularly advises governments in New Zealand, Australia and the United States. Has authored or co-authored 17 books and more than 500 papers. Presenter may want to display the 3 Hattie books with a short descriptor of each: Visible Learning (white book) original publication of the research : Visible Learning for Teachers (blue) book a more teacher and reader-friendly book around the research; and Visible Learning and the Science of How We Learn – the most recent book that provides a more in depth discussion of the HOW and WHY behind the research findings.

Meta-analysis & Effect Size The vast majority of innovations or educational strategies can be said to “work” because they can be shown to have a positive effect. But a student left to work on his own, with the laziest teacher, would be likely to show improvement over a year. In 1976 Gene Glass introduced the notion of meta-analysis – whereby the effects of each study are converted to a common measure or effect size. Refer to the “meta-analysis” done by Marzano (What Works in Schools, 2003) http://www.amazon.com/What-Works-Schools-Translating-Research/dp/0871207176 Summary pdf of Marzano’s meta-analysis: https://www.montgomeryschoolsmd.org/departments/development/documents/what_works.pdf What Works In Schools: Translating Research into Action By Robert Marzano Synopsis by Debbie Ambrose, Center for Skillful Teaching Who is Gene Glass? http://en.wikipedia.org/wiki/Gene_V._Glass Transition to the next two slides : Slide 14 -Marzano did an analysis of 800 research studies for his meta-analysis – Hattie’s research is a meta-analysis of meta-analysis Slide 15 -Marzano divided his “12 Key Factors” that impact student achievement into 3 categories – school-level factors, teacher-level factors, and student-level factors

The Typical Influence on Achievement So what is the typical “effect” across: 800+ meta-analysis 50,000 studies, and 200+ million students? Slide 14 -Marzano did an analysis of 800 research studies for his meta-analysis – Hattie’s research is a meta-analysis of meta-analysis including over 50,000 studies and 200+million students Visible Learning is the result of 15 years’ research and synthesises over 800 meta-analyses (over 50,000 studies) relating to the influences on achievement in school-aged students. It presents the largest ever collection of evidence-based research into what actually works in schools to improve learning (and what doesn’t).

Categorically Speaking Influence Category Effect Size Teacher .50 Curricula .45 Teaching .43 Student .39 Home .35 School .23 Average .40 Slide 15 -Marzano divided his “12 Key Factors” that impact student achievement into 3 categories – school-level factors, teacher-level factors, and student-level factors Hattie categorized 150 different factors into 6 categories. With those factors related to the teacher averaging out to the highest effect size – in other works the most profound impact – of all of the other categories. Refer to pre-reading article “Teachers Make a Difference”

http://www.youtube.com/watch?v=9ousr8uKosk Video: What Great Schools Do? – John Hattie (5:22 minutes) http://www.youtube.com/watch?v=9ousr8uKosk Because of the difficulty in understanding John Hattie’s accent, a transcript of the video is included in the materials package and may be distributed to participants. Participants could be prompted to use the transcript as a video viewing guide, annotating things that “pop out” at them as the video plays. Prompt participants to listen closely for what John Hattie says specifically regarding teacher impact. Quote lines 93-95 in transcript… “One of the things I’m not saying is that they’re teachers that make the difference. It is some teachers who think in a particular way that make the difference …” Highlight some teachers WHO THINK IN A PARTICULAR WAY that make a difference to transition to next slide -8 mindframes Listen closely for what John Hattie says specifically regarding teacher impact.

Mindframe Jigsaw (Part 1) Participants Count off in groups of 4 This group –made up of a 1,2,3,&4 will be the HOME group On cue, participant move to “same number” groups. These groups will read and become experts on their assigned mindframes using the Jigsaw Activity Worksheet to record their thoughts. (8-10 minutes) 1’s Mindframe 1 pp. 3-6 2’s Mindframes 2,3,&4 pp 6-9 3’s Mindframes 5 & 6 pp 9-12 4’s Mindframes 7 & 8 pp 12-13 (Lastly, all read last section, So how do we get there from where we are now? Article: Know Thy Impact: Teaching, Learning and Leading - An interview with John Hattie Article Source: http://www.eosdn.on.ca/docs/In%20Conversation%20With%20John%20Hattie.pdf

Mindframe Jigsaw (Part 2) Participants Return to their HOME groups and share assigned mindframe information with the rest of the group. (10-20 minutes) Each report session should be no longer that 3-5 minutes (including questions) After all of the information has been shared, discussed and clarifying questions asked and answered, participants work independently on the Mindframes Reflection Worksheet. (5-10 minutes) Whole group share (5-10 minutes) Article: Know Thy Impact: Teaching, Learning and Leading - An interview with John Hattie Article Source: http://www.eosdn.on.ca/docs/In%20Conversation%20With%20John%20Hattie.pdf Mindframes Reflection Worksheet

Maximizing Impact

Maximizing Impact “The biggest effects on student learning occur when teachers become learners of their own teaching, and when students become their own teachers.” John Hattie

Effect Size – The Research Ruler Effect size is a method of comparing results on different measures (standardized tests, teacher tests, student work) over time or between groups that allows multiple comparisons. One of the challenges of a meta-analysis that includes so many individual research studies (50,000+) id coming up with a measure that provides comparable scores for each practice studied. Effect size was the measure that Hattie used. Transition to next slide – Hattie’s effect size measure is based on the “standard deviation” calculation that is common in statistical studies.

Standard Deviation +1.00 -1.00 MEAN Students do better than 84% of students not in that initiative Typically represent 2 years growth in one year MEAN A Standard deviation of 1.00 reflects all those practices that fell within a certain distance from the mean (average) score. One standard deviation represents those practices that reported that 84% of the students exposed to that practice did better than students not exposed to the practice. An effect size of 1.00 represents one standard deviation or approximately 2 years worth of growth in one year;\’s time.

Hattie’s “Barometer of Influence” Hinge Point hp= 0.40 Medium 0.4 0.15 Low 0.7 High Teacher Effects Desired Effects 0.0 Developmental Effects Hattie’s Barometer of Influence is based on a “hinge point” of .40 which equals approximately 1 year’s worth of growth in 1 tear’s time. By charting the effect size of each of the practices included in the study, Hattie shows that most everything works to some degree (with the exception of the reverse effects of those practices with an effect size below zero). What is important is for teachers to implement those practices that have an effect size greater than .40 – these are the desired effects that, when implemented with fidelity, result in a minimum of 1 year’s growth in 1 year’s time. Those practices with effect sizes of .60 and above should result in even greater impact. 1.0 Negative Reverse Effects -0.2 1.2 © John Hattie Visible Learning

Make an Educated Guess Work alone, with a partner, or as a small group to determine the “Barometer” placement – high, medium, or low - of the practices listed on the activity sheet. Distribute Effect Size Quiz Sheet. Participants may work alone or with a partner.

Make an Educated Guess HIGH MEDIUM LOW 0.60 or higher 0.40 - 0.59 0.39 or lower This will be a guide for “small” “medium” or “large” Before they begin, move to next slide to have participants make a prediction as to how they will score on the pop quiz.

Fist to Five Prediction Make a prediction of how successful you think you will be on this exercise. I have no idea what I’m doing, but odds are I will get a few correct! 50-64% Correct (16-20/32) 65-75% Correct (21-24/32) 76-89% Correct (25-28/32) 90-100% Correct (29-32/32) Have participants predict their scores by holding up the appropriate # of fingers in a fist-to-five showing. Have them record their predictions on the back of their quizzes. The next slide is a 5-minute timer slide which presenter may or may not use. To use click to next slide, then click on timer to start.

Make an Educated Guess 5 4 3 2 1 Click timer arrow to start. “Time’s Up” graphic will appear at the end of 5 minutes.

Influence Rank (x/150) Effect Size High-Medium-Low Ability Grouping/Tracking/Streaming 131 0.12 Low Acceleration 15 0.68 High Comprehension Programs 26 0.60 Concept Mapping 27 Cooperative vs. Individualistic learning 28 0.59 Medium Direct Instruction 29 Feedback 10 0.75 Gender (male vs. female achievement) 133 ? ? ? ? ? Remind participants that it is important to know how Hattie describes each of these practices in order to fully understand the results. The descriptions of all 150 practices can be found in the Visible Learning book. Go over answers with whole group using the “Effective Stats Hattie Research Ordered” document to clarify descriptors for each practice. Click to reveal the answers for each row. ? ? ?

Influence Rank (x/150) Effect Size High-Medium-Low Home Environment 44 0.52 Medium Individualizing Instruction 109 0.22 Low Influence of Peers 41 0.53 Matching Teaching with Student Learning Styles 125 0.17 Metacognitive Strategy Programs 14 0.69 High Phonics Instruction 36 0.54 Professional Development on Student Achievement 47 0.51 Providing Formative Evaluation for Teachers 5 0.90 ? ? ? ? ? Remind participants that it is important to know how Hattie describes each of these practices in order to fully understand the results. The descriptions of all 150 practices can be found in the Visible Learning book. Go over answers with whole group using the “Effective Stats Hattie Research Ordered” document to clarify descriptors for each practice. Click to reveal the answers for each row. ? ? ?

Influence Rank (x/150) Effect Size High-Medium-Low Providing Worked Examples 32 0.57 Medium Reciprocal Teaching 11 0.74 High Reducing Class Size 113 0.21 Low Retention (Holding back a year) 148 -0.13 Student Control Over Learning 144 0.04 Self-Reported Grades/Student Expectations 1 1.44 Teacher Credibility in the Eyes of Students 4 0.90 Teacher Expectations 62 0.43 ? ? ? ? ? Remind participants that it is important to know how Hattie describes each of these practices in order to fully understand the results. The descriptions of all 150 practices can be found in the Visible Learning book. Go over answers with whole group using the “Effective Stats Hattie Research Ordered” document to clarify descriptors for each practice. Click to reveal the answers for each row. ? ? ?

Influence Rank (x/150) Effect Size High-Medium-Low Teacher Subject Matter Knowledge 136 0.09 Low Student-Teacher Relationships 12 0.72 High Using Simulations and Gaming 86 0.33 Vocabulary Programs 17 0.67 Whole Language Programs 140 0.06 Within-Class Groupings 120 0.18 Response to Intervention 3 1.07 Family Structure 122 ? ? ? ? ? Remind participants that it is important to know how Hattie describes each of these practices in order to fully understand the results. The descriptions of all 150 practices can be found in the Visible Learning book. Go over answers with whole group using the “Effective Stats Hattie Research Ordered” document to clarify descriptors for each practice. Click to reveal the answers for each row. ? ? ?

I have no idea what I’m doing, but odds are I will get a few correct! Fist to Five Revisited I have no idea what I’m doing, but odds are I will get a few correct! 50-64% Correct (16-20/32) 65-75% Correct (21-24/32) 76-89% Correct (25-28/32) 90-100% Correct (29-32/32) How did your actual performance compare with you prediction? Have participants compare their actual score to their prediction. Presenter may pole, using a show of hands, as to how many - met their prediction? scored higher? Scored lower?

What do you have questions about? What surprised you? What do you have questions about? What does this research mean for students and teachers in your - District? Building? Whole group discussion (10 minutes) What does this research mean for your classroom?

Three Major Messages for Teachers the more transparent the teacher makes the learning goals, then the more likely the student is to engage in the work needed to meet the goal. Transparent goals the more the student is aware of the criteria of success, then the more the student can see the specific actions that are needed to attain these criteria Success criteria the more there is feedback about progress from prior to desired outcomes the more positive attributes to learning are developed Rapid formative feedback These three concepts sum up the implications of the major findings of the Hattie research. These three major practices, supported by other “desired effect” practices are the basis of instructional effectiveness.

Visible Learning Implementation

Precision Teaching: Word Sort: Making The Learning Visible Presenter should choose from one of the following video slides to use depending on the group. Slide 44 – Precision teaching: Word Sort (Elementary ELA) Slide 45 – Precision Teaching: Inference Game (Elementary ELA) Slide 46 – Clarity of Learning Targets; Samples of Work: Feedback (8th Grade Math)

What behaviors and practices (both teacher and student) do you notice that provide evidence of “visible learning”? Precision Teaching: Word Sort: Making The Learning Visible Video Link: https://www.youtube.com/watch?v=tSoo0K5eMmI Presenter should prompt individual participants to jot down specific behaviors – both teacher and student – that they observe in response to the prompt on the slide. Once the video has finished (4minutes 1 seconds) participants should share and discuss their findings as a table group. Groups then discuss implications for practice in their own classrooms. Groups appoint one person to speak for the group, citing “aha” moments and /or highlights of their discussions during a whole-group share-out, https://www.youtube.com/watch?v=tSoo0K5eMmI

What behaviors and practices (both teacher and student) do you notice that provide evidence of “visible learning”? Precision Teaching: Inference Game Video Link: https://www.youtube.com/watch?v=z-Vu_CqyEpo Presenter should prompt individual participants to jot down specific behaviors – both teacher and student – that they observe in response to the prompt on the slide. Once the video has finished (4 minutes 7 seconds) participants should share and discuss their findings as a table group. Groups then discuss implications for practice in their own classrooms. Groups appoint one person to speak for the group, citing “aha” moments and /or highlights of their discussions during a whole-group share-out, https://www.youtube.com/watch?v=z-Vu_CqyEpo

What behaviors and practices (both teacher and student) do you notice that provide evidence of “visible learning”? Visible Learning: Effective Feedback Video Link: https://www.youtube.com/watch?v=z-Vu_CqyEpo This is a short video to explain part of the Strategies of Assessment for Learning. Jen Bigenwald explains Strategy 3 (Effective Feedback) and illustrates how it works with a student. Presenter should prompt individual participants to jot down specific behaviors – both teacher and student – that they observe in response to the prompt on the slide. Once the video has finished (4 minutes 7 seconds) participants should share and discuss their findings as a table group. Groups then discuss implications for practice in their own classrooms. Groups appoint one person to speak for the group, citing “aha” moments and /or highlights of their discussions during a whole-group share-out, https://www.youtube.com/watch?v=PpKajKMuABs

What behaviors and practices (both teacher and student) do you notice that provide evidence of “visible learning”? Clarity of Learning Targets: Samples of Work: Feedback Video Link: https://www.youtube.com/watch?v=RX5iwws52AI&index=22&list=PLY8NQxM1fI0-_GdSYdFp1oeDIP-LPkKjo Presenter should prompt individual participants to jot down specific behaviors – both teacher and student – that they observe in response to the prompt on the slide. Once the video has finished (4 minutes 30 seconds) participants should share and discuss their findings as a table group. Groups then discuss implications for practice in their own classrooms. Groups appoint one person to speak for the group, citing “aha” moments and /or highlights of their discussions during a whole-group share-out, https://www.youtube.com/watch?v=RX5iwws52AI&index=22&list=PLY8NQxM1fI0-_GdSYdFp1oeDIP-LPkKjo

Practice Profile Share the Practice Profile. Discuss the contents of the practice profile in terms of the video that was shared. Note that all of the practices that are included as part of the Collaborative Work trainings and expectation have been identified by Hattie as high impact practices – that why they were chosen. The key guidelines for implementation of any of the practices are; Fidelity Consistency Frequency Feedback (data-driven)

Fidelity Checklist – Assess and Share YES PARTIALLY NO I ensure that my instructional practices and learning behaviors are consistent with mindframes that contribute to positive levels impact on student learning. I implement instructional practices and strategies determined to have “hinge points” of 0 .40 or greater, with fidelity, consistency, frequency, and based on causal data. I ensure all instructional practices and strategies are implemented with fidelity, consistency, frequency and actionable feedback. I have established a classroom learning environment based on effective, interactive student/teacher relationships. I implement instructional practices and behaviors that positively impacts learning for every student. Can provide defensible evidence of positive impacts of the teaching on student learning. Introduce the participants to the Fidelity Checklist as a tool that can be used at various times throughout the year to gauge professional growth in the effective implementation of Visible Learning practices. Have participants go through the checklist and gauge their current status using the Visible Learning Implementation Fidelity Checklist Self-Assessment. After completing the checklist, have a table discussion about strengths an weaknesses of the group. Share out with whole group. Remind them that the checklist has a space for listing evidence that should be utilized as part of the self-monitoring and assessment process.

Final Reflection Most important thing you learned and why? One thing you learned that surprised you and why? One thing you learned that you want to know more about? Ask participants to do a final reflection. These may be shared using a strategy of your choice if time permits.

What are Your Goals? What is Your Plan? As the next step in the Plan-Do-Act cycle, have participants discuss at their tables and write one action step that they will take within the next week. Share action steps out with the whole group. What is Your Plan?