Using assessment to improve learning: why, what and how? Dylan Wiliam Institute of Education Cambridge Assessment Network seminar on Assessment for Learning: Cambridge, UK; September 2006

Overview of presentation Why raising achievement is important Why investing in teachers is the answer Why assessment for learning should be the focus How we can put this into practice Confidential & Proprietary

Raising achievement matters For individuals Increased lifetime salary Improved health For society Lower criminal justice costs Lower health-care costs Increased economic growth Confidential & Proprietary

Confidential & Proprietary Where’s the solution? Structure Small schools Big schools Alignment Curriculum reform Textbook replacement Governance Specialist schools Vouchers Technology Confidential & Proprietary

Confidential & Proprietary It’s the classroom Variability at the classroom level is up to 4 times greater than at school level It’s not class size It’s not the between-class grouping strategy It’s not the within-class grouping strategy It’s the teacher Confidential & Proprietary

Confidential & Proprietary Teacher quality: A labour force issue with 2 solutions Replace existing teachers with better ones? No evidence that more pay brings in better teachers No evidence that there are better teachers out there deterred by certification requirements Improve the effectiveness of existing teachers The “love the one you’re with” strategy It can be done We know how to do it, but at scale? Quickly? Sustainably? Confidential & Proprietary

Learning power environments Key concept: Teachers do not create learning Learners create learning Teaching as engineering learning environments Key features: Create student engagement (pedagogies of engagement) Well-regulated (pedagogies of contingency) Confidential & Proprietary

Why pedagogies of engagement? Intelligence is partly inherited So what? Intelligence is partly environmental Environment creates intelligence Intelligence creates environment Learning environments High cognitive demand Inclusive Obligatory Confidential & Proprietary

Motivation: cause or effect? high arousal Flow anxiety challenge control worry relaxation apathy boredom low low competence high (Csikszentmihalyi, 1990) Confidential & Proprietary

Why pedagogies of contingency? Several major reviews of the research Natriello (1987) Crooks (1988) Kluger & DeNisi (1996) Black & Wiliam (1998) Nyquist (2003) All find consistent, substantial effects Confidential & Proprietary

Cost/effect comparisons Intervention Effect (sd) Cost/yr/ classroom Class-size reduction (by 30%) 0.1 £20k Increase teacher content knowledge by 1 sd ? Formative assessment/ Assessment for learning 0.2~0.3 £2k Confidential & Proprietary

Confidential & Proprietary Five key strategies… Clarifying and understanding learning intentions and criteria for success Engineering effective classroom discussions that elicit evidence of learning Providing feedback that moves learners forward Activating students as instructional resources for each other Activating students as the owners of their own learning Confidential & Proprietary

Confidential & Proprietary …and one big idea Use evidence about learning to adapt instruction to meet student needs Confidential & Proprietary

Keeping Learning on Track (KLT) A pilot guides a plane or boat toward its destination by planning a route, taking constant readings and making careful adjustments in response to wind, currents, weather, etc. A KLT teacher does the same: Plans a carefully chosen (possibly differentiated) route ahead of time (in essence building the track) Takes readings along the way Changes course as conditions dictate Confidential & Proprietary

Formative assessment & Assessment for Learning Assessment for learning is any assessment for which the first priority in its design and practice is to serve the purpose of promoting pupils’ learning. It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence. An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils, in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged. Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs. Black et al., 2002 Confidential & Proprietary

Types of formative assessment Long-cycle Focus: across units, terms Length: four weeks to one year Medium-cycle Focus: within and between teaching units Length: one to four weeks Short-cycle Focus: within and between lessons Length: day-by-day: 24 to 48 hours minute-by-minute: 5 seconds to 2 hours Confidential & Proprietary

Putting it into practice

A model for teacher learning Content (what we want teachers to change) Evidence Ideas (strategies and techniques) Process (how to go about change) Choice Flexibility Small steps Accountability Support Confidential & Proprietary

Content: strategies and techniques Distinction between strategies and techniques Strategies define the territory of AfL (no brainers) Teachers are responsible for choice of techniques Allows for customization/ caters for local context Creates ownership Shares responsibility Key requirements of techniques embodiment of deep cognitive/affective principles relevance feasibility acceptability Confidential & Proprietary

Design and intervention Our design process cognitive/affective insights synergy/ comprehensiveness set of components Teachers’ implementation process set of components synergy/ comprehensiveness cognitive/affective insights Confidential & Proprietary

Techniques for embedding the strategies in practice

Questioning in Science What can we do to preserve the ozone layer? Reduce the amount of carbon dioxide produced by cars and factories Reduce the greenhouse effect Stop cutting down the rainforests Limit the numbers of cars that can be used when the level of ozone is high Properly dispose of air-conditioners and fridges Confidential & Proprietary

Questioning in English Which of these is a good thesis statement? The typical TV show has 9 violent incidents There is a lot of violence on TV The amount of violence on TV should be reduced Some programs are more violent than others Violence is included in programs to boost ratings Violence on TV is interesting I don’t like the violence on TV The essay I am going to write is about violence on TV Confidential & Proprietary

Confidential & Proprietary Practical techniques Feedback Not giving complete solutions Three-fourths-of-the-way-through-a-unit test Sharing learning intentions Annotated examples of different standards to ‘flesh out’ assessment rubrics (e.g. lab reports) Opportunities for students to design their own tests Students as owners of their own learning Red/green discs Students as instructional resources for one another Pre-flight checklist The research evidence suggests that feedback in terms of scores, grades and levels is unlikely to improve achievement, but that feeding back in terms of comments (whether written or verbal) is. This immediately raises the question “what kind of comments”, and although there is no specific research evidence on this point, it seems that, to be useful, a comment should cause thinking to take place. This feature of ‘mindfulness’ is one of the crucial features of effective formative assessment—effective learning involves having most of the students thinking most of the time. This notion of ‘mindfulness’ also gives some clues about what sort of marking is most helpful. Many teachers say that formative feedback is less useful in mathematics, because an answer is either wrong or right. But even where answers are wrong or right, we can still encourage students to think. For example, rather than marking answers right and wrong and telling the students to do corrections, teachers could, instead, feed back saying simply “Three of these ten questions are wrong. Find out which ones and correct them”. After all, we are often telling our students to check their work, but rarely help them develop the skills to do so. Other strategies that are useful are focused gradin—ie grading a particular piece of work for one aspect (such as sentence structure, expression or spelling) rather than trying to correct everything. This is particularly useful if the comments can be related directly to the assessment criteria for the work. Of course, it is very difficult for feedback to function formatively at the end of a unit so rather than an ‘end-of-unit test’ it may be more useful to have a ‘two-thirds-of-the-way-through-a-unit test’. Those students who have understood something can then help those who haven’t. Many teachers sometimes worry that such strategies may hold back abler students, but the research evidence suggests that it is the students who give help who benefit most from such peer-tutoring. While this may not accelerate more able students through the curriculum, it does lead to better long-term retention. Confidential & Proprietary

Putting it into practice

Why research hasn’t changed teaching The nature of expertise in teaching Aristotle’s main intellectual virtues Episteme: knowledge of universal truths Techne: ability to make things Phronesis: practical wisdom What works is not the right question Everything works somewhere Nothing works everywhere What’s interesting is “under what conditions” does this work? Teaching is mainly a matter of phronesis, not episteme Confidential & Proprietary

Confidential & Proprietary Knowledge ‘transfer’ Confidential & Proprietary After Nonaka & Tageuchi, 1995

Supporting Teachers and Schools to Change through Teacher Learning Communities

Implementing AfL requires changing teacher habits Teachers “know” most of this already So the problem is not a lack of knowledge It’s a lack of understanding what it means to do AfL That’s why telling teachers what to do doesn’t work Experience alone is not enough—if it were, then the most experienced teachers would be the best teachers—we know that’s not true (Hanushek, 2005) People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999) Confidential & Proprietary

That’s what TLCs are for: TLCs contradict teacher isolation TLCs reprofessionalize teaching by valuing teacher expertise TLCs deprivatize teaching so that teachers’ strengths and struggles become known TLCs offer a steady source of support for struggling teachers They grow expertise by providing a regular space, time, and structure for that kind of systematic reflecting on practice They facilitate sharing of untapped expertise residing in individual teachers They build the collective knowledge base in a school Confidential & Proprietary

Confidential & Proprietary The synergy Content: assessment for learning Process: teacher learning communities Components of a model Initial workshops Support for TLC leaders Monthly TLC meetings Peer observations ‘Drip-feed’ resources Web-site Writings New ideas Confidential & Proprietary

Confidential & Proprietary Summary Raising achievement is important Raising achievement requires improving teacher quality Improving teacher quality requires teacher professional development To be effective, teacher professional development must address What teachers do in the classroom How teachers change what they do in the classroom AfL + TLCs A point of (uniquely?) high leverage A “Trojan Horse” into wider issues of pedagogy, psychology, and curriculum Confidential & Proprietary

Confidential & Proprietary Questions? Comments? Confidential & Proprietary