1. Using assessment to improve learning: why, what and how?
Dylan Wiliam
Institute of Education
Cambridge Assessment Network seminar on Assessment for Learning:
Cambridge, UK; September 2006

2. Overview of presentation
Why raising achievement is important
Why investing in teachers is the answer
Why assessment for learning should be the focus
How we can put this into practice
Confidential & Proprietary

3. Raising achievement matters
For individuals
Increased lifetime salary
Improved health
For society
Lower criminal justice costs
Lower health-care costs
Increased economic growth
Confidential & Proprietary

4. Confidential & Proprietary
Where’s the solution?
Structure
Small schools
Big schools
Alignment
Curriculum reform
Textbook replacement
Governance
Specialist schools
Vouchers
Technology
Confidential & Proprietary

5. Confidential & Proprietary
It’s the classroom
Variability at the classroom level is up to 4 times greater than at school level
It’s not class size
It’s not the between-class grouping strategy
It’s not the within-class grouping strategy
It’s the teacher
Confidential & Proprietary

6. Confidential & Proprietary
Teacher quality:
A labour force issue with 2 solutions
Replace existing teachers with better ones?
No evidence that more pay brings in better teachers
No evidence that there are better teachers out there deterred by certification requirements
Improve the effectiveness of existing teachers
The “love the one you’re with” strategy
It can be done
We know how to do it, but at scale? Quickly? Sustainably?
Confidential & Proprietary

7. Learning power environments
Key concept:
Teachers do not create learning
Learners create learning
Teaching as engineering learning environments
Key features:
Create student engagement (pedagogies of engagement)
Well-regulated (pedagogies of contingency)
Confidential & Proprietary

8. Why pedagogies of engagement?
Intelligence is partly inherited
So what?
Intelligence is partly environmental
Environment creates intelligence
Intelligence creates environment
Learning environments
High cognitive demand
Inclusive
Obligatory
Confidential & Proprietary

9. Motivation: cause or effect?
high
arousal
Flow
anxiety
challenge
control
worry
relaxation
apathy
boredom
low
low
competence
high
(Csikszentmihalyi, 1990)
Confidential & Proprietary

10. Why pedagogies of contingency?
Several major reviews of the research
Natriello (1987)
Crooks (1988)
Kluger & DeNisi (1996)
Black & Wiliam (1998)
Nyquist (2003)
All find consistent, substantial effects
Confidential & Proprietary

11. Cost/effect comparisons
Intervention
Effect (sd)
Cost/yr/ classroom
Class-size reduction (by 30%)
0.1
£20k
Increase teacher content knowledge by 1 sd ?
Formative assessment/
Assessment for learning
0.2~0.3
£2k
Confidential & Proprietary

12. Confidential & Proprietary
Five key strategies…
Clarifying and understanding learning intentions and criteria for success
Engineering effective classroom discussions that elicit evidence of learning
Providing feedback that moves learners forward
Activating students as instructional resources for each other
Activating students as the owners of their own learning
Confidential & Proprietary

13. Confidential & Proprietary
…and one big idea
Use evidence about learning to adapt instruction to meet student needs
Confidential & Proprietary

14. Keeping Learning on Track (KLT)
A pilot guides a plane or boat toward its destination by planning a route, taking constant readings and making careful adjustments in response to wind, currents, weather, etc.
A KLT teacher does the same:
Plans a carefully chosen (possibly differentiated) route ahead of time (in essence building the track)
Takes readings along the way
Changes course as conditions dictate
Confidential & Proprietary

15. Formative assessment & Assessment for Learning
Assessment for learning is any assessment for which the first priority in its design and practice is to serve the purpose of promoting pupils’ learning. It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence. An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils, in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged.
Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs.
Black et al., 2002
Confidential & Proprietary

16. Types of formative assessment
Long-cycle
Focus: across units, terms
Length: four weeks to one year
Medium-cycle
Focus: within and between teaching units
Length: one to four weeks
Short-cycle
Focus: within and between lessons
Length:
day-by-day: 24 to 48 hours
minute-by-minute: 5 seconds to 2 hours
Confidential & Proprietary

17. Putting it into practice

18. A model for teacher learning
Content (what we want teachers to change)
Evidence
Ideas (strategies and techniques)
Process (how to go about change)
Choice
Flexibility
Small steps
Accountability
Support
Confidential & Proprietary

19. Content: strategies and techniques
Distinction between strategies and techniques
Strategies define the territory of AfL (no brainers)
Teachers are responsible for choice of techniques
Allows for customization/ caters for local context
Creates ownership
Shares responsibility
Key requirements of techniques
embodiment of deep cognitive/affective principles
relevance
feasibility
acceptability
Confidential & Proprietary

20. Design and intervention
Our design process
cognitive/affective
insights
synergy/
comprehensiveness
set of components
Teachers’ implementation process
set of
components
synergy/
comprehensiveness
cognitive/affective
insights
Confidential & Proprietary

21. Techniques for embedding the strategies in practice

22. Questioning in Science
What can we do to preserve the ozone layer?
Reduce the amount of carbon dioxide produced by cars and factories
Reduce the greenhouse effect
Stop cutting down the rainforests
Limit the numbers of cars that can be used when the level of ozone is high
Properly dispose of air-conditioners and fridges
Confidential & Proprietary

23. Questioning in English
Which of these is a good thesis statement?
The typical TV show has 9 violent incidents
There is a lot of violence on TV
The amount of violence on TV should be reduced
Some programs are more violent than others
Violence is included in programs to boost ratings
Violence on TV is interesting
I don’t like the violence on TV
The essay I am going to write is about violence on TV
Confidential & Proprietary

25. Confidential & Proprietary
Practical techniques
Feedback
Not giving complete solutions
Three-fourths-of-the-way-through-a-unit test
Sharing learning intentions
Annotated examples of different standards to ‘flesh out’ assessment rubrics (e.g. lab reports)
Opportunities for students to design their own tests
Students as owners of their own learning
Red/green discs
Students as instructional resources for one another
Pre-flight checklist
The research evidence suggests that feedback in terms of scores, grades and levels is unlikely to improve achievement, but that feeding back in terms of comments (whether written or verbal) is. This immediately raises the question “what kind of comments”, and although there is no specific research evidence on this point, it seems that, to be useful, a comment should cause thinking to take place.
This feature of ‘mindfulness’ is one of the crucial features of effective formative assessment—effective learning involves having most of the students thinking most of the time. This notion of ‘mindfulness’ also gives some clues about what sort of marking is most helpful. Many teachers say that formative feedback is less useful in mathematics, because an answer is either wrong or right. But even where answers are wrong or right, we can still encourage students to think. For example, rather than marking answers right and wrong and telling the students to do corrections, teachers could, instead, feed back saying simply “Three of these ten questions are wrong. Find out which ones and correct them”. After all, we are often telling our students to check their work, but rarely help them develop the skills to do so.
Other strategies that are useful are focused gradin—ie grading a particular piece of work for one aspect (such as sentence structure, expression or spelling) rather than trying to correct everything. This is particularly useful if the comments can be related directly to the assessment criteria for the work.
Of course, it is very difficult for feedback to function formatively at the end of a unit so rather than an ‘end-of-unit test’ it may be more useful to have a ‘two-thirds-of-the-way-through-a-unit test’. Those students who have understood something can then help those who haven’t. Many teachers sometimes worry that such strategies may hold back abler students, but the research evidence suggests that it is the students who give help who benefit most from such peer-tutoring. While this may not accelerate more able students through the curriculum, it does lead to better long-term retention.
Confidential & Proprietary

26. Putting it into practice

27. Why research hasn’t changed teaching
The nature of expertise in teaching
Aristotle’s main intellectual virtues
Episteme: knowledge of universal truths
Techne: ability to make things
Phronesis: practical wisdom
What works is not the right question
Everything works somewhere
Nothing works everywhere
What’s interesting is “under what conditions” does this work?
Teaching is mainly a matter of phronesis, not episteme
Confidential & Proprietary

28. Confidential & Proprietary
Knowledge ‘transfer’
Confidential & Proprietary
After Nonaka & Tageuchi, 1995

29. Supporting Teachers and Schools to Change through Teacher Learning Communities

30. Implementing AfL requires changing teacher habits
Teachers “know” most of this already
So the problem is not a lack of knowledge
It’s a lack of understanding what it means to do AfL
That’s why telling teachers what to do doesn’t work
Experience alone is not enough—if it were, then the most experienced teachers would be the best teachers—we know that’s not true (Hanushek, 2005)
People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)
Confidential & Proprietary

31. That’s what TLCs are for:
TLCs contradict teacher isolation
TLCs reprofessionalize teaching by valuing teacher expertise
TLCs deprivatize teaching so that teachers’ strengths and struggles become known
TLCs offer a steady source of support for struggling teachers
They grow expertise by providing a regular space, time, and structure for that kind of systematic reflecting on practice
They facilitate sharing of untapped expertise residing in individual teachers
They build the collective knowledge base in a school
Confidential & Proprietary

32. Confidential & Proprietary
The synergy
Content: assessment for learning
Process: teacher learning communities
Components of a model
Initial workshops
Support for TLC leaders
Monthly TLC meetings
Peer observations
‘Drip-feed’ resources
Web-site
Writings
New ideas
Confidential & Proprietary

33. Confidential & Proprietary
Summary
Raising achievement is important
Raising achievement requires improving teacher quality
Improving teacher quality requires teacher professional development
To be effective, teacher professional development must address
What teachers do in the classroom
How teachers change what they do in the classroom
AfL + TLCs
A point of (uniquely?) high leverage
A “Trojan Horse” into wider issues of pedagogy, psychology, and curriculum
Confidential & Proprietary

34. Confidential & Proprietary
Questions? Comments?
Confidential & Proprietary