Prof Robert Coe School of Education Tel: (+44 / 0) Fax: (+44 / 0) Assessment Lecture for PGCE September 2012

2 Definition of a grade ‘An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.’ Dressell (1983)

Assessment What different types of assessment are there? Problematising assessment: assessment criteria; assessment for accountability; teacher assessment Why does assessment matter? How can teachers use assessment to promote learning? 3

© 2010 Robert Coe, Durham University 4 Background: Types of assessment and their problems

5 Assessment types Functions of assessment To certify achievement (summative) To inform learning (formative) To control curriculum, teachers, students (accountability) Types of assessment Formal / informal Criterion / norm / self referenced Continuous / final / progressive Questions of quality Validity: does it test what you want students to learn? Reliability: would it give the same result again?

6 Criterion-referenced assessment Define what you want learners to be able to do Assess whether they can ‘Add numbers up to 20’ Context is everything! Approp- riate referen- cing and citation details. Good in terms of structure, compre- hensibility and quality of presen- tation. Good level of knowledge and understanding in terms of central ideas and concepts within the field of education theory, policy and context with appropriate supporting factual detail allied to an ability to make explicit the relationship of the ideas to the wider field of educational knowledge. A relevant line of argument is presented and the relationship of this to the wider field of educational knowledge is made explicit. Demonstrat es a critical understandi ng of the appropriaten ess of educational* research and its relationship to the topic. The theoretical justification for practice and the practical implications of theoretical ideas are discussed as are some of the dilemmas which arise from the relationship of educational theory to practice. Evidence of sufficient reading focussing on a range of educational source material [e.g. government reports, research articles, books, personal experience etc.]

© 2010 Robert Coe, Durham University 7 Teaching to the test: the ‘Texas miracle’ Klein et al, 2000

© 2010 Robert Coe, Durham University 8 Teacher assessment Pros Based on sustained performance Based on assessment of authentic learning tasks (validity) Incorporates a range of tasks Motivates students Cons Judgements often have low reliability Hard to translate into absolute standards Halo effects Teacher workload Problems if teachers are judged by results

Rising standards

25 point rise in PISA = +£4,000,000,000,000 GDP

© 2010 Robert Coe, Durham University 11 Assessment: Perhaps the single most important determinant of learning

© 2010 Robert Coe, Durham University 12 What factors influence achievement most? John Hattie and his team have analysed data from all available studies on the effects of different strategies From 50,000 studies Involving over 200,000,000 students Compiled in over 750 meta-analyses Which are most promising? Personalised learning?0.20 Smaller classes?0.21 Ability grouping?0.25 Better headteacher?0.30

© 2010 Robert Coe, Durham University 13 Assessment… InfluenceEffect Size Self-report grades1.44 Feedback.72 Providing formative evaluation to teachers.70 Frequent/ Effects of testing.46 Teaching test taking skills.22

‘Best buy’ strategies Cost per pupil Effect Size (months gain) £ £1000 Feedback Meta-cognitive Peer tutoring Pre-school 1-1 tutoring Homework ICT AfL Parental involvement Sports Summer schools After school Individualised learning Learning styles Arts Performance pay Teaching assistants Smaller classes Ability grouping Promising May be worth it No way

© 2010 Robert Coe, Durham University 16 “… we have each been asked several times by teachers, ‘What makes for good feedback?’—a question to which, at first, we had no good answer. Over the course of two or three years, we have evolved a simple answer—good feedback causes thinking.” (Black & Wiliam, 2003)

Focus on learning Does your ‘theory of learning’ explain why ability grouping (setting) after-school clubs teaching assistants smaller classes do not work? © 2011 Robert Coe, Durham University 17

Do we care about learning? Which of the following are evidence of learning? Students are busy: lots of work is done Students are engaged, interested, motivated Classroom is ordered, calm, under control What do school students value most? Social interactions & status with peers Keeping out of trouble Pleasing teachers: good marks, neat writing, polite Thinking hard about really challenging problems © 2011 Robert Coe, Durham University 18

Assessment can help focus on learning How can you know what your students are learning? How often do they need to think hard? Learning is invisible, and is low on the agenda of many teachers and students If you want to evaluate what students are learning, you need appropriate tools to help you © 2010 Robert Coe, Durham University 19

© 2010 Robert Coe, Durham University 20 If you want your students to learn something difficult … You need to know how many of them have ‘got it’ They need to know whether they have ‘got it’ If they haven’t, you need to be able to do something about it

© 2010 Robert Coe, Durham University 21 Formative assessment Effect sizes between 0.4 and 0.7 Advice to improve formative assessment: Feedback should focus on work, with advice for improvement, not comparisons with others Pupils should be trained in self-assessment - to understand learning aims Teaching should allow pupils to express their understanding Create reflective dialogue between teacher and pupil Tests should be frequent and relevant; feedback and support to improve should be given “Information used as feedback to modify teaching and learning” Black and Wiliam, 1998

© 2010 Robert Coe, Durham University 22 Formative assessment in practice Sharing success criteria with learners Classroom questioning Feedback through marking, eg comment-only Peer-assessment and self-assessment Formative use of summative tests Black et al, 2002

© 2010 Robert Coe, Durham University 23 Comments: instead or as well? Comments group Comment and grade group Butler, 1988

© 2010 Robert Coe, Durham University 24 “If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.” Wiliam, 1999

© 2010 Robert Coe, Durham University 25 One evaluation of comment-only marking “I would prefer to be in another form because we don’t get our test marks back” Smith and Gorard, 2005

Which feedback is best? a)Well done, that is very good work b)Well done, that is very good work (for you) c)Well done, you have shown a high level of ability in solving these problems d)Well done, that is one of the best essays in the class e)Well done, that is a big improvement on your previous work f)Well done, I can see you have worked hard at this g)Well done, I can see you have concentrated on reading the questions carefully and it has paid off © 2011 Robert Coe, Durham University 26

© 2010 Robert Coe, Durham University 27 Feedback does not always enhance performance relate performance to clear, specific and challenging goals make people focus on the task, not themselves, nor compare them with others compare their performance with their own past performance be seen as informative, not controlling make people feel competent, but not complacent Coe, 1998 Ideally, feedback should …

© 2010 Robert Coe, Durham University 28 Feedback does not always enhance performance suggest reasons for performance that are alterable (eg effort, strategies) follow soon after performance be specific and task focused be given to individuals on their individual performance correct errors or inadequacies Coe, 1998 Ideally, feedback should …

© 2010 Robert Coe, Durham University 29 Further advice on assessment Feedback should relate to individuals’ (mis)understandings Praise should be infrequent, credible, contingent, specific and genuine Praise should be related to factors within an individual’s control Don’t help too much: ‘minimal intervention’ Feedback should support a view of ability as incremental rather than fixed Students often don’t understand what they are meant to be doing and how it is assessed Wiliam, 1999, 2000