Presentation on theme: "From Evidence to Great Teaching"— Presentation transcript:
1From Evidence to Great Teaching Robert Coe, Durham UniversityASCL Annual Conference, 20 March 2015What can research tell us about the kinds of classroom practices that are likely to create the most learning for students, and the best bets for introducing interventions with greatest cost-effectiveness? Most important – and hardest – of all, how can we best support teachers in improving their classroom practices and realising the full benefits that research seems to promise? In this session Prof Rob Coe will draw on a range of research, including two reviews of which he is an author: the EEF Toolkit and the Sutton Trust’s ‘What makes great teaching?’ report.
2What makes great teaching improving educationeef toolkitWhat makes great teachingCoe, R., Aloisi, C., Higgins, S. and Elliot Major, L. (2014) ‘What makes great teaching? Review of the underpinning research’. Sutton Trust, October 2014Coe, R. (2013) Improving Education: A triumph of hope over experience. Inaugural Lecture of Professor Robert Coe, Durham University, 18 June Essay version available at Video at https://vimeo.com/Higgins, S., Katsipataki, M., Kokotsaki, D., Coleman, R., Major, L.E., & Coe, R. (2013). The Sutton Trust-Education Endowment Foundation Teaching and Learning Toolkit. London: Education Endowment Foundation. [Available at
3The argumentSuccessful implementation of evidence-based strategies requires deep understanding of the evidenceFor most people this requires learningEven if you do ‘what works’ it may not work: always evaluate
5True or false?Reducing class size is one of the most effective ways to increase learning [evidence]Differentiation and ‘personalised learning’ resources maximise learning [evidence]Generous praise encourages learners and helps them persist with hard tasks [evidence]Technology supports learning by engaging and motivating learners [evidence]The best way to raise attainment is to enhance motivation and interest [evidence]Rob Coe EMC blog on class size:John Hattie on individualisation (eg https://cdn.auckland.ac.nz/assets/education/about/research/documents/influences-on-student-learning.pdf).See also Harry Webb’s blog: ‘The Evidence on Differentiation’Deborah Stipek on praise: How Do Teachers' Expectations Affect Student LearningSteve Higgins et al’s summary for EEF: The Impact of Digital Technology on LearningStephen Gorard et al for JRF on aspirations: The impact of attitudes and aspirations on educational attainment and participation
6Most promising for raising attainment Small effects / high cost Impact vs costMost promising for raising attainment8May be worth itFeedbackMeta-cognitivePeer tutoringEarly YearsHomework (Secondary)1-1 tuitionEffect Size (months gain)CollaborativeBehaviourSmall gp tuitionPhonicsParental involvementSmaller classesSocialICTSummer schoolsIndividualised learningAfter schoolSmall effects / high costMentoringTeaching assistantsHomework (Primary)Performance payAspirationsAbility grouping£0£1000Cost per pupil
7Poor Proxies for Learning Students are busy: lots of work is done (especially written work)Students are engaged, interested, motivatedStudents are getting attention: feedback, explanationsClassroom is ordered, calm, under controlCurriculum has been ‘covered’ (ie presented to students in some form)(At least some) students have supplied correct answers, even if theyHave not really understood themCould not reproduce them independentlyWill have forgotten it by next week (tomorrow?)Already knew how to do this anyway
8A better proxy for learning? Learning happens when people have to think hard
9What makes great teaching? (According to the evidence)
11Dimensions of great teaching (Pedagogical) content knowledge (PCK)Quality of instructionClassroom management / behaviour / controlClassroom climate / relationships / expectationsBeliefs (theory) about subject, learning & teachingWider professional elements: collegiality, PD, stakeholder relationshipsCoe, R., Aloisi, C., Higgins, S. and Elliot Major, L. (2014) ‘What makes great teaching? Review of the underpinning research’. Sutton Trust, October 2014
121. We do that already (don’t we?) Reviewing previous learningSetting high expectationsUsing higher-order questionsGiving feedback to learnersHaving deep subject knowledgeUnderstanding student misconceptionsManaging time and resourcesBuilding relationships of trust and challengeDealing with disruption
132. Do we always do that?Challenging students to identify the reason why an activity is taking place in the lessonAsking a large number of questions and checking the responses of all studentsRaising different types of questions (i.e., process and product) at appropriate difficulty levelGiving time for students to respond to questionsSpacing-out study or practice on a given topic, with gaps in between for forgettingMaking students take tests or generate answers, even before they have been taught the materialEngaging students in weekly and monthly review
14For more on what kinds of praise can be harmful and why, see: Richard 1Nov2014 ‘The Problem With Praise’ https://www.psychologytoday.com/blog/smart-moves/201411/the-problem-praiseNick 1Jun2014 Growth mindset: It’s not magic https://evidenceintopractice.wordpress.com/2014/06/01/growth-mindset-its-not-magic/
153. We don’t do that (hopefully) Use praise lavishlyAllow learners to discover key ideas for themselvesGroup learners by abilityEncourage re-reading and highlighting to memorise key ideasAddress issues of confidence and low aspirations before you try to teach contentPresent information to learners in their preferred learning styleEnsure learners are always active, rather than listening passively, if you want them to remember
16If we know what it looks like, can we just do it?
17Improving Teaching Teacher quality is what matters We need to focus on teacher learningTeachers learn just like other peopleBe clear what you want them to learnGet good information about where they are atGive good feedback
18Just a check-list of techniques? No! Great teaching involvesselecting, integrating, orchestrating, adapting, monitoring, responding, etc,and depends oncontext, history, personalities, relationships, etc,But without the skills, a teacher’s choices are more limitedDeveloping these skills & techniques takes dedicated, extended practice, with feedback
19What CPD benefits students? Promotes ‘great teaching’PCK, assessment, learning, high expectations, collective responsibilityFocuses on student outcomesSupported byExternal input: challenge and expertisePeer networks: communities of practiceSchool leaders must actively leadBuilds teacher understanding and skillsChallenges and engages teachersIntegrates theory and active skills practiceEnough learning time (monthly for min 6 months: 30hrs+)Timperley, H., Wilson, A., Barrar, H. & Fung, I. (2007) Teacher professional learning and development: Best evidence synthesis iteration. Wellington, New Zealand: Ministry of Education.Timperley et al 2007
20When ‘what works’ doesn’t work The case for evaluation
21Why monitor teaching quality? Good evidence of (potential) benefit fromPerformance feedback (Coe, 2002)Target setting (Locke & Latham, 2006)Accountability (Coe & Sahlgren, 2014)Individual teachers matter mostTeachers typically stop improving after 3-5 yearsEveryone can improveAssessment is an essential part of learning (including teacher learning)Coe, R. (2002) ‘Evidence on the Role and Impact of Performance Feedback in Schools’ in A.J.Visscher and R. Coe (eds.) School Improvement Through Performance Feedback. Rotterdam: Swets and ZeitlingerLocke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current directions in psychological science, 15(5),Lee J. (2008), ‘Is Test-Driven External Accountability Effective? Synthesizing the Evidence From Cross-State Causal-Comparative and Correlational Studies’, Review of Educational Research 78(3):608–644Coe, R. and Sahgren G.H, (2014) “Incentives and ignorance in qualifications, assessment, and accountability”. In G.H. Sahlgren (ed.) Tests worth teaching to: incentivising quality in qualifications and accountability. Centre for Market Reform of EducationWILIAM, D. (2010) 'Standardized Testing and School Accountability', Educational Psychologist, 45: 2, 107 — 122Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94(2),Kukla-Acevedo, S. (2009). Do teacher characteristics matter? New results on the effects of teacher preparation on student achievement. Economics of Education Review, 28(1),Murphy, R. (2013) ‘Testing Teachers: What works best for teacher evaluation and appraisal’. Sutton Trust.
22Methods of identifying effectiveness classroom observations by peers, principals or external evaluators‘value-added’ models (assessing gains in student achievement)student ratingsprincipal (or headteacher) judgementteacher self-reportsanalysis of classroom artefacts and teacher portfolios
24Lesson ObservationTwo teachers observe the same lesson, one rates it ‘Inadequate’. What is the probability the other will agree?a) 10% b) 40% c) 60% d) 80%An observer judges a lesson ‘Outstanding’. What is the probability that pupils are really making sustained, outstanding progress?a) 5% b) 30% c) 50% d) 70%Answers:1: probably a; best case for a research-validated, quality-assured, rigorous process is b2: probably a; best case for a research-validated, quality-assured, rigorous process is bSee “Classroom observation: it’s harder than you think” and my presentation toTeach First / Teacher Development Trust Event on Lesson Observation, 13 Jan 2014, LondonPowerpoint (http://www.cem.org/attachments/Lesson%20Observation%2013Jan14.pptx) and video (http://www.l4l.co.uk/?p=13050)
25Beware these traps Overconfidence about knowledge of what is effective Focus on teaching rather than learningThinking that we are doing it alreadyOverconfidence in assessments (even if formative) of teaching qualityThinking that if we assess teaching we must attach consequences to that (cf ‘assessment for learning’)
26Problems with assessment criteria If you know what it means, you know what it means (eg from KS1 Performance Descriptors)capital letters for some names of people, places and days of the week (below)capital letters for some proper nouns and for the personal pronoun ‘I’ (towards)capital letters for almost all proper nouns (at)correctly punctuated (mastery)Teaching by numbers (from KS guidance)a range of openings, e.g. adverbials (some time later, as we ran, once we had arrived...), subject reference (they, the boys, our gang...), speech.Some variety in subordinating connectives, e.g. because, if, whichuse of modals to express prediction, possibility, permission, e.g. should, might, could.2014 Key stage 2 writing – moderation. Exemplification materials forteacher assessment (Level 4 - Annotated)
27Bias in Teacher Assessment (vs standardised tests) Systematic bias againstPupils with SEN, EAL & FSMPupils with challenging behaviourReinforcing stereotypesEg boys perceived to be better at mathsethnic minority / subject combinationsPupil/teacher interactionBias against pupils whose personality is different from the teacher’sHarlen W (2004) A systematic review of the evidence of reliability and validity of assessment by teachers used for summative purposes. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=116&language=en-US]Bennett et al., 1993Peter Tymms: ‘Teachers show bias to pupils who share their personality’. The Conversation 25 Feb 2015 https://theconversation.com/teachers-show-bias-to-pupils-who-share-their-personality-38018Burgess, S. and Greaves, E (2009) Test Scores, Subjective Assessment and Stereotyping. Centre for Market and Public Organisation, Bristol University. Working Paper No. 09/221.of Ethnic Minorities
28‘Improvement’ often isn’t School/college would have improved anywayVolunteers/enthusiasts improve: misattributed to interventionChance variation (esp. if start low)Poor outcome measuresPerceptions of those who worked hard at itNo robust assessment of pupil learningPoor evaluation designsWeak evaluations more likely to show positive resultsImproved intake mistaken for impact of interventionSelective reportingDredging for anything positive (within a study)Only success is publicisedCoe, R (2009) ‘School Improvement: Reality and Illusion’ British Journal of Educational Studies, 57, 4,Coe, R. (2013) Improving Education: A triumph of hope over experience. Inaugural Lecture of Professor Robert Coe, Durham University, 18 June Essay version available at Video at https://vimeo.com/See also and for a US perspective.(Coe, 2009, 2013)
29Key elements of good evaluation EEF DIY Evaluation GuideClear, well defined, replicable interventionGood assessment of appropriate outcomesWell-matched comparison groupCoe, R., Kime, S., Nevill, C. and Coleman, R. (2013) ‘The DIY Evaluation Guide’. London: Education Endowment Foundation. [Available atWhat could you evaluate?
31“After 30 years of doing such work, I have concluded that classroom teaching…is perhaps the most complex, most challenging, and most demanding, subtle, nuanced, and frightening activity that our species has ever invented…The only time a physician could possibly encounter a situation of comparable complexity would be in the emergency room of a hospital during or after a natural disaster.”Lee Shulman, The Wisdom of Practice