1. USING EVIDENCE TO INFORM YOUR LEADERSHIP APPROACH AND SUPPORT SCHOOL IMPROVEMENT ROB CARPENTER 26 TH SEPTEMBER 2013 rob.carpenter@unitedlearning.org.uk @carpenter_rob http://robcarpenter1971.tumblr.com @carpenter_rob robcarpenter1971.tumblr.com

2. The Triple whammy

3. The work of EEF Building the evidence of what works to raise the attainment of disadvantaged pupils in schools. Sharing the evidence with schools by providing independent and accessible information. Promoting the use of evidence-based practice both through our projects, events and resources. @carpenter_rob robcarpenter1971.tumblr.com

4. Problems with research Quality varies, but a lot is not very good Quality really matters How do you know who or what to trust? Academic papers are inaccessible Academic debates are (mostly) pointless Peer review doesn’t work @carpenter_rob robcarpenter1971.tumblr.com

5. Spending and outcomes Capturing the benefits of increased spending is difficult. Internationally, the relationship between spending and outcomes is very weak. @carpenter_rob robcarpenter1971.tumblr.com

6. Spending and outcomes In England, spending has increased by 47% since 2001, but over this period improvements in pupil outcomes have been marginal. @carpenter_rob robcarpenter1971.tumblr.com

7. Scenario: Spending the Pupil Premium A large secondary school receives £140,000 from the Pupil Premium in 2012-13. How should the school decide to use this money? Would you rather spend your Pupil Premium on professional development to improve the quality of feedback pupils receive, or small group tuition, or class size reduction? (WHAT? WHY? HOW?) The Toolkit doesn’t tell you what to do, but we hope that it will help you make a more informed decision. @carpenter_rob robcarpenter1971.tumblr.com

8. Impact vs cost Cost per pupil Effect Size (months gain) £0 0 8 £1000 Meta-cognitive Peer tutoring Early Years 1-1 tuition Homework (Secondary) Mentoring Summer schools After school Aspirations Performance pay Teaching assistants Smaller classes Ability grouping Promising May be worth it Not worth it Feedback Phonics Homework (Primary) Collaborative Small gp tuition Parental involvement Individualised learning ICT Behaviour Social www.educationendowmentfoundation.org.uk/toolkit @carpenter_rob robcarpenter1971.tumblr.com

9. The Sutton Trust-EEF Teaching and Learning Toolkit The Toolkit is an accessible, teacher-friendly summaries of educational research Practice focused: giving schools in the information they need to make informed decisions and narrow the gap Based on meta-analyses provided by Durham University @carpenter_rob robcarpenter1971.tumblr.com

10. Toolkit overview @carpenter_rob robcarpenter1971.tumblr.com

11. Feedback Approach Average impact Cost Evidence estimate Summary Feedback 9 months££ Very high impact for low cost Research suggests that providing effective feedback is challenging. To be effective, it should be: About challenging tasks or goals rather than easy ones. Given sparingly so that it is meaningful. About what is right more often than about what is wrong. Specific, accurate and clear, e.g. not just “correct” or “incorrect”. Provide examples of what is correct and not just tell students when they are wrong. Encouraging and supportive of further effort without threatening a learner’s self-esteem. @carpenter_rob robcarpenter1971.tumblr.com

12. An evidence informed improvement process Identify school priorities using internal data and professional judgement. Step 1: What do you want to achieve? External evidence summarised in the Toolkit can be used to inform decision-making. Step 2: How can you get there? Evaluate the impact of your decisions and identify potential improvements for the future. Step 3: Did it work? Mobilise the knowledge and use the findings to inform the work of the school to grow or stop the intervention. Step 4: Taking it to scale? @carpenter_rob robcarpenter1971.tumblr.com

13. An evidence informed improvement process – leadership matters Identify school priorities using internal data and professional judgement. Step 1: What do you want to achieve? External evidence summarised in the Toolkit can be used to inform decision-making. Step 2: How can you get there? Evaluate the impact of your decisions and identify potential improvements for the future. Step 3: Did it work? Mobilise the knowledge and use the findings to inform the work of the school to grow or stop the intervention. Step 4: Taking it to scale? Generating your hypothesis and designing an intervention – linked to values and school priorities Disciplined innovation– a rigorous and transparent process – building trust and confidence Publish the findings (positive or negative) and set out the scale up options and strategy. Using the DIY guide to establish the impact or effect of the change. What difference did it make and for what cost? @carpenter_rob robcarpenter1971.tumblr.com

14. Evidence can help If used in the right way, internal and external evidence can help us capture the maximum possible benefit from effort and spending. Collected over time Gathered from a range of indicators Includes an estimate of impact and cost Includes an estimate of confidence Useful internal evidence Summarises all available studies Doesn’t rely on anecdote Includes an estimate of impact and cost Includes an estimate of confidence Useful external evidence @carpenter_rob robcarpenter1971.tumblr.com

15. Four steps to improvement Think hard about learning Invest in good professional development Evaluate teaching quality Evaluate impact of changes @carpenter_rob robcarpenter1971.tumblr.com

16. How do we get students to learn hard things? Eg Place value Persuasive writing Music composition Balancing chemical equations Explain what they should do Demonstrate it Get them to do it (with gradually reducing support) Provide feedback Get them to practise until it is secure Assess their skill/ understanding @carpenter_rob robcarpenter1971.tumblr.com

17. How do we get teachers to learn hard things? Eg Using formative assessment Assertive discipline How to teach algebra Explain what they should do @carpenter_rob robcarpenter1971.tumblr.com

18. Summary … A lot of educational research is poor, but some is very good: relevant and rigorous Four steps to improve practice: Think hard about learning Invest in good CPD Evaluate teaching quality (but not with dodgy observation) Evaluate impact of changes