An example of evidence-informed policy in practice Scoping Home Access - An example of evidence-informed policy in practice Title slide of presentation. Moritz Bilagher Manager – Schools Monitoring & Evaluation Becta's Research Conference 2008 - Exploring technology-enabled change in education 6 November 2008

Evidence and policy – an old debate The idea that evidence can inform policy goes back a long way – concept of ‘statistics’ related to ‘state’ (affairs) Known problems: Data ambiguity Absence Low quality Wrong questions asked No policy initiative can be fully evidence-based / value-less

Home Access – a programme in development Aim: to provide learners in state-maintained education 5 – 19 with access to technology for learning at home Multifocal rationale including elements of social justice, education and economic benefits Official announcements at: BETT 2006: Computers for Pupils (£ 50 mln) LPC 2008: Home Access Programme

Home Access THE EVIDENCE PLAN

Sources of evidence First: Review of literature to frame expectations Second: Evaluation of CfP (NFER), 2006 – 2009 Third: Evaluation of Proofs of Concept (PwC), 2007 – 2008 Full evaluation of Home Access to start soon

Evidence pyramid Efficiency: are effects and outcomes achieved cost-effectively? Impact: does (purposeful) use lead to desired outcomes? Effectiveness: does home access lead to (purposeful) use? Implementation: devices and connectivity in place

Evidence matrix Implementation Effectiveness Impact Efficiency Literature review x Evaluation of CfP Evaluation of PoC Evaluation of HA

What did we already know? Home Access What did we already know?

Educational benefits: The fundamental case for Home Access Benefits evidenced from Test Bed and, given certain conditions, Schmidt & Wandsworth (2006); Fuchs & Wößmann (2004) Passey et al. (2004): clear impact on learner engagement and motivation

A growing number of teachers in secondary schools set electronic homework Homework requiring use of ICT A growing number of teachers in secondary schools are setting homework requiring a computer and / or internet access (in primary schools there seems to be a small regression). Is this in line with a growing number of learners having access to technology at home, or are some learners being educationally disadvantaged? Source: Harnessing Technology schools surveys 2007 & 2008

Internet take-up among households in England Internet take-up slowing down? Although overall take-up rates of internet are still increasing, especially of broadband internet, the curve seems to lose its ‘steepness’ This may be because the remaining households: Can not afford access. Do not want it. Do not know (about) it. Source: ONS report Internet access 2007: households and individuals

Barriers to Home Access Financial reasons play a role in decision not to have home access but not necessarily the main one Social barriers and fear create similar barriers

Home Access What did we find?

Evaluation of CfP (NFER) The vast majority of devices ordered were mobile Preparation for initiative took more time than expected Overall, enthusiastic reception by learners, parents and teachers Perception of fairness is key

Evaluation of PoCs (PwC) Positive synergies with other initiatives possible (eg learning platforms) Engendered engagement of parents that were hardly ‘seen’ before Some problems with administrative burdens on schools Sometimes, fragile home-school links under increased pressure

Harnessing Technology Schools Survey 2008 (NFER) School leaders on average believe around 30 per cent of learners do not have home access The main thing that is expected to change with home access is pedagogy Some schools may also change their procurement policies Can Home Access be aligned with the HT e-strategy?

How did this influence policy? Home Access How did this influence policy?

Redesign of policy I Some elements non-negotiable, eg e-safety Some elements explored but inconclusive, eg environmental sustainability Some elements beyond exploration, eg impact on achievement

Redesign of policy II Decision on formula for schools: pedagogical change yes, administrative burden no For parents with no bank accounts separate provision is established Move away from 1-2-1 although recognised that presence of a computer does not mean access

Evidence-informed policy: lessons learnt; required pre-requisites A policy side of equation willing to hear evidence-side To this end, research side must play a constructive rather than antagonistic role This relation can be mediated by a research manager Policy is shaped in multi-dimensional field where time, money, public opinion, ethical values and evidence all play a role

Contact… Moritz Bilagher E moritz.bilagher@becta.org.uk Becta Millburn Hill Road Science Park Coventry CV4 7JJ T +44 (0)24 7641 7293 F +44 (0)24 7641 1418 E becta@becta.org.uk www.becta.org.uk