1. Conducting Efficacy Trials
Dr Sarah Miller | EEF Evaluators’ Conference | June 2016

2. Characteristics of an efficacy trial
A trial in which the intervention is tested under ‘ideal’ conditions and often implemented with input from the programme developer. It answers the question ‘can it work’?
Usually an earlier pilot trial has already indicated that that the intervention is feasible to implement in a classroom setting (there may also be some evidence of effects)
In fact, a good pilot trial is key to a good efficacy trial

3. Theory of change In an efficacy trial:
There is a greater focus on understanding the theory of change and underlying causal mechanism(s)
The intervention should be underpinned by a conceptually feasible and evidence informed theory of change
Consequently, relevant outcomes are selected on the basis of this logic model

4. Internal validity
An efficacy trial will have high internal validity (at this stage external validity is less crucial), that is:
Sufficient power to detect the estimated effect
Robust (and tested) allocation procedures are employed
Valid and reliable instruments are used to measure outcomes (as you would plan to in the effectiveness trial)
Data collection methods are independent, tried and tested

5. A sufficiently in-depth process evaluation
The process evaluation should be sufficiently in-depth to allow you to:
Qualitatively test the proposed theory of change
Ensure that all key agents are included in the process evaluation and use a range of methods where possible e.g. teacher survey, parent interviews, pupil focus groups
Identify issues related to scaling up the intervention e.g. any challenges associated with implementation and delivery
Understand and monitor inevitable adaptation in practice

6. A sufficiently in-depth process evaluation
Reflect and learn lessons where possible e.g.
Attrition (at school and pupil level) – avoid where possible but aim to fully understand it if and when it happens so that it can be avoided/minimised in an effectiveness trial
Efficacy trials also provide an opportunity to better understand recruitment and retention - why schools are not keen to participate or drop out post randomisation
Importance of regular and efficient communication between developers and evaluators