1. YSGOL GWYDDORAU GOFAL IECHYD / SCHOOL OF HEALTHCARE SCIENCES
PRIFYSGOL BANGOR / BANGOR UNIVERSITY
Lynne Williams, Bangor University
@SHSBangor

2.  How do we assess how and why interventions achieve (or not) outcomes?
 Are there differences between expected and observed
outcomes?
 How does context influence implementation and outcomes?
 What’s the causal pathway for the intervention?
 What are the mechanisms?
Craig P, Dieppe P, MacIntyre S et al (2008). Developing and Evaluating Complex Interventions: New Guidance. London: MRC.
Raine R, Fitzpatrick R, Barratt H et al (2016). Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Services and Delivery Research NIHR Vol 4 (16).

3.  To explain “for whom, how and why a complex intervention had a particular impact” (Liu et al, 2016)
 To open the “black box” to examine what lies beneath (the “underlying processes”) (Grant et al, 2013)
 Show how an intervention is implemented and what was delivered (as compared with what was intended to be delivered)
 Can provide a “snapshot” of receipt, implementation and context of trial intervention (Audrey et al, 2006)
Audrey S, Holliday J, Parry-Langdon N et al (2006). Meeting the challenges of implementing process evaluation within randomized controlled trials: the example of ASSIST (A Stop Smoking in Schools Trial). Health Education Research 21 (3)
Grant A, Treweek S, Dreischulte T et al (2013). Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials 14:15
Liu H, Muhunthan J, Hayek A et al (2016). Examining the use of process evaluations of randomised controlled trials of complex interventions addressing chronic disease in primary health care—a systematic review protocol Systematic Reviews 5:138

4.  Process Evaluation runs parallel with the trial to understand the processes in relation to the intervention background, setting, and people involved
 A logic model represents the intervention, and describes the resources required to ensure the implementation of the intervention, the components of the intervention, the mechanisms of impact, and the intended outcomes (UK Medical Research Council (MRC) Guidance)
 Where implementation within process evaluation is described, this often refers to “the implementation, receipt and setting of an intervention and help in the interpretation of outcomes” (Oakley et al, 2006).

6. Understanding the context
Engagement with policy makers, practitioners, patients
Understanding
the outcomes
Acknowledging the social world we live in
What are the causal mechanisms in the programme?

7.  Explanatory, contingent theory development
 Context may be defined as space or place, whereby human interaction takes place under the “appropriate social and cultural conditions” (Pawson & Tilley, 1997: 57)
 “change as the results of actions of social agents operating in a specific context whereby the action leads to outcomes by triggering mechanisms” evaluation/#realist

8. Context Mechanisms Outcomes
Changes (intended & unintended), resulting from an intervention
Context
Outcomes
Mechanisms
What leads to response?
How we respond in what ways and why?
Features which are likely
to affect how, and for
whom, a programme is expected to work?

9. Sackley CM, Walker MF, Burton CR et al
PRIFYSGOL BANGOR / BANGOR UNIVERSITY
An Occupational Therapy intervention for residents with stroke-related disabilities in
UK Care Homes (OTCH): cluster randomised controlled trial with economic evaluation
Sackley CM, Walker MF, Burton CR et al
The Health Technology Assessment 2016; Vol. 20: No. 15 DOI: /hta20150
Pragmatic Phase III, parallel-group, cluster randomised controlled trial with an
economic evaluation
Primary outcome:
The primary outcome measure was the Barthel Index (BI) score at 3 months after randomisation. The BI assesses dependency in 10 categories of self-care
Aim: The predominant aim was to perform a definitive evaluation of OT for stroke and transient ischaemic attack (TIA) survivors in long-term institutional care
Task related interventions on activities of daily living (feeding, dressing, toileting, bathing, transferring, mobilising)
Supply of adaptive equipment
Seating and positioning assessment
Care home staff training

10. Example of the realist approach as used to advance learning about process evaluation function
Process evaluation examining the fidelity of the occupational therapy intervention for residents
a programme theory of fidelity to underpin the process evaluation
multiple methods of data collection (e.g. in-depth interviews and critical incident reports)
Four potential mechanisms through which fidelity within the trial could be investigated;
the balancing of research and professional requirements
building a positive rapport with care home staff
working at re-engineering care homes’ environment
learning about the intervention and its impacts over time

11. Process evaluations alongside trials of complex interventions can illuminate the underlying processes
Provide a picture of the receipt, implementation and context of trial intervention
Drawing on principles of the realist approach can guide process evaluation in order to identify what works, for whom and in what contexts