1. Pragmatic trials at 50 – Back to the future?
Jeremy Grimshaw
Senior Scientist and professor
Canada Research Chair in Health Knowledge Transfer and Uptake
@GrimshawJeremy
9TH MAY 2017

2. Foundational paper
746 cites (SCOPUS)

3. Explanatory vs pragmatic trials
Purpose
Hypothesis testing
Inform decision making
Design
Equalised
Minimise/equalise non specific effects
Optimal
NSE seen as part of intervention
Inclusion criteria
Arbitrary
Broad
Intervention
Emphasis on intervention fidelity and optimal implementation
Prototypical elements of intervention defined but variations in intervention and uptake accepted
Data collection
Multiple end points
Fewer end points - unobtrusive

4. Explanatory vs pragmatic trials

5. Large simple trials

6. Large simple trials ISIS-2 17,187 patients in 417 centres
2 x 2 factorial design
I-v streptokinase (clot buster) or control
1 month aspirin or control
Outcome – vascular mortality at 5 weeks and 15 month follow up, harms
Highly significant reduction in vascular mortality at 5 weeks and longer follow up for both streptokinase and aspirin alone and in combination. Larger effects with earlier administration
Practice changing (but required considerable further research to determine optimal implementation and delivery approaches)

7. Large simple trials Explanatory Pragmatic Purpose Hypothesis testing
Inform decision making
Design
Equalised
Minimise/equalise non specific effects
Optimal
NSE seen as part of intervention
Inclusion criteria
Arbitrary
Broad
Intervention
Emphasis on intervention fidelity and optimal implementation
Prototypical elements of intervention defined but variations in intervention and uptake accepted
Data collection
Multiple end points
Fewer end points - unobtrusive

8. Large simple trials Drug trials Patient randomised trials
Sample size - large (thousands)
Commonly (but not exclusively) two arm trials
Limited number of clinical endpoints
Industry + public funding
Address clinical uncertainty (clinical benefits/harms)
Relatively silent on health service/implementation issues (QOL, patient experience, economic, implementation/delivery issues)

9. Health services research trials
BMJ 1994;309:979-83

10. Health services research trials
204 women randomised to hysterectomy (control) vs endometrial ablation vs endometrial resection (2:1:1)
Outcomes: operative complications, postoperative recovery, relief of menstrual symptoms, patient satisfaction at 6 and 12 months (psychological distress, longer term clinical outcomes, economic evaluation)
Hysteroscopic surgery led to fewer post op complications, shorter post op recovery, similar relief of (non bleeding) menstrual symptoms, slightly less patient satisfaction, cost savings for patient and NHS over 12 months.
Small number of women treated with hysteroscopic surgery required hysterectomy within 12 months.

11. Health services research trials
Explanatory
Pragmatic
Purpose
Hypothesis testing
Inform decision making
Design
Equalised
Minimise/equalise non specific effects
Optimal
NSE seen as part of intervention
Inclusion criteria
Arbitrary
Broad
Intervention
Emphasis on intervention fidelity and optimal implementation
Prototypical elements of intervention defined but variations in intervention and uptake accepted
Data collection
Multiple end points
Fewer end points - unobtrusive

12. Health services research trials
(Drugs), devices/technology, service delivery arrangements
Patient and cluster randomised trials
Sample size – moderate (hundreds)
Diverse designs (multi-arm, factorial, patient preference etc); greater use of process evaluations etc
Broad range of endpoints (clinical, QOL, implementation/delivery, economic)
Public funding
Address clinical and health services uncertainty

13. Implementation research trials
Lancet 2001; 357:1406-9

14. Implementation research trials
244 UK general practices served by six radiology departments
2 x 2 factorial design
Audit and feedback
Educational messages
Outcome – number of knee and lumbar spine x-rays, economic evaluation (focusing on NHS costs)
Educational messages led to with 20% reduction in x-rays; audit and feedback had no effect. Cost saving to NHS.

15. Implementation research trials
Explanatory
Pragmatic
Purpose
Hypothesis testing
Inform decision making
Design
Equalised
Minimise/equalise non specific effects
Optimal
NSE seen as part of intervention
Inclusion criteria
Arbitrary
Broad
Intervention
Emphasis on intervention fidelity and optimal implementation
Prototypical elements of intervention defined but variations in intervention and uptake accepted
Data collection
Multiple end points
Fewer end points - unobtrusive

16. Implementation research trials
Implementation strategies (including service delivery arrangements
Cluster randomised trials
Sample size – small to moderate (tens of clusters)
Diverse designs (multi-arm, factorial, patient preference etc); greater use of process evaluations etc
Broad range of endpoints (implementation/delivery, economic)
Public funding
Address health services uncertainty

17. Recent developments Research policy interest Innovative designs
Comparative effectiveness ….
Innovative designs
Stepped wedge designs
Cohort and registry embedded trials
Innovative partnerships
Collaboration for Leadership in Applied Health Research and Care (CLARHC) program
Learning healthcare organisations
Implementation laboratories

18. summary
Long tradition of pragmatic thinking and design in randomised trials
Multiple approaches adopted to address pragmatic questions addressing different uncertainties and actors
Pragmatic trials addressing clinical uncertainties may not address health service/implementation uncertainties
Opportunities to consider how to maximise information (and research efficiency) from pragmatic trials to address wider range of uncertainties
Public funding essential (and needs to be preserved!)

19. Heretical thought
Systematic reviews of randomised trials should be the knowledge unit to inform clinical and health system decision making.
Time to rethink Schwartz and Lellouch’s central argument that pragmatic trials aim to inform (local) decision making to consider how pragmatic trials efficiently contribute to global evidence production to inform decision making.

20.
@GrimshawJeremy