1. Involving the Public in the Systematic Review Process in Health and Social Care: A Narrative Review of Eight Case Examples
Eighth International Conference on Interdisciplinary Social Sciences, Charles University, Prague
30th July – 1st August 2013
Dr Jonathan Boote, Dr Wendy Baird, Anthea Sutton
University of Sheffield, Sheffield, UK

2. Background to public involvement
Public involvement is health research policy in the UK and other developed countries
In the UK, the NIHR expects public involvement in the design and conduct of research it funds
The public includes patients, service users, carers, and groups and organisations that represent their interests
Active involvement means the public having a decision-making impact on the research process
Active public involvement is distinct from participation in the study
People actively involved are not the subjects of the research
15/04/2018
© The University of Sheffield

3. Public involvement in systematic reviews
A systematic review identifies, appraises, selects and synthesizes all high quality research evidence relevant to that question.
Systematic reviews of high-quality RCTs are crucial to EBM, and therefore they are one of the most important types of health research for the public to be involved
Importance of the Cochrane Collaboration in driving the public involvement agenda
The Collaboration includes 28,000 specialists worldwide who review RCTs of the effects of prevention, treatments and rehabilitation as well as health systems interventions
Cochrane Consumers Network exists to ensure that the public are involved in the design and conduct of Cochrane reviews
15/04/2018
© The University of Sheffield

4. Background to our narrative review
Detailed published accounts of public involvement within individual systematic reviews remain scarce
Purpose of our review was to identify and review case examples of public involvement in the design and conduct of individual systematic reviews, to:
examine the methods and stages of involvement reported
synthesise the identified contributions of involving the public
discuss the tensions and recommendations for good practice that have been identified
15/04/2018
© The University of Sheffield

5. Identifying the evidence
Health research databases were searched:
PsycINFO, MEDLINE, and MEDLINE In-Process, CINAHL, EMBASE, Cochrane Methodology Register, HMIC, invoNET
Search terms included ‘consumer’, ‘user’, ‘participant’, ‘involvement’, ‘inclusion’, ‘review’, ‘systematic review’ and ‘research’.
A total of 744 documents were identified in the search of databases, discounting duplicates
Abstracts were scrutinised for possible inclusion in the review
Eight case examples finally identified for inclusion in the review
15/04/2018
© The University of Sheffield

6. Evidence sources (1) Systematic review reference
Source describing the public involvement
Trujillo-Martín MM et al Effectiveness and safety of treatments for degenerative ataxias: a systematic review. Mov. Disord. 24(8), 1111–1124 (2009).
Serrano-Aguilar P et Paltient involvement in health research: a contribution to a systematic review on the effectiveness of treatments for degenerative ataxias. Social Science and Medicine 2009;69(6):920–5.
Rose D, Wykes T, Leese M, Bindman J,
Fleischmann P. Patients’ perspectives on
electroconvulsive therapy: systematic review. BMJ 326, 1363–1367 (2003).
Carr S et al Systematic review of consumers’ perspective on electro-convulsive therapy. In: Carr S, Coren E, editors. Collection of examples of service user and carer participation in systematic reviews. London: SCIE;
Rees R, et al. HIV Health Promotion and MSM: a Systematic Review of EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, London, UK (2004).
Rees R, An example from sexual health promotion. In: Carr S, Coren E, editors. Collection of examples of service user and carer participation in systematic reviews. London: SCIE; 2007
Braye S et al. Law in social work education: reviewing the evidence on teaching, learning and assessment. Social Work Education 24(5), 547–563 (2005).
Braye S et al, Emerging out of the shadows? Service user and carer involvement in systematic reviews. Evidence and Policy: A Journal of Research, Debate and Practice 2005;1(2):173–94.
15/04/2018
© The University of Sheffield

7. Evidence sources (2) Systematic review reference
Source describing the public involvement
Smith E et al User Involvement in the Design and Undertaking of Nursing, Midwifery and Health Visiting
Research. NCCSDO, London, UK (2005).
Smith E et all. Getting ready for user involvement in a systematic review. Health Expectations 2009;12:197–208.
Chemoradiotherapy for Cervical Cancer Meta-Analysis Collaboration. Reducing uncertainties about the effects of
chemoradiotherapy for cervical cancer: a systematic review and meta-analysis of individual patient data from 18 randomized trials. J Clin. Oncol. 26, 5802–5812 (2008).
Vale C et al Evaluation of patient involvement in a systematic review and meta-analysis of individual patient data in cervical cancer treatment. Syst. Rev. 1(1), 23 (2012).
Brett J et al The PIRICOM Study: a systematic review of the conceptualisation, measurement, impact and outcomes of patients and public involvement in health and social care research. UK Clinical Research Collaboration; 2010
Ibid
Nilsen ES et al Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database of Systematic Reviews 2006;3(September),ncbi-p: X.
15/04/2018
© The University of Sheffield

8. Identified reviews: topic area and public involved (1)
Review topic area
Members of the public involved
Treatments for degenerative ataxias (Spain)
53 patients with degenerative ataxias
Patients’ perspectives on electro- convulsive therapy (ECT) (UK)
2 service user researchers with direct experience of ECT and representatives of user and voluntary groups with a stake in ECT research
HIV health promotion for men who have sex with men (UK)
Representatives of a number of health promotion charities e.g. Terrence Higgins Trust
Chemo-radiotherapy for
women with cervical cancer (UK)
A group of five Patient Research Partners – women who have had experience of
Chemo-radiotherapy
15/04/2018
© The University of Sheffield

9. Identified reviews: topic area and public involved (2)
Review topic area
Members of the public involved
Teaching, learning and assessment of law in social work education, (UK)
15 participants identified as providing a service user or carer perspective
User involvement in nursing, midwifery and health visiting research (UK)
Members of 24 national service user organisations
Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material (Norway)
A panel consisting of members of the Cochrane Consumer Network
The conceptualisation, measurement,
impact and outcomes of public involvement in
health research (UK)
3 lay people on advisory group. Members of UNTRAP and Diabetes User Network Research Group
15/04/2018
© The University of Sheffield

10. Methods/approaches to public involvement in systematic reviews
One review (ECT) was publicly-led
Led by the Service User Research Enterprise at the Institute of Psychiatry, London
Collaborative approaches to public involvement included:
A service user being a co-reviewer
Service users as members of the review advisory group
Consultative approaches to public involvement included:
Conferences of lay people/professionals at key stages of the process
Service user reference group/patient research partnership group
Three-round Delphi process
discussion list
15/04/2018
© The University of Sheffield

11. Contributions of the public to systematic reviews (1)
Refining the scope of the review
In the review of HIV health promotion for MSM, the public identified HIV+ and young MSM as subgroups on which the review should focus
Suggesting and locating relevant literature
In the review of patients’ experiences of ECT, a user group located an unpublished study of ECT for the benefit of the review
Appraising the literature
In the review of patients’ experiences of ECT, user researchers appraised the identified literature. A template for analyzing the substantive themes was developed that ensured that no single approach to ascertaining patients’ views on ECT was privileged by the review team
15/04/2018
© The University of Sheffield

12. Contributions of the public to systematic reviews (2)
4. Interpreting the findings
In the review of chemo-radiotherapy for cervical cancer, the Patient Research Partners commented on the first results of the review and initial meta-analysis were presented. This brought the results of the study to life as it evidenced the experience of real people’
5. Writing up the review
Three reviews reported the public being involved in the writing up stage of the review process: either as first author (ECT), co-author (user involvement in midwifery research), or as the author of the review’s foreword (conceptualisation, measurement and impact of public involvement in research)
15/04/2018
© The University of Sheffield

13. Tensions identified when involving the public in the review process (1)
Time pressures
It can take time to build up trust between researchers and the public in the review process
Resourcing problems
Lack of funding at the review development (pre-grant) stages to support involvement
Continuity issues
It can be difficult for the public to attend all review meetings
Concerns about group dynamics
The user-led review of experiences of ECT reported tensions regarding the amount of power the user-researchers had in the review process
15/04/2018
© The University of Sheffield

14. Tensions identified when involving the public in the review process (2)
Research Ethics Committee involvement
One review reported that they had to obtain approval from a REC to involve the public in the review process
Tensions associated with the representativeness of the members of the public involved
Some reviews mentioned that those who get involved in reviews will not be representative of the wider population
Researchers’ concerns about involving people who are not well & discussing ‘taboo’ subjects
Chemo-radiotherapy review team: We had no idea how well or poorly the women were going to be or whether any of them were coming into it with preconceptions that would make it difficult to work with them”
15/04/2018
© The University of Sheffield

15. Good practice for involving the public in systematic reviews
Funding and payment
The public should be paid for the time they spend contributing to reviews and that this should be budgeted for
Identifying a lead or advisory group for public involvement
This person/group should be the contact point for the public and this role should be costed into the review budget
Training, briefing and information provision
Provision of training, briefing notes and background information (such as a glossary) has been recommended.
Structured methods of involvement
Delphi or NGT prevent review advisory groups being dominated by particular, more vocal individuals.
15/04/2018
© The University of Sheffield

16. Conclusions and implications
Only eight detailed case examples were found.
Only one-publicly-led review found. Most review examples used consultative/collaborative methods of involvement
The public was found to be involved in all stages of the review process, from scoping the review to dissemination
Review commissioners need to recognise the cost and time implications of public involvement
Financial support is needed to resource public involvement at the design (pre-grant) stage of systematic reviews
Abstracts of systematic reviews should describe the extent of public involvement (if any) in the review process to facilitate more accurate future reviews of the evidence base
15/04/2018
© The University of Sheffield

17. Further reading
Boote J, Baird W, Sutton A (2012)  Involving the public in systematic reviews: a narrative review of organisational approaches and eight case examples.  Journal of Comparative Effectiveness Research, 1(5),
Boote J, Baird W, Sutton A. (2011) Public involvement in the systematic review process in health and social care:  a narrative review of case examples.  Health Policy. 102,
15/04/2018
© The University of Sheffield

18. Contact details: Dr Jonathan Boote Research Fellow and PPI lead
Design, Trials and Statistics
School of Health and Related Research
University of Sheffield
Regent Court
30 Regent Street
Sheffield
S1 4DA
Tel:
15/04/2018
© The University of Sheffield