1. Patient and Public Involvement: Developing an Evidence Base
Carole Mockford, Research Fellow, NCI
Sophie Staniszewska, Principal Research Fellow and Director, NCI
Sandy Herron-Marx, Principal Research Fellow, NCI
Frances Griffiths, Associate Clinical Professor, Warwick Medical School

2. Aims of the presentation
To describe
why an evidence base for PPI is needed
how the review will be conducted
the next steps

3. Partnership
Academics and others specialising in PPI and systematic reviews
People Bank – paid lay members
INVOLVE
Information specialist

4. Our advisory group
Pam Attree Institute for Health Research, Lancaster University.
Rosemary Barber – Honorary senior research fellow, University of Sheffield.
Peter Beresford – Professor and director of the Centre for Citizen participation, Brunel University, London
Jo Brett – Research Fellow, University of Warwick.
Sarah Buckland – Director of INVOLVE.
Jim Elliott – Head of Research, MacMillan Cancer Support.
Tina Miller – Reader in sociology, Oxford Brookes University.
Mark Petticrew – Professor of Public Health Evaluation, London School of Hygiene and Tropical Medicine.
Maryrose Tarpey – INVOLVE.
Joy Tweed – People Bank.
Frank Palma – People Bank.

5. Why do we need an evidence base for PPI?
Patient and public involvement is becoming an increasing requirement therefore it is important to develop our conceptual, theoretical and methodological understanding.
Important to understand what works and what does not.

6. Overview of extant reviews
Evidence remains partial and lacks coherence
Problems in reviewing diverse literature.
There is some evidence that poor reporting of PPI has led to misleading results e.g. lack of emphasis or description on some potentially useful processes or important outcomes.

7. Challenges to overcome
Several problems have arisen in the reporting of PPI (Boote et al 2002:231, Oliver et al 2008:81; Daykin et al 2007:56; Crawford et al 2002: )
Difficulties relating change directly to patient involvement (Daykin et al 2007:57;Crawford et al 2002:1266)
Low quality reporting (Crawford et al 2002:1264)

8. Gaps to address
Need a sound theoretical and methodological basis to underpin the development of PPI.
Need to focus on what has not been reported on, or the lack of detail, in some papers.
Need to synthesise the existing evidence in relation to involvement of fundamental methodology to include:
definition, conceptualisation,
measurement, impact and outcomes.

9. Aims of the review The review aims to explore the: Definitions of PPI
Use of theory
Methods of involvement
Methods for capturing data and measurement
Analysis and Findings
Economic evaluation
Quality of study – reliability and validity of findings

10. Our research questions
What impact has patient and public involvement had on NHS activity (service evaluation (including audit), planning, design and provision)?
Sub questions:
How has PPI been conceptualised, defined and measured and reported?
Is there any link between methods used and ways of evaluating the impact of involvement?
On what level do the changes (if any) take place e.g. user, community, staff and organisation?

11. Three phases of the review
Phase 1 preparation for the review
Phase 2 main review stage
Robust research question
Systematic
Boundaries
Need familiarity with the literature prior to analysis
Phase 3 re-thinking the evidence

12. Phase 1 - preparation Map existing evidence Form an advisory group
Develop a protocol
Refine the research question
Develop search strategies
Decide on what data to extract

13. Phase 2 – main review Robust research question
Undertaking the search strategy
Search terms
Type of literature
Where to search
Inclusion/exclusion criteria
Data extraction
Quality criteria
Analysis – phase 3

14. Search terms to be used
Patient* User* Carer* Caregiver* Public Citizen* Client* Consumer* Lay Stakeholder* Representative* Relative* Famil* Survivor
Health* NHS “National Health Service”
Involv* Participa* Collaboration Engag* Evaluat* Consult* Audit*
Empower* Experience* Reform* Develop* Economic* Cost* Chang* Reconfig* Redesign* Impact* Outcome* Effect* Decision making Policy making Health planning Health priorities

15. Search for literature
Both quantitative and qualitative study types will be included.
Identified databases
Grey/unpublished literature
Contact with key authors

16. Published and unpublished literature
1997 to mid 2008 (to be updated early 2009 )
Medline Picker
Embase SCIE
Cinahl Conference proceedings
HMIC Theses websites
SSCI Charitable and voluntary groups websites
BNI
Cochrane
PsychInfo
Any other grey/unpublished literature
Contact with key authors
Handsearching of bibliographies

17. Inclusion and exclusion criteria
INCLUDED EXCLUDED
1997 onwards Pre 1997
English language Papers which need translation
Health care Social care
Private health care
Voluntary groups offering health care
Charitable groups offering health care
Pt/pub/cr aged 18 or over Childrens’ services or paediatrics
Involvement is a training course
Contributes to further education Contributes to Higher education
Self Management
Self Help
Individual treatment
Individual decision making about health

18. Preparation for data extraction
Abstract search
A consensus
An Endnote database will be set up to record all documents.
Full texts of relevant literature will be downloaded, printed or ordered for data extraction.

19. Data Extraction Identification features of each paper
Any definition of PPI given
Conceptual and theoretical underpinning
Study characteristics e.g. how service users were involved, how many, how long for, and in what capacity?
What was measured, if anything, and how?
Any analysis of involvement
Reporting of effects, outcomes or impact on service users, community, staff, organisation
Effectiveness, efficacy, efficiency and economic evaluation?

20. Quality of studies Difficulty in assessing diverse literature
Using a range of criteria for literature reviews, quantitative, qualitative and mixed methods approaches.

21. Phase 3 – re-conceptualisation of the evidence base for PPI
Analysis
Gaps and limitations
Re-conceptualisation of evidence
What should the future evidence base look like? .

22. Conclusions – Current state of review
Established the project team
Protocol developed
Information specialist appointed
Searching published and unpublished literature (abstract)
Member checking
Tested the data extraction mechanism
Completion due August 09

23. Conclusions - the next steps
Guidance on how/why evaluation can be built into PPI activity – what to look for etc
Developing guidance for researchers/reviewers when reporting PPI in papers
Developing a tool to measure the impact of PPI in the NHS
Looking for new ways to use other forms of evidence (e.g. patient stories). How can these inform and drive the outcomes of/ for PPI activity .

24. Contact us nhscentreforinvolvement@warwick.ac.uk
Website:

25. References
Oliver S, Rees R, Clarke-Jones L, Milne R, Oakley A, Gabbay J, Stein K, Buchanan P and Gyte G. (2008) A multidimensional conceptual framework for analysing public involvement in health services research. Health Expectations 11:
Daykin N, Evans D, Petsoulas C and Sayers A. (2007) Evaluating the impact of patient and public involvement initiatives on UK health services: a systematic review. Evidence and Policy vol 3; 1:
Boote J, Telford R and Cooper C. (2002) Consumer involvement in health research: a review and research agenda. Health Policy 61:
Crawford M, Rutter D, Manley C, Weaver T, Bhui K, Fulop N and Tyrer P (2002) Systematic review of involving patients in the planning and development of health care. BMJ 325: