Avoiding waste in research: the role of public involvement Iain Chalmers Coordinator, James Lind Initiative ‘Putting people first in research’ INVOLVE Conference Nottingham, 13 November 2012

The skeletons in academic medicine’s cupboards

Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Questions relevant to clinicians & patients? Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation Appropriate design and methods? Over 50% of studies never published in full Biased under- reporting of studies with disappointing results Accessible full publication? Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Unbiased and usable report? 50 % 85% Research waste = over $85 Billion / year 50 %

Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Questions relevant to clinicians & patients? Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation Appropriate design and methods? Over 50% of studies never published in full Biased under- reporting of studies with disappointing results Accessible full publication? Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Unbiased and usable report? 50 % 85% Research waste = over $85 Billion / year 50 %

Mismatch of patients’ and researchers’ priorities for osteoarthritis of the knee Tallon, Chard and Dieppe. Lancet, 2000.

Priority treatment outcome from a survey of patients with rheumatoid arthritis was not pain

It was fatigue

New Healthtalkonline interview study A chance to share views/experiences of patient and public involvement in research If you’d like to contribute, contact

How can patients and the public help to reduce waste in research?

Chalmers I. Lancet 2000;356:774

Conclusions of a 2002 study commissioned by INVOLVE “There was widespread support for the concept of a publicly available resource containing information about ongoing clinical trials designed for use by potential participants.” Campbell M, Entwistle V, Langston A, Skea Z (2002). Scoping study to explore the most appropriate way to produce and disseminate information on the quality of randomised controlled trials for potential participants.

INVOLVE 6 TH National Conference 11 November 2008 The UK Clinical Trials Gateway: should provide information about ongoing clinical research in the NHS Iain Chalmers James Lind Initiative

For every ongoing trial being conducted within the NHS, the UK Clinical Trials Gateway should aim to provide access to:  a lay summary  the patient information sheet  the WHO 20-item dataset  the protocol, with links to the systematic review(s) showing why the trial is needed  the trial website (if one exists)

Reliable, user-friendly information about specific ongoing clinical trials is still NOT generally available

What is the James Lind Initiative and what has it done to try to address some of these problems?

2003- The James Lind Initiative Funded by the National Institute of Health Research and the Medical Research Council “to promote acknowledgement of uncertainties about the effects of treatments, and research to address them.”

Programme of work of The James Lind Initiative 1.Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments 2.Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance 3.Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library and Testing Treatments

The UK Database of Uncertainties about the Effects of Treatments Established to publish uncertainties about the effects of treatments which cannot currently be answered by referring to relevant and reliable, up-to-date systematic reviews of existing research evidence

Mark Fenton Editor, DUETs The UK DUETs Development Team Hazim Timimi Update Software

UK DUETs draws on three main sources Patients', carers' and clinicians' unanswered questions about the effects of treatments Research recommendations in reports of systematic reviews and clinical guidelines Ongoing research, both systematic reviews in preparation and new 'primary' studies

Programme of work of The James Lind Initiative 1.Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments 2.Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance 3.Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library and Testing Treatments

Established in 2004 to see if it would work Iain Chalmers (James Lind Library) John Scadding (Royal Society of Medicine) Nick Partridge (INVOLVE)

To increase the focus of the therapeutic research agenda on questions and priorities shared by patients and clinicians. To promote Priority Setting Partnerships involving patients and clinicians to identify and promote their shared priorities for therapeutic research. To increase general awareness and understanding of the need to refocus the therapeutic research agenda.

Sally Crowe Chair, JLA Monitoring & Implementation Group Lester Firkins Chair, JLA Strategy & Development Group Patricia Atkinson Administrator, JLA Secretariat Katherine Cowan Editor, JLA Guidebook

Involving patients, carers and clinicians in research priority setting The JLA’s principles Inclusive Balance of perspectives Accessible to all Supportive Recognising a range of capacities and skills Transparent and democratic Data sharing Agreed protocol Declaration of interests Neutral facilitation Communication and feedback

JLA Priority Setting Partnerships CompletedCurrent Asthma Urinary incontinence Vitiligo Prostate cancer Schizophrenia Type 1 diabetes ENT aspects of balance Life after stroke Eczema Tinnitus Cleft lip and palate Acne Childhood disability Dementia Dialysis Head and neck cancer Inflammatory bowel disease Lyme disease Multiple sclerosis Pressure ulcers Pre-term birth Sight loss and vision

Stage 2 Agree shortlist of highest priority uncertainties Notify highest priority uncertainties to research funders Stage 3 Stage 5 Derive ‘indicative uncertainties’ Identifying and prioritising uncertainties about the effects of treatment Stage 1 Publish ‘indicative uncertainties’ in UK DUETs Stage 4 Prioritize uncertainties, James Lind Alliance Priority Setting Partnerships Harvest ‘raw uncertainties’ 19,

Research priority themes [across asthma, incontinence, vitiligo, eczema, stroke, prostate cancer, schizophrenia, aspects of balance, and type 1 diabetes] Assessment of long-term effects (wanted and unwanted) of treatments Assessment of safety and adverse effects of treatments Assessment of complementary and non-prescribed treatments Assessment of strategies to improve early diagnosis and treatments, and harmonisation of practice Assessment of the effectiveness and safety of self- care

For more information… – become an Affiliate – practical

From 1 April 2013, the JLA will be incorporated in the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) Tom Kenny, Pamela Young, Sarah Fryett

Low priority questions addressed Important outcomes not assessed Clinicians and patients not involved in setting research agendas Questions relevant to clinicians & patients? Over 50% studies designed without reference to systematic reviews of existing evidence Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation Appropriate design and methods? Over 50% of studies never published in full Biased under- reporting of studies with disappointing results Accessible full publication? Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence Unbiased and usable report? 50 % 85% Research waste = over $85 Billion / year 50 %

Publication (2007) after registration (1999) Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009). Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 6(9): e Country Size Phase Funder

“Studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported.” PLoS ONE, August 2008;3:e3081

Alessandro Liberati

Because research results have not been made public… …patients have suffered and died unnecessarily and resources for health care and health research have been wasted.

TGN1412

Why is medical academia content to acquiesce in biased under-reporting of research?

What is the position of the Academy of Medical Sciences?

2006 letter to Prof John Bell, President, Academy of Medical Sciences

What should be done? The public needs to be made aware of how the resources they provide for research are being wasted.

What should be done? The public needs to be made aware of how the resources they provide for research are being wasted. The public needs to hold the research community to account, and be critically involved in research, from agenda setting to dissemination of results.

Programme of work of The James Lind Initiative 1.Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments 2.Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance 3.Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library and Testing Treatments

Write your suggestions on the postcards in your conference packs. Pin them up on the ‘Burning Issues’ noticeboard.

Promote research on the effects of treatments…

…but only if it meets scientific and ethical principles Promote research on the effects of treatments…

“Bad Science introduces the basic scientific principles to help everyone become a more effective bullshit detector.”