Transparent and accurate reporting increases reliability, utility and impact of your research: Reporting guidelines and the EQUATOR Network Iveta Simera The EQUATOR Network Centre for Statistics in Medicine, Oxford, UK 13 November 2010

2 Biomedical research & its publication Biomedical research should advance scientific knowledge and – directly or indirectly – lead to improvements in prevention or management of illness Publications are usually the only tangible evidence that a study was done, how it was done, and what the findings were –PubMed: more then 20 mil citations (500,000 added each year on average = around 1 per minute)

3 Purpose of research publication Large number of articles but are they reliable? Can they be used? Articles are written for multiple readerships: –Clinicians To learn how to treat their patients better –Researchers: To inform their own research To help plan a similar study To include the study in a systematic review –Patients/consumers: To aid personal decision-making –Policy makers/purchasers: To aid policy decision-making … should present sufficiently detailed information to allow assessment of study reliability and relevance and comparison across studies (important for SR and CPG)

4 Much evidence of poor reporting Hundreds of reviews of methodology of published research articles –Highlighting severe deficiencies in reporting (biased or unusable research reports) –These deficiencies limit or prevent use of many of the published findings further in research or clinical practice

5 Examples: Poor reporting of RCT methodolgoy 519 RCTs published in Dec 2000: failing to report key aspects of methodology –79% did not report the method of random sequence generation * –82% did not report the method of allocation concealment * –73% did not report a sample size calculation * –55% did not report a defined primary outcome (s) –60% did not report whether blinded (Chan & Altman Lancet 2005) 616 RCTs published in Dec 2006: –Reporting of several important aspects of trial methods improved between 2000 and 2006 (*); however, the quality of reporting remains well below an acceptable level. (Hopewell BMJ 2010)

6 Example: Poor description of intervention Glasziou et al. (BMJ 2008) –assessed descriptions of treatments in 80 published articles (55 randomised trials & 25 systematic reviews) –crucial elements of the interventions were missing in 41 of those studies (of 25 SR only 3 provided intervention description sufficient for implementation) Herbert & Bo (BMJ 2005) –Discuss effect of quality of intervention on SR –“Assessment of the quality of the intervention relies on sufficient detail in trial reports, but many reports provide only superficial description of complex interventions... Interventions should be described in sufficient detail to enable reader to assess if the intervention was administered well as the quality of intervention can affect the results of clinical trials.”

7 Systematic reviews and clinical guidelines Poor reporting is a serious problem for SR and CPG In the UK – Cochrane reviews are highly regarded and yet.. –“The biggest problem was the quality of reporting, which did not allow us to judge the important methodological items...” –“Data reporting was poor. 15 trials met the inclusion criteria for this review but only 4 could be included as data were impossible to use in the other 11.” –“If the CONSORT recommendations were followed in the reporting of future studies, the effects of Morita therapy would be clearer. Much important data within the included studies were so poorly reported that clinicians, funders and recipients of care might have reason to feel let down by the research community.” (Cochrane Library, accessed on 18 Sept 07)

8 Examples: Poor reporting of systematic reviews Reviews are not immune to the problems of poor reporting –Epidemiological and reporting characteristics and bias-related aspects of 300 systematic reviews (of which 125 were Cochrane reviews). The overall quality of reporting of key aspects of methodology was very inconsistent with particularly discouraging findings for non-Cochrane reviews. ( Moher; PLoS Med, 2007, 4(3), e78)

9 Common problems in research reporting Unethical reporting practices with serious adverse consequences –Non-reporting or delayed reporting of whole studies –Omissions or misinterpretation of results in abstracts –Omission of crucial information in the description of research methods and interventions –Inconsistencies between study protocol and publication –Selective reporting of only certain outcomes –Inadequate statistical reporting –Presenting data (graphs) in confusing or misleading ways (particularly important for presenting benefits and harms) –General misinterpretation of study results (spin)

10 How to improve research reporting?

11 How to improve research reporting?

12 How to improve research reporting?

13 Reporting guidelines (RG) RG provide structured advice on what to include in a research report RG specify a minimum set of items required for a clear and transparent account of what was done and what was found in a research study, reflecting in particular issues that might introduce bias into the research Benefits of using RG: –Improved accuracy and transparency of publications –Easier appraisal of reports for research quality and relevance –Better further use of presented findings –Improved efficiency of literature searching

14 RG example: CONSORT Statement

15

16

17 Other reporting guidelines PRISMA (SR/meta-analyses of RCTs) STARD (diagnostic studies) STROBE (observational studies)... and many others (over 100 RG identified) As yet most guidelines have had limited impact –Passive dissemination through publication only - not widely known –Complience not required by journals

18 EQUATOR Network EQUATOR Network is an international initiative set up to improve reliability and value of medical research literature EQUATOR promotes transparent accurate complete and timely reporting of health research studies

19 EQUATOR main goals and activites Website launched in Oct EQUATOR seven major goals Promote responsible research reporting in practice (wider use of RG) Develop a comprehensive online resource centre Develop and establish an education and training programme Assist in the development, dissemination and implementation of robust reporting guidelines Expand EQUATOR activities globally Assess use of reporting guidelines Audit reporting quality across the health research literature

20 EQUATOR online resources All collated resources are available in our Library Some of the resources are grouped according to relevance to our main user groups

21 Resources for editors

22 Guidance on reporting in journals’ I to A: examples General medical journals: Prakticky lekar – –Very brief and general instructions on research reporting BMJ (British Medical Journal) – article/researchhttp://resources.bmj.com/bmj/authors/types-of- article/research –Very detailed, clearly worded requirements and links to internationally accepted guidelines

23 Prakt Lekar – I to A

24 BMJ - I to A

25 Guidance on reporting in journals’ I to A: examples Specialty (Association) journals: Cesko-Slovenska Dermatologie – dermatologie-pokynyhttp:// dermatologie-pokyny –Some instructions on reporting scientific content, no reference to any internationally accepted RG British Journal of Dermatology – 0963&site=1http:// 0963&site=1

26 Cesko-Slovenska Dermatologie

27 BJD

28 How to shift the ‘reporting culture’ Collaboration of all parties involved in research publishing needed on a global scale –Scientists, research organisations, funders and regulators –Journals (editors, peer reviewers, publishers) –Other organisations (higher education, REC,..) Working towards: –.. accurate, complete, transparent and timely reporting of research studies is considered a ‘norm’ (not something extra) How to achieve this? –Clearly defined policies, requirements and expectations –Provision of tools and other resources –Education and training –Motivation and incentives –Application of safeguards and checks See ‘Steps to consider’ in Simera et al. BMC Medicine 2010,8:24

29 EQUATOR on a global scale Last year, our website has been visited from most countries in the world Traffic doubled compared with the previous year

30 EQUATOR Spanish website – launched July 2010 We are looking for collaborators to establish local centres of activities supporting better reporting of research studies Signed collaboration agreement with PAHO to raise standards of research reporting in South America and Caribbean

31 Czech medical research contributes to SR More difficult to write a good paper (although many really bad papers were written by English native speakers!) –Important is a good understanding of principles of good research methodology (including terminology) and principles of responsible reporting –Translating educational materials (e.g. E&E papers) into ‘local’ languages can improve understanding -Selected for inclusion in Cochrane SR of RCTs -No information about pts allocation into groups -6 s with the authors -Final answer: -“It was a double-blind study”

Dr Iveta Simera, Head of Programme Development EQUATOR Network, Centre for Statistics in Medicine, Oxford, UK

33 EQUATOR Steering Group Doug Altman, Centre for Statistics in Medicine, University of Oxford, UK John Hoey, University of Toronto, Canada Ana Marusic, University of Split, Croatia David Moher, Ottawa Health Research Institute, Canada Kenneth F. Schulz, Family Health International, Chapel Hill, USA