Pubmed vs. Full-Text Query Performance in Systematic Reviews : Application to Non-inferiority Clinical Trials André Nguyen Van Nhieu 1, 2, Katet Moez 1, Michel Nougairede 2, Xavier Duval 4, Michaël Schwarzinger 1 1 ATIP-AVENIR Inserm “Modélisation, Aide à la Décision, et Coût-Efficacité en Maladie Infectieuses”, U738, Université Denis Diderot, Paris, France; 2 Département de Médecine Générale, Université Denis Diderot, Paris, France 3 Inserm U738, Université Denis Diderot, Paris, France; 4 Inserm CIC 007, AP-HP, Hôpital Universitaire Bichat, Paris, France 1

I declare no conflicts of interest 2

Introduction (1)  Systematic review = extensive research of appropriate publications in the literature  Usually performed through Pubmed using key-words  Methodology appropriate when keywords in title/abstract and MeSH 3

Introduction (2)  Comparison of Pubmed VS FULL-TEXT  Application: Non-inferiority trials in infectious diseases 1  According to Piaggio CONSORT statement JAMA 2006: improving quality of reporting Non-inferiority trials  Hypothesis : Pubmed as sensitive as FULL-TEXT 4

Objective  To compare the performance of 2 query strategies to identify non-inferiority trials with mortality as a primary outcome in infectious diseases: Pubmed Full-text using the search engine of each journal 5

Methods (1)  Original articles published in  In : Generalist journals : N Engl J Med, Lancet, JAMA, Ann Intern Med, BMJ, Arch Intern Med Specialist journals : Lancet Infectious Disease, Clinical Infectious Disease, Journal of Infectious Disease, AIDS, Vaccine, Pediatrics, PlosMed 6

Methods (2) 7

8

9

 Using : PUBMED : keywords : [randomi* AND (non?inferior* OR not inferior OR is inferior OR was inferior) AND (surviv* OR alive OR mortality OR death* OR fatal*)] + indexation (randomized controlled trial, Kaplan-Meier method, survival, mortality, death) FULL-TEXT method with the same keywords adapted according to each search engine 10 Methods (3)

Methods (4)  Definitions : True positive (TP): Non-inferiority trial (NIT) with mortality as primary outcome found by one or the other query strategy False positive (FP): Original article but it is not a NIT or a NIT with primary outcome ≠ mortality False negative (FN): NIT fount by one query search but not the other Positive predictive value (PPV): probability to find a TP among all Positive of a query strategy 11

Results (1) 12 Flowchart Articles excluded (n=101): - 88 articles with survival as a secondary outcome - 13 articles with per protocol analysis 256 articles in infectious diseases identified in PUBMED and FULL-TEXT : 81 in common and 175 more with FULL-TEXT Articles excluded (n= 28): - 28 NIT with outcome different from mortality Articles excluded(N=72) : - 8 Reviews - 3 Observational studies - 17 Pooled analysis - 11 SUP RCT phase SUP RCT phase equivalence trials - 2 secondary analysis - 4 follow-up of NIT 184 Non-inferiority trials 156 Non-inferiority with mortality in the outcome 55 Non-inferiority with mortality as a primary outcome = TP False positive True positive

Results (2) 13 FULL-TEXT PUBMEDSensitivityPositive Predictive Value 55/55 (100%) 19/55 (34,5%) 55/256 (21,5%) 19/45 (42,2%) False negative 0/55 (0%) 36/55 (65,5%)

Results (3)  Subgroup analysis : Sensitivity not different between generalist / specialist journals (p=0,14) Sensitivity FULL-TEXT & Pubmed not different 2008 (p=0,07) 14

Limits of FULL-TEXT method 1)Repetition in each journal search engine 2)Specificity of each journal search engine : learning curve 3)Access to journals for GP is not free 4)Time consuming 15

Conclusion (1)  FULL-TEXT is 100% sensitive  Pubmed detects only 34,5% of non- inferiority trials in infectious diseases  High rate of false positive in FULL-TEXT  However, False Positive easily identified and excluded through reading 16

Conclusion (2)  Testing the Full-text method in other medical fields or repeating in a few years (improving Pubmed?)  Improvement to be made with CONSORT statement  Adding Non-inferiority in MeSh terms 17

 THANK YOU 18

Bibliography  Le Henanff et al. / RAVAUD Quality of Reporting of Noninferiority and Equivalence Randomized Trial JAMA 2010  Piaggio et al. Reporting of Noninferiority and Equivalence Randomized Trials JAMA 2006 and Extension of the CONSORT 2010 Statement JAMA 2012