1. April 26, 2006 Class 9 Class 9 Tutor Professor: Dra. Cristina Santos Introdução à Medicina Did the quality of diagnostic test accuracy studies in infectious diseases improve in the last twelve years ? Systematic Review

2. April 26, 2006 Better diagnostic test Faster and better diagnostic Better treatmentBetter prognostic Improvement in patients’ quality of life Introduction

3. April 26, 2006 STARD To improve the quality of reporting of diagnostic accuracy studies the Standards for Reporting of Diagnostic Accuracy (STARD) was published. (1) The STARD project group developed a checklist of 25 items and a prototypical flow chart to represent the design of the study and the flow of patients through the study. (2) (1) Bossuyt PM et al, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy, the STARD initiative. Clinical Chemistry 2003 (2) Bossuyt PM et al, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy, the STARD initiative. Clinical Chemistry 2003Introduction

4. April 26, 2006 Aim Evaluate the quality of reporting of diagnostic studies in infectious diseases in high impact factor journals between 1995 and 2006 using the Standards for Reporting of Diagnostic Accuracy (STARD) initiative parameters. Compare the quality of reporting of diagnostic studies in infectious diseases before and after STARD  any improvement?Introduction

5. April 26, 2006 Query used on PubMed Expressions related to the evaluation of diagnostic tests accuracy (1) Expressions related to infectious diseases 13 journals: - 8 journals – table 1 - 5 journals – table 2 Exclusion of review articles and letters to the author (1) Devillé, W.L. et al.: Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Medical Research Tethodology Jully 2002 Methods

6. April 26, 2006Methods Rank Abbreviated Journal Title (linked to journal information) Impact Factor 1LANCET INFECT DIS10.008 2CLIN INFECT DIS6.510 3AIDS5.835 4EMERG INFECT DIS5.308 5ANTIVIR THER5.286 6J INFECT DIS4.953 7CURR OPIN INFECT DIS4.258 8INFECT IMMUN3.933 9J ANTIMICROB CHEMOTH3.886 10JAIDS-J ACQ IMM DEF3.871 11PEDIATR INFECT DIS J3.047 12CURR HIV RES3.000 13DIAGN MICR INFEC DIS2.738 13SEX TRANSM DIS2.738 15CLIN MICROBIOL INFEC2.679 16SEX TRANSM INFECT2.668 17J VIRAL HEPATITIS2.550 18AIDS RES HUM RETROV2.531 19INT J ANTIMICROB AG2.428 20INFECT CONT HOSP EP2.413 1. Smidt et al. Quality of Reporting of Diagnostic Accuracy Studies. Radiology, 2005. 235:350

7. April 26, 2006 Articles review Years selected: 1995 to 2006 Random selection of 20 articles / year 10 reviewers Each abstract was read by 2 reviewers Apply inclusion / exclusion criteria Disagreement  3 rd reviewer  consensus between 3 reviewersMethods

8. April 26, 2006 Exclusion criteria 1 – Doesn’t refer to infectious diseases 2 – It’s not an evaluation of diagnostic tests 3 – Other language than English or Portuguese 4 – It’s not an article Note: If the abstract isn’t available, the article should be included.Methods

9. April 26, 2006Methods Articles evaluation  STARD’s 25 items Each article is evaluated by 2 reviewers Article score = number of STARD items Average score per year

10. April 26, 2006Methods Agreement between reviewers: In abstract inclusion – PA and kappa In full text evaluation – ICC and Bland and Altman limits Check if the distribution of the items is normal One - WAY ANOVA test to compare the different biennium T-test to compare the quality of pre and post STARD studies

11. April 26, 2006 Query 2128 studies found Results

12. April 26, 2006 Articles / year 1995 – 95 1996 – 92 1997 – 114 1998 – 104 1999 – 121 2000 – 96 2001 – 125 2002 – 132 2003 – 112 2004 – 161 2005 – 143 2006 – 138 Random selection of 20 studies per year Application of exclusion criteria 1995 – 6 1996 – 8 1997 – 6 1998 – 2 1999 – 3 2000 – 7 2001 – 4 2002 – 4 2003 – 10 2004 – 5 2005 – 5 2006 – 8Results

13. April 26, 2006Results New random selection to obtain 10 articles/year 1995 – 10 1996 – 8 1997 – 10 1998 – 3 1999 – 10 2000 – 10 2001 – 10 2002 – 10 2003 – 10 2004 – 10 2005 – 10 2006 – 10 Application of exclusion criteria Total = 111 articles Articles / year

14. April 26, 2006Results Agreement between the reviewers in abstract inclusion PA = 0. 89 Kappa = 0.77

15. April 26, 2006 Results Year of publicationMean 1995 14,00 1996 13,75 1997 13,60 1998 15,67 1999 17,60 2000 17,40 2001 14,90 2002 14,00 2003 15,10 2004 18,00 2005 16,80 2006 14,70 Total 15,48 Means of the STARD score/year

16. April 26, 2006 Results Year of publicationMean 95 /96 13,89 97 / 98 14,08 99 /00 17,50 01 / 02 14,45 03 / 04 16,55 05 / 06 15,75 Total 15,48 Means of the STARD score/ biennium

17. April 26, 2006Results Agreement between the reviewers in full text evaluation ICC = 0,92 Bland and Altman plot

18. April 26, 2006Results Kolmogorov-Smirnov Test p=0.383 Normal distribution

19. April 26, 2006 p = 0.008 Differences in STARD score between years ARE statistically significantResults One-way ANOVA test

20. April 26, 2006Results Post Hoc Tests Differences between 95/96 and 99/00 ARE statistically significant (p = 0, 027)

21. April 26, 2006Results Graph of the means of the STARD score/ biennium

22. April 26, 2006Results Compare means before and after STARD T - test (independent samples) p = 0,072

23. April 26, 2006Results Presence of STARD items

24. April 26, 2006 Quality of the studies: medium Mean = 15.48 Std. Deviation = 3.648 Discussion improve between 95/96 and 99/00 didn’t improve after 2000 didn’t improve after STARD (2003)

25. April 26, 2006 50% or more of the studies analysed didn’t refer: The number of the persons executing the tests if the readers were blind (masked) beginning and ending dates of patients’ recruitment time interval from the index tests to the reference standard adverse events and indeterminate results and how they were handled Discussion

26. April 26, 2006 Discussion The items 13 and 24 13Methods for calculating test reproducibility, if done 24Measures of test reproducibility, if done These STARD items should NOT be facultative!! If the evaluations followed the STARD statement and measured the reproducibility, they would probably be better.

27. April 26, 2006 The End