1. EXPERIMENTAL PAPERS IN THE BJS – LESSONS LEARNED AND ROOM FOR IMPROVEMENT Editors Assistant Project Malin Sund 2011

2. Background The importance, relevance and quality of experimental papers in the BJS are under debate – Lack of clinical relevance – lack of interest among the BJS readership? – Do experimental papers reach the correct readership? – Are the experimental papers a “burden” for the journal and its IF? – What experimental papers are successful in a clinical journal?

3. Definition of an experimental paper All papers in which a novel (not in clinical practice) topic is explored using in vitro, animal and/or human samples

4. Materials & Methods Experimental papers in the BJS 2004-2008 were reviewed in detail, and a database constructed (145 original articles + 21 leaders/reviews) – Basic article descriptors – Field of surgery, disease/topic, aim, set-up, ethics, statistics – Evaluation of research question – clinical vs. basic science – Citation metrics Total number and toward IF Total number and toward IF in surgical journals Total number and toward IF in non-surgical journals Similar database for ASO and Annals 2007-2008 (188 original articles) Citation metrics for J Surg Res 2007-2008 Statistical analysis using non-parametric ANOVA (Kruskal-Wallis) for multiple groups and Mann-Whitney when comparing two groups

5. BJS 2004-2008 Number of experimental papers stable Number of leaders/reviews fluctuate more Average (SD) number of figures3 (1,95) tables2 (1,64) references30 (8,70) authors6 (1,45) disciplines2 (1,07) Recommendations to authors 2011: A maximum of 5 figures/tables and 30 references!

6. 2008 2004 2005 2006 2007 Where do experimental papers come from?

7. BJS 2004-2008 Issue Tend to be front-loaded, except August peak!

8. BJS 2004-2008

9. As expected the total average citations 2004 are significantly higher than 2007-2008 No significant differences in citations within 2 years 2004-2008 *** ns

10. BJS 2004-2008 No significant differences among citations among papers in relation to field of surgery (ANOVA) In pairwise comparison both GI groups were significantly more cited than Breast/Endo (p = 0.02) Most citations total and within 2 years in non-surgical journals Surgical journals cite experimental papers less Most surgical citations – general, transplantation, CIT < 2 groups

11. BJS vs. the competition The number of experimental papers is not significantly different between BJS, ASO and Annals J Surg Res has a focus on experimental surgical research and the majority of papers are experimental

12. BJS vs. the competition Both total citations and citations within 2 years (toward IF) are significantly higher for experimental papers in Annals, when compared to BJS, ASO and J Surg Res ASO has higher citations on experimental papers than BJS *** * 0,052

13. BJS vs the competition Half of all citations within 2 years. Note J Surg Res! Most citations total and within 2 years in non-surgical journals Surgical journals cite experimental papers less

14. BJS CIT ≤ 2 Only 9 (6.2%) “0-citation” papers among experimental papers in the BJS 2004-2008 24 (16%) papers with 2 or less citations (CIT ≤ 2) 68% of these citations were within 2 years (count toward IF) Top 10 - Average total citations 41,3 (range 28-80), 27% within 2 years 53 papers have ≥ 10 citations (37%), 101 ≥ 5 citations (70%) Most citations in non-surgical journals (74%) Less citations in surgical journals (26%) BEST IN SHOW

15. BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2 Field of surgery Successful experimental papers TEND to be…. “GI related translational papers on cancer, with a topic related to biomarkers, prognosis prediction or pathogenesis”

16. BJS Top 10

17. BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2 Top 10 Europe (n=113) Asia (n=29) North America (n=2) Top 1050%40%10% CIT ≥ 1070%28%2% CIT ≤ 279%21%0%

18. Experimental paper IF Do the experimental papers have a negative impact in the IF? What would the IF of BJS be based on experimental papers only? YearIFExperimental IFDifference 2006 4,0923,397 ns 2007 4,3043,985 ns 2008 4,9214,148 ns An IF factor constructed only on experimental papers is not significantly different than the actual IF of BJS

19. Questions IF of experimental papers similar to that of the BJS – but would it be better without these? – Likely not… How to predict what will be highly cited? – GI, surgical device/intervention, cancer, biomarker, translational Why is there such a difference in papers related to field of surgery? – Low numbers and/or quality on trauma, breast/endo – Does this reflect the overall low number/quality of papers within the field published in the BJS? How to cut the tail of poor papers?

20. Conclusions The overall quality of experimental papers is decent 70% have ≥ 5 citations total, and 27% ≥ 5 citations within 2 years The papers attract the majority of citations from outside the surgical readership – good/bad? Citations totally and within two years are low in surgical journals Translational papers attract most citations – perceived higher clinical relevance?

21. Possibilities Focus on translational papers with a clear clinical relevance – clinical relevance box Attract more experimental research on trauma, breast/endo GI papers highly cited - maintain position Cancer and surgical interventions/device related papers are highly cited The experiments and set-up needs to be scrutinized - Editors with specific interest in experimental research?

22. Acknowledgements Supervisor Kjetil Søreide – Tack, tack! Bibliometrics Gavin Stewart /Wiley Editorial Team for good discussions