EXPERIMENTAL PAPERS IN THE BJS – LESSONS LEARNED AND ROOM FOR IMPROVEMENT Editors Assistant Project Malin Sund 2011

Background The importance, relevance and quality of experimental papers in the BJS are under debate – Lack of clinical relevance – lack of interest among the BJS readership? – Do experimental papers reach the correct readership? – Are the experimental papers a “burden” for the journal and its IF? – What experimental papers are successful in a clinical journal?

Definition of an experimental paper All papers in which a novel (not in clinical practice) topic is explored using in vitro, animal and/or human samples

Materials & Methods Experimental papers in the BJS were reviewed in detail, and a database constructed (145 original articles + 21 leaders/reviews) – Basic article descriptors – Field of surgery, disease/topic, aim, set-up, ethics, statistics – Evaluation of research question – clinical vs. basic science – Citation metrics Total number and toward IF Total number and toward IF in surgical journals Total number and toward IF in non-surgical journals Similar database for ASO and Annals (188 original articles) Citation metrics for J Surg Res Statistical analysis using non-parametric ANOVA (Kruskal-Wallis) for multiple groups and Mann-Whitney when comparing two groups

BJS Number of experimental papers stable Number of leaders/reviews fluctuate more Average (SD) number of figures3 (1,95) tables2 (1,64) references30 (8,70) authors6 (1,45) disciplines2 (1,07) Recommendations to authors 2011: A maximum of 5 figures/tables and 30 references!

Where do experimental papers come from?

BJS Issue Tend to be front-loaded, except August peak!

BJS

As expected the total average citations 2004 are significantly higher than No significant differences in citations within 2 years *** ns

BJS No significant differences among citations among papers in relation to field of surgery (ANOVA) In pairwise comparison both GI groups were significantly more cited than Breast/Endo (p = 0.02) Most citations total and within 2 years in non-surgical journals Surgical journals cite experimental papers less Most surgical citations – general, transplantation, CIT < 2 groups

BJS vs. the competition The number of experimental papers is not significantly different between BJS, ASO and Annals J Surg Res has a focus on experimental surgical research and the majority of papers are experimental

BJS vs. the competition Both total citations and citations within 2 years (toward IF) are significantly higher for experimental papers in Annals, when compared to BJS, ASO and J Surg Res ASO has higher citations on experimental papers than BJS *** * 0,052

BJS vs the competition Half of all citations within 2 years. Note J Surg Res! Most citations total and within 2 years in non-surgical journals Surgical journals cite experimental papers less

BJS CIT ≤ 2 Only 9 (6.2%) “0-citation” papers among experimental papers in the BJS (16%) papers with 2 or less citations (CIT ≤ 2) 68% of these citations were within 2 years (count toward IF) Top 10 - Average total citations 41,3 (range 28-80), 27% within 2 years 53 papers have ≥ 10 citations (37%), 101 ≥ 5 citations (70%) Most citations in non-surgical journals (74%) Less citations in surgical journals (26%) BEST IN SHOW

BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2 Field of surgery Successful experimental papers TEND to be…. “GI related translational papers on cancer, with a topic related to biomarkers, prognosis prediction or pathogenesis”

BJS Top 10

BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2 Top 10 Europe (n=113) Asia (n=29) North America (n=2) Top 1050%40%10% CIT ≥ 1070%28%2% CIT ≤ 279%21%0%

Experimental paper IF Do the experimental papers have a negative impact in the IF? What would the IF of BJS be based on experimental papers only? YearIFExperimental IFDifference ,0923,397 ns ,3043,985 ns ,9214,148 ns An IF factor constructed only on experimental papers is not significantly different than the actual IF of BJS

Questions IF of experimental papers similar to that of the BJS – but would it be better without these? – Likely not… How to predict what will be highly cited? – GI, surgical device/intervention, cancer, biomarker, translational Why is there such a difference in papers related to field of surgery? – Low numbers and/or quality on trauma, breast/endo – Does this reflect the overall low number/quality of papers within the field published in the BJS? How to cut the tail of poor papers?

Conclusions The overall quality of experimental papers is decent 70% have ≥ 5 citations total, and 27% ≥ 5 citations within 2 years The papers attract the majority of citations from outside the surgical readership – good/bad? Citations totally and within two years are low in surgical journals Translational papers attract most citations – perceived higher clinical relevance?

Possibilities Focus on translational papers with a clear clinical relevance – clinical relevance box Attract more experimental research on trauma, breast/endo GI papers highly cited - maintain position Cancer and surgical interventions/device related papers are highly cited The experiments and set-up needs to be scrutinized - Editors with specific interest in experimental research?

Acknowledgements Supervisor Kjetil Søreide – Tack, tack! Bibliometrics Gavin Stewart /Wiley Editorial Team for good discussions