1. Academic Publishing is Evolving… How Should We ‘Evaluate’ Scientific Publications Today? Pete Binfield Co-Founder and Publisher PeerJ Samuel Merritt - 10/30/2013 @ThePeerJ https://peerj.com @p_binfield pete@peerj.com

2. Academic Publishing is Evolving… What do we mean when we say ‘Evaluate’? Evaluating ‘Impact’ or ‘Reception’ or ‘Reach’ or ‘?’ Providing Subjective Opinions & Evaluations Evaluating ‘Integrity’ And at what level of granularity? The journal? The article? The paragraph?

3. Academic Publishing is Evolving… #1. Evaluating ‘Impact’ or ‘Reception’ or ‘Reach’ or ‘Interest’ or ‘Readership’ or, or, or

4. Academic Publishing is Evolving… Online-only, peer-reviewed, open access journals covering a very broad subject area selecting content based only on ‘technical soundness’ (or similar) with a business model which allows each article to cover its own costs Open Access ‘MegaJournals’

5. Academic Publishing is Evolving… YearPubsNotes 20071,200 Larger than ~ 95% of all journals 20082,800Largest OA journal in world 20094,4003 rd largest journal in world 20106,750Largest journal in world 201113,800~1.4% of PubMed output in that year 201223,500~2.4% of PubMed output in that year 2013~31,000>3% of the literature PLOS ONE Quarterly Output

6. Academic Publishing is Evolving… Known MegaJournals (Oct 2013)

7. Academic Publishing is Evolving… In addition, if we allow for narrow scope ‘megajournals’ then we should also include: All of the “Frontiers in…” Series (part of Nature) All of the “BMC Series” (~ half of BMC) ~ 1/3 of Hindawi’s current output All of these titles refuse to pre-judge what the audience should be reading (other than determining that the content should join the literature).

9. Academic Publishing is Evolving… An OA future containing MegaJournals PLoS ONE SAGE Open PeerJ ALL OTHER OA JOURNALS etc.

10. Academic Publishing is Evolving… The Effect of the ‘MegaJournal’ Rapidly Approaching ~10% of all published content, spurring new developments Require, and have given rise to, Article-Level Metrics Publish Negative Results, Replication Studies, Incremental Articles Dramatic Improvement to the Speed of the Ecosystem Dramatic Improvement to the Efficiency of the Ecosystem

11. Academic Publishing is Evolving… From “Article-­ ‐ Level Metrics, A SPARC Primer” - http://www.sparc.arl.org/sites/default/files/sparc-alm-primer.pdf http://www.sparc.arl.org/sites/default/files/sparc-alm-primer.pdf

12. Academic Publishing is Evolving… Screenshot from ~ Nov 2009 but Way Back Machine has examples from April 2008

13. Academic Publishing is Evolving… PLOS ALMs

14. Academic Publishing is Evolving… PeerJ ALMs

15. Academic Publishing is Evolving… http://www.altmetric.com/details.php?doi=10.7717/peerj.182

16. Academic Publishing is Evolving…

17. Rapidly Approaching ~10% of all published content, spurring new developments Require (and have stimulated) Article-Level Metrics Publish Negative Results, Replication Studies, Incremental Articles Dramatic Improvement to the Speed of the Ecosystem Dramatic Improvement to the efficiency of the way the ecosystem currently ‘filters’ content The Effect of the ‘MegaJournal’

18. Academic Publishing is Evolving… “rejected from at least six journals (including Nature, Nature Genetics, Nature Methods, Science) and took a year to publish before going on to be my most cited research paper (150 last time I looked)” – Cameron Neylon

19. Academic Publishing is Evolving… http://grigoriefflab.janelia.org/rejections

20. Academic Publishing is Evolving… http://blog.rubriq.com/2013/06/03/how-we-found-15-million-hours-of-lost-time/ “…in a recent report Kassab and his colleagues estimated that Elsevier currently rejects 700,000 out of 1 million articles each year.” http://poynder.blogspot.co.uk/2013/10/media-research-analyst-at-exane-bnp.html

21. Academic Publishing is Evolving… #2. ‘Subjective’ Opinions & Evaluations (i.e. contextual, human evaluations)

22. Academic Publishing is Evolving… PeerJ Q&A

24. Academic Publishing is Evolving… PMC Commons

25. Academic Publishing is Evolving… #3. Evaluating ‘Integrity’

26. Academic Publishing is Evolving…

30. Open Peer Review would make this problem disappear. Overnight

31. Academic Publishing is Evolving… Atmospheric Chemistry and Physics - Reviewers comments published on pre-pub discussion site. Reviewer names optional. Biology Direct - Reviewer comments published, and reviewers named BMJ Open - All reviewers named, all reports public eLife - Decision letter published with articles with author approval. Reviewers anonymous, but editor named. EMBO journal - Review process file published with articles. Reviewers anonymous, editor named. F1000Research - All reviewers named, all reports public. Frontiers journals - Reviewers named, but reports not public GigaScience - Pre-publication history published with articles, and reviewers named. (encouraged, opt-out) Medical journals in the BMC series - Pre-publication history published with articles, and reviewers named (encouraged). PeerJ - Peer review history published with articles with author approval. Reviewers encouraged to sign report. Journals Practicing Open Peer-Review

32. Academic Publishing is Evolving… ~40% of PeerJ Reviewers name themselves. ~80% of PeerJ Authors reproduce their peer review history

33. Academic Publishing is Evolving… What do we mean when we say ‘Evaluate’? Evaluating ‘Impact’ or ‘Reception’ or… Providing ‘Subjective’ Opinions & Evaluations Evaluating ‘Integrity’ And at what level of granularity? The journal? The article? The paragraph?

34. Academic Publishing is Evolving… Thank You Pete Binfield Co-Founder and Publisher @p_binfield pete@peerj.com @ThePeerJ