1. THE WILD AND UNWIELDY WORLD OF RETRACTIONS
Amy Riegelman, Social Sciences Librarian
Caitlin Bakker, Research Services Librarian
z.umn.edu/3cot
Library Technology Conference
March 14-15, 2018
Title: The Wild and Unwieldy World of Retractions
Description:
Scholarly publications are retracted for a plethora of reasons (e.g., irreproducible research, data fraud, fake peer review, honest error), and as shown in U.S. National Library of Medicine statistics and other sources, retraction notices have been trending upward in recent years. The Committee on Publication Ethics and other associations have guidelines for how retractions should be identified whether in a bibliographic database platform or on a journal website. Riegelman and Bakker conducted a study and found inconsistent retraction notifications across platforms. Tools like Crossmark and others are available to help scholars determine the status (e.g., retracted, corrected) of sources. Come to this session to learn about the various complexities of retractions and gain awareness of tools that could help users discover status changes. The presenters will demonstrate Crossmark, Retraction Watch Database and more!
Learning outcomes:
Attendees will be able to explain why publications may be retracted.
Attendees will be able to identify tools that could be used when attempting to identify publications that have had a status change

2. Agenda Background & Definitions Methods & Results Tools Q&A (amy)
Background and definitions - 15 min.
Methods and results - 20 min.
Tools - 10 min.
Q&A - 15 min.

3. 1. BACKGROUND & DEFINITIONS

4. Retraction: The action of withdrawing a statement, accusation, etc
Retraction: The action of withdrawing a statement, accusation, etc., which is now admitted to be erroneous or unjustified; disavowal; recantation; an instance of this; a statement making such a withdrawal.
(Amy)
What is a retraction? The Oxford English Dictionary defines it as a withdrawal or cancellation, and the action of withdrawing a statement, accusation, or other form of communication, and then now admitting it to be erroneous unjustified.
So retractions happen in many forms of communication. Today we are focusing on scholarly articles and how they appear in bibliographic platforms.
retraction, n. (2016, March). OED Online. Oxford University Press

5. HONEST ERROR CITED 13x MISCONDUCT CITED 54x
Mahajan, N., Barnes, J. L., Blanco, M., & Santos, L. R. (2009). Enumeration of objects and substances in non‐human primates: Experiments with brown lemurs (Eulemur fulvus). Developmental Science, 12(6),
CITED 13x
MISCONDUCT
Stapel, D., Suls, J., & Dovidio, John F. (2004). Method matters: Effects of explicit versus implicit social comparisons on activation, behavior, and self-views. Journal of Personality and Social Psychology, 87(6),
CITED 54x
(Amy)
One doesn’t always know why something is retracted b/c the publishers aren’t always transparent about that.
Honest Error:
In this first example, the authors from a Yale Psychology Lab filed their own retraction request due to inaccurate coding that was discovered after the article was published by Developmental Science in Retracted in According to Web of Science, this article was cited 13x including at least 2 citations AFTER the retraction. So you can kind of see the ripple effect of a both honest error retractions and more malicious retractions. In this case, there was a formal investigation at Yale that discovered that the inaccurate coding was not intentional, reckless, or negligent.
Misconduct:
This article was retracted for fabricated data, and the first listed author, Diedrik Stapel, a social psychologist, has quite the reputation for fabricating data. He has authored over 50 articles that have now been retracted.
This particular article has been cited 51 times according to Web of Science.

6. (Amy)
Retracted publications are considered a rarity, making up an estimated 0.02% of the biomedical literature (Grieneisen & Zhang, 2012), but retractions are on the rise in biomedical literature as well as other disciplines.
According to the National Library of Medicine, retraction notices in MEDLINE increased significantly between 2014 and 2015 compared to the number of retraction notices between 2013 and 2014 (Key MEDLINE® Indicators, 2016).

7. 2% to 4% of researchers admit having fabricated, falsified, or altered data or results
(Amy)
Other studies have shown that on top of the publications that are retracted due to known flawed science, surveys of researchers have found that more flawed research is yet to be discovered because between approximately 2% and 4% of researchers admit to having fabricated, falsified or altered data or results (Fanelli 2009, List et al. 2001).
Fanelli, D. (2009) How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), e5738.

8. Reasons for Retraction
21.3% error
67.4% misconduct
43.4% fraud
14.2% duplicate publication
9.8% plagiarism
(Amy)
Research shows that retractions happen for many different reasons, but according to Fang, Steen, and Casadevall (2012) in a review of the biomedical and life-science literature, some of the leading reasons include misconduct (67.4%) which included fraud (43.4%) and plagiarism (9.8%) while 21.3% of retractions were due to error (Fang, Steen, & Casadevall, 2012).
Fang, Steen & Casadevall, 2012

9. Reasons for Retraction
(Amy)
A Nature blog post back in 2011 included analysis of various studies with smaller samples, and their findings were quite different than the data on the previous slide. You’ll see here that a contingent of the retractions due to errors included irreproducible results.
Van Noorden, R. (05 Oct 2011). The reasons for retraction. Nature News Blog. Retrieved from

10. How do we find out about retractions?
Journals publish a notice
Retraction Watch (blog)
News sources (e.g., Wakefield’s now retracted autism-vaccine claim article)
Ideally: notice is eventually featured in library databases or aggregators
(Amy)
We are tipped off to retracted publications in a number of different inconsistent ways.

11. Discovering Retractions while using Library tools

12. Perspectives on similar studies
Biomedical literature focus
Focus on citation lifecycle
Validity of research & harm
Steen, R.G. (2011). Retractions in the medical literature: How many patients are put at risk by flawed research? Journal of Medical Ethics, 37: doi: /jme

13. 2.
METHODS &
RESULTS

14. The Life of an Article Article Journal Database Aggregator
Images: Document by Michael Loupos, Book by Chris Homan, Carousel by Lil Squid, Database by Shmidt Sergey

16. MEDLINE Available through… PubMed EBSCO OVID
And most MEDLINE content is also available through Scopus and Embase

17. EBSCOhost Includes content from ~80,000 publishers
Contains ~375 databases, including both citation/abstract and full-text resources
Individual databases can include thousands of journal titles

18. RETRACTED
RETRACTED
RETRACTED
RETRACTED
RETRACTED
RETRACTED

20. What We Did
Examine a study of retracted mental health literature identified through Retraction Watch
Consider how these retractions are represented across a variety of resources relevant to the discipline
See how consistently these resources abide by best practices

21. U.S. Department of Education, 2016
Why Mental Health?
18.5% of all U.S. university degrees since 1970 granted in mental health related fields
U.S. Department of Education, 2016
17.9% of all Americans deal with mental health issues annually
Center for Behavioral Health Statistics and Quality, 2016

22. “Retracted articles should be clearly identified as such in all electronic sources (e.g. on the journal website and any bibliographic databases).... The retraction should appear on all electronic searches for the retracted publication”
Committee on Publication Ethics. (September 2009). Retraction Guidelines. Retrieved from

23. Retraction Notice Characteristics
COPE Guidelines
Our Questions
Retraction notices should include identifying information from the retracted article (e.g., title, authors). Retractions should be clearly identified and distinct from other status changes (e.g., corrections) and “editors are responsible for ensuring that retractions are labelled in such a way that they are identified by bibliographic databases” (p. 2).
Is the fact that the article was retracted shown in the record?
If so, where (e.g., in the title, in the abstract)?
Is the retracted status indicated in any available PDFs?
If so, where?

24. Retraction Notice Accessibility
COPE Guidelines
Our Questions
Retraction notices should be indicated on all electronic sources such as “on the journal website and any bibliographic databases” (p. 2). Notices should link to the retracted article and “should appear on all electronic searches for the retracted publication” (p. 2).
Is there a separate retraction notice available?
If so, is that notice linked to the original article?
Is it indexed with subject headings to ensure discoverability?

25. Retraction Notice Timeliness
COPE Guidelines
Our Questions
Retraction notices should be “published promptly to minimize harmful effects from misleading publications” (p. 1).
As specific guidance is not offered surrounding what would constitute prompt publication, it was not possible to measure consistent application of this guideline.

26. What We Found 1015 observations of 144 retracted publications
812 observations were included in the analysis
7 main resources were explored:
Publisher sites
Medline via OVID
PsycINFO via OVID
4. EBSCOhost
5. Scopus
6. Web of Science
7. Medine via PubMed

27. Out of these 812 records, 487 (60%) indicated that the article had been retracted

28. Publisher %
Springer
68.2
Sage
65.6
APA
63.1
Oxford
62.5
Elsevier
58.9
Taylor & Francis
58.3
Wiley
56.6
PsychOpen
36.8
Resource %
Publisher Sites
100.0
Medine via PubMed
90.9
Medline via OVID
83.7
PsycINFO via OVID
78.8
Web of Science
71.0
EBSCOhost
5.2
Scopus
4.5

29. Out of these 812 records, 82.8% (672/812) have separate retraction notices.
54.8% (368/672) of those retraction notices are not linked to the original article or indexed with subject headings.

31. What We Found Out of 144 articles...
10 were marked retracted in all available resources
For the other 134 articles...
1/3 were not retracted in 50% or more of all resources

32. Why Does It Matter?
Insufficient detail in correcting misinformation is associated with an ongoing belief in the misinformation
Johnson & Seifert, 1994; Wilkes & Leatherbarrow, 1988
If corrections aren’t communicated at an equal or greater level to the original report, belief in the original report continues
Ecker, Lewandowsky, Swire & Chang, 2011

33. What Else Do We Know?
Clinicians pursue answers to 51% of their questions and spend less than 3 minutes searching for an answer
Del Fiol, Workman & Gorman, 2014
Titles attract the most attention in digital documents
Ohno, 2004
Users most frequently focus on titles, abstracts and keywords when determining relevance of an item in the library catalog
Janes, 1991; Marcus, Kugel & Benefeld, 1978; Wang & Soergel, 1998

34. Connecting to the ACRL Framework
Amy
I want to take a moment to further connect the results of this study to our work as librarians. For example, whether in reference consultations or classroom presentations, are you teaching students about how to differentiate between popular and scholarly resources, if yes how are you describing peer reviewed content. Are you inappropriately conveying or overpromising the the quality of scholarly content or the level of vetting?
What about library subscriptions and free discovery tools like Google Scholar or PubMed. Are you over-assuring the quality and reliability of these tools?
Perhaps the students would be better served if we were critical of our systems and upfront about infallibility of our scholarly literature and discovery systems. I want to empower you. We are qualified to help users navigate these problems, and in some ways it is our duty to explain retractions and the complexities of scholarly communication in a world where retractions, erratum and corrections exist. In a world where research errors and research fraud exist.

35. ACRL Framework Authority is Constructed & Contextual
Information Creation as a Process
Information Has Value
Research as Inquiry
Scholarship as Conversation
Searching as Strategic Exploration
Language from the ACRL Framework could be used to explain these higher level concepts. I would argue that all 6 of the threshold concepts are relevant in this discussion. In our article, we referenced the framework specifically about encouraging skepticism of the peer review and publication process and encouraging our patrons to “question traditional notions of granting authority.” This language is right out of the Authority Is Constructed and Contextual frame.
Next, Information Creation as a Process----
One of the knowledge practices is “recognize the implications of information formats that contain static or dynamic information.” This is applicable to this conversation because if you download an article to a citation manager, that’s now static and will no longer speak to a bibliographic platform. You won’t be alerted to any status changes.
Information has Value--
Research as Inquiry-- One of the dispositions is to “maintain an open mind and a critical stance;” This critically about every single resource you find from a blog post to a scholarly article. Maintain awareness of flaws. Understand that all journal articles are eligible for status changes.
Scholarship as Conversation-- Text from a disposition reads: “recognize they are often entering into an ongoing scholarly conversation and not a finished conversation”
Searching as Strategic Exploration-- Lastly, “Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops.”
Authority Is Constructed and Contextual
Information Creation as a Process
Information Has Value
Research as Inquiry
Scholarship as Conversation
Searching as Strategic Exploration

36. 3.
TOOLS

37. Crossmark
(Amy)
There is currently one tool that has some traction for identifying retractions while you are viewing an article in real-time. Crossmark was created by CrossRef, which some of you may already have familiarity. Crossref is a non-profit widely known for DOI registration. Crossmark is a logo that you possibly could see within article records, and if one clicks on the CrossMark logo, the status is revealed, and one would be notified if that version of an article is up to date or if there has been a retraction or any other update like a correction, but considering user experience, seeing Crossmark logos on documents is inconsistent because a publisher would need to be Crossref members in order to display the CrossMark logos within search results. A CrossMark logo will be visible on some publisher pages but not in other commonly used databases and aggregators. Several publishers are members of CrossMark including Elsevier BV, Springer, and Oxford University Press (Publishers & Societies, 2015). Perhaps the number of publishers participating in Crossref will continue to grow. CrossMark has potential for being a consistent way for users to identify retractions, but currently it’s not interoperable with bibliographic platforms like Ebsco, OVID, or Web of Science.
Demo:
Use a journal webpage to demonstrate the crossmark logo
Then show the same article in a library database w/o the crossmark logo
Retracted:
Not retracted:
dx.doi.org/ /

38. RetractionDatabase.org (Amy)
One could search this database if they had concerns about an article containing flawed science, or they could use this database to get an explanation of WHY an article was retracted since not all retraction notices make this clear. While this database is a nice option as a container for retracted article metadata, it would be nice if the various platforms could clearly identify retractions or at least triangulate linking to retraction notices so that one wouldn’t need to take the extra step to search a database like this to determine if an article has been retracted.
Retraction Watch Database demo:
Search for Stapel, Diederik (social psychologist who is infamous for data fraud)
Notice that you have an opportunity in viewing these results to click on the original article as well as the retraction notice. See also that the original publication dates and the retraction dates are noted.
The reasons for retraction are listed, but keep in mind that retraction notices vary in what is actually reported.
For practical purposes, perhaps you have been working on a paper for several months, and now you want to doublecheck that all of the sources on your references page are still current and not retracted. You could search for every single citation on your list.
Test the following DOI:
/S (97) (Wakefiled)
/bmj.c5347
You could analyze date ranges and search with any of the fields that you see here.

39. Learn More About Our Project
Bakker, C., & Riegelman, A. (2018). Retracted Publications in Mental Health Literature: Discovery across Bibliographic Platforms. Journal of Librarianship and Scholarly Communication, 6(1).
Riegelman, A., & Bakker, C. (2018). Understanding the complexities of retractions: Recommended resources. College & Research Libraries News, 79(1). Retrieved from

40. Thanks! Any questions? Amy || aspringe@umn.edu
Caitlin ||
Presentation template by SlidesCarnival

41. References
Association of College & Research Libraries. (2015). Framework for information literacy for higher education. Retrieved March 13, 2017, from
Bakker, C., & Riegelman, A. (2018). Retracted Publications in Mental Health Literature: Discovery across Bibliographic Platforms. Journal of Librarianship and Scholarly Communication, 6(1).
Center for Behavioral Health Statistics and Quality. (2016). Results from the National Survey on Drug Use and Health: Detailed Tables. Rockville, MD. Retrieved from DetTabs-2015/NSDUH-DetTabs-2015/NSDUH-DetTabs-2015.pdf
Committee on Publication Ethics. (September 2009). Retraction Guidelines. Retrieved from
Crossref. (2017b). Crossmark. Retrieved August 11, 2017, from /services/crossmark/

42. References
Del Fiol, G., Workman, T. E., & Gorman, P. N. (2014). Clinical questions raised by clinicians at the point of care: a systematic review. JAMA internal medicine, 174(5),
Department of Education. (2016). Digest of Education Statistics-Advance Release of Selected 2016 Digest Tables. Retrieved from
Ecker, U. K., Lewandowsky, S., Swire, B., & Chang, D. (2011). Correcting false information in memory: Manipulating the strength of misinformation encoding and its retraction. Psychonomic Bulletin & Review, 18(3),
Fanelli, D. (2009) How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), e5738.
Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028–

43. References
Janes, J. W. (1991). Relevance judgments and the incremental presentation of document representations. Information Processing & Management, 27(6), 629–646.
Johnson, H. M., & Seifert, C. M. (1994). Sources of the continued influence effect: When misinformation in memory affects later inferences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(6), 1420–
Marcus, R. S., Kugel, P., & Benenfeld, A. R. (1978). Catalog information and text as indicators of relevance. Journal of the American Society for Information Science, 29(1), 15–30.
Ohno, T. (2004, October). EyePrint: support of document browsing with eye gaze trace. In Proceedings of the 6th international conference on Multimodal interfaces (pp ). ACM.
retraction, n. (2016, March). OED Online. Oxford University Press
Retraction Watch Retraction Database. (2018). Retrieved March 13, 2018, from

44. References
Riegelman, A., & Bakker, C. (2018). Understanding the complexities of retractions: Recommended resources. College & Research Libraries News, 79(1). Retrieved from
Steen, R.G. (2011). Retractions in the medical literature: How many patients are put at risk by flawed research? Journal of Medical Ethics, 37: doi: /jme
U.S. National Library of Medicine. (2016). Key MEDLINE Indicators. Retrieved from
Van Noorden, R. (05 Oct 2011). The reasons for retraction. Nature News Blog. Retrieved from

45. References
Wang, P., & Soergel, D. (1998). A cognitive model of document use during a research project. Study I. Document selection. Journal of the American Society for Information Science, 49(2), 115– / /(SICI) (1998)49:2<115::AID-ASI3>3.0.CO;2-1
Wilkes, A. L., & Leatherbarrow, M. (1988). Editing episodic memory following the identification of error. The Quarterly Journal of Experimental Psychology, 40(2),