The Use & Abuse of Citation Data Sophie Bury Peter F. Bronfman Business Library & Ilo Maimets Steacie Science & Engineering Library

What is Citation Data? Generated by counting numbers of times publications have been cited Used to generate widely used impact factors e.g. journal impact factor Another name for this field: Bibliometrics First proposed by Eugene Garfield in 1955 Garfield founded Institute for Scientific Information (ISI)

3 Kinds of Citation Data Articles Citation Impact Authors E.g. h-index Journals Journal Impact Factor

Kinds of Citation Data Citation Impact Times Cited (Web of Science Example)

Kinds of Citation Data Author Impact H-index h-index: “An index to quantify an individual’s scientific research output” h-index: a scholar with an index of h has published h papers with at least h citations each Proposed by Jorge E. Hirsch in 2005, Physics Professor, University of California at San Diego H-index Proposed in 2005 by Jorge E. Hirsch Intended to measure: Quality Sustainability of research output

Kinds of Citation Data Journal Impact Factor Measuring Journal Quality Definition: The average number of times published papers are cited in the two calendar years following publication

The Uses of Citation Data By Researchers: To look for research which up-dates earlier findings Identify related articles on same topic To look for replications of empirical research To look for who is citing ones own research To follow a scholarly debate through time Trace the development of an accepted theory over time To identify “significant” publications or authors in a discipline

The Uses of Citation Data Administrative Purposes: Hiring, tenure & promotion Funding / grant proposals Augment Institutional prestige and ranking by emphasizing research in high impact fields Eg. Open Access increases journal citation rates Government bodies and granting agencies will often use citation stats to det4emine funding impact factors of journals published in citations received by grant applicants Tenure and promotion hiring

The Uses of Citation Data Government funding agencies Publishers / Editors Libraries use impact factors to make collection decisions Gov agency may want to see impact of projects funded Overriding reasons for focus on impact factors: to increase prestige to attract advertising funding to attract high profile researchers to publish with them Publisher may encourage authors to cite papers within their publication to increase their own impact factors They may also increase the number of review articles to increase their own impact factor. Publishers are compelled to change their citation practices to increase their impact factors (e.g. Lancet moved in-text references within editorials and comments to a reference list increasing their journal impact factor by 35%)

Potential Pitfalls Problems with the finite window (2 years) Areas where ideas turn over quickly are favoured because they publish more More articles published per issue and more issues published per year will also increase the JIF Journals that specialize in publishing reviews are cited more frequently Authors that publish reviews are cited more frequently The greater number of journals the higher the impact factor Impact factors will be much lower for disciplines or journals with slower publication cycles (ecology is apparently slower than molecular biology) because impact factors are calculated based on a 2 year window.

Manipulation of Rankings Self-citation Journals publish more review articles to increase IF. Some journals encourage citation from same journal

Resulting Effects “The rich get richer and poor get poorer” A centrifugal effect High IF rankings in journals are self-perpetuating Hard for new journals to break into high rankings Anglo-American bias Non-English language citations undercounted

What Does It Really Measure? Citation does not necessarily equal value or even usage of an article e.g. Professional application: clinical article being used to “improve diagnoses and save lives” Recent study proves that many references are not actually read by authors – reference lists are simply copied.

Sources of Citation Data ISI Products Web of Science (ISI) Scopus Google Scholar Harzing’s Publish or Perish

Databases Compared: Scope/Comprehensiveness Subjects Covered Number of Sources Covered Years Covered Types of Sources Covered Geographic Coverage Language Coverage A lot of the studies look at qualitiative features rather than quantitative (Gavel & Iselid, 2008)

Web of Science - Introduction From Thomson Scientific Pioneer in field Includes: Science Citation Index (1900-present) Social Sciences Citation Index (1956-present) Arts & Humanities Citation Index (1975-present) Can search across all or limit to one or more

Web of Science - Scope Best source where historic coverage is important Citation data available from 1945 onwards for Science, 1956 for Social Sciences and 1975 for Arts and Humanities Limited to citations from some 9,300 scholarly journals No citations from books and few conference proceedings Covers predominantly North American & English language titles Offers coverage in all of the academic disciplines although is strongest in Science Source: Jasco, P. 2005 Only game in town for a while but that’s changed significantly…

Scopus - Introduction Launched Nov. 2004 Elsevier product Includes its own web search engine Scirus Draws on Medline, Embase, Compendex, World Textile Index, Fluidex, Geobase, Biobase Uses all of their controlled vocabularies making searching more powerful

Scopus - Scope Best source where international coverage is important Citation data available from 1996 onwards (bibliographic data back to 1966) Covers over 15,000 titles total Broader range of types of publications than WoS Covers more serials Very heavy weighting toward coverage of Biomedical literature, though some social sciences and physical sciences coverage Source: SCOPUS Help File 2008

Scopus - scope Source: SCOPUS Help File 2008 Covers mainly Western European, North American, & English language titles

Google Scholar - Scope Unknown number of records, unknown number of sources, unknown year coverage Various studies conducted; some general conclusions are possible: Over 30 different document types – especially noted for much better coverage of conference proceedings and grey literature Coverage of social sciences, arts and humanities generally better than WoS and Scopus Coverage of international, non-English journals better than WoS and Scopus Sources: Meho and Yang, 2007

Brief Demonstration Sources of Citation Data Web of Science & SCOPUS

Google Scholar Citations are not of the same quality as in Wos and SCOPUS because there is no selection or indexing by subject specialists Search functionality is very basic and limited: Not possible to limit to scholarly high impact journals No reliable way of searching or limiting searches Can sort by « recent articles » but this does not result in most recent publications Significant improvement in results display needed

Introducing Harzing’s Publish or Perish Like a metasearch gateway Draws on citation data in Google Scholar Freely available for download on the web: http://www.harzing.com/pop.htm Attractive interface features

So which citation source should I use and when? No one or best source (Meho and Yang, 2007, Bakkalbasi et al., 2006, Gavel and Iselid, 2008) Overlaps vary with discipline, date range, type of publication, place of publication, etc. Depends on the kind of analysis and searching you need to do

Source: Gavel & Iselid, 2008 Based on a study done in 2008 in Sweden