1. The Use & Abuse of Citation Data
Sophie Bury Peter F. Bronfman Business Library
&
Ilo Maimets
Steacie Science & Engineering Library

2. What is Citation Data?
Generated by counting numbers of times publications have been cited
Used to generate widely used impact factors e.g. journal impact factor
Another name for this field: Bibliometrics
First proposed by Eugene Garfield in 1955
Garfield founded Institute for Scientific Information (ISI)

3. 3 Kinds of Citation Data Articles Citation Impact Authors E.g. h-index
Journals
Journal Impact Factor

4. Kinds of Citation Data Citation Impact Times Cited
(Web of Science Example)

5. Kinds of Citation Data Author Impact H-index h-index:
“An index to quantify an individual’s scientific research output”
h-index:
a scholar with an index of h has published h papers with at least h citations each
Proposed by Jorge E. Hirsch in 2005, Physics Professor, University of California at San Diego
H-index
Proposed in 2005 by Jorge E. Hirsch
Intended to measure:
Quality
Sustainability of research output

6. Kinds of Citation Data Journal Impact Factor Measuring Journal Quality
Definition: The average number of times published papers are cited in the two calendar years following publication

7. The Uses of Citation Data
By Researchers:
To look for research which up-dates earlier findings
Identify related articles on same topic
To look for replications of empirical research
To look for who is citing ones own research
To follow a scholarly debate through time
Trace the development of an accepted theory over time
To identify “significant” publications or authors in a discipline

8. The Uses of Citation Data
Administrative Purposes:
Hiring, tenure & promotion
Funding / grant proposals
Augment Institutional prestige and ranking by emphasizing research in high impact fields
Eg. Open Access increases journal citation rates
Government bodies and granting agencies will often use citation stats to det4emine funding
impact factors of journals published in
citations received by grant applicants
Tenure and promotion
hiring

9. The Uses of Citation Data
Government funding agencies
Publishers / Editors
Libraries use impact factors to make collection decisions
Gov agency may want to see impact of projects funded
Overriding reasons for focus on impact factors:
to increase prestige
to attract advertising funding
to attract high profile researchers to publish with them
Publisher may encourage authors to cite papers within their publication to increase their own impact factors
They may also increase the number of review articles to increase their own impact factor.
Publishers are compelled to change their citation practices to increase their impact factors (e.g. Lancet moved in-text references within editorials and comments to a reference list increasing their journal impact factor by 35%)

10. Potential Pitfalls Problems with the finite window (2 years)
Areas where ideas turn over quickly are favoured because they publish more
More articles published per issue and more issues published per year will also increase the JIF
Journals that specialize in publishing reviews are cited more frequently
Authors that publish reviews are cited more frequently
The greater number of journals the higher the impact factor
Impact factors will be much lower for disciplines or journals with slower publication cycles (ecology is apparently slower than molecular biology) because impact factors are calculated based on a 2 year window.

11. Manipulation of Rankings
Self-citation
Journals publish more review articles to increase IF.
Some journals encourage citation from same journal

12. Resulting Effects “The rich get richer and poor get poorer”
A centrifugal effect
High IF rankings in journals are self-perpetuating
Hard for new journals to break into high rankings
Anglo-American bias
Non-English language citations undercounted

13. What Does It Really Measure?
Citation does not necessarily equal value or even usage of an article
e.g. Professional application: clinical article being used to “improve diagnoses and save lives”
Recent study proves that many references are not actually read by authors – reference lists are simply copied.

14. Sources of Citation Data
ISI Products
Web of Science (ISI)
Scopus
Google Scholar
Harzing’s Publish or Perish

15. Databases Compared: Scope/Comprehensiveness
Subjects Covered
Number of Sources Covered
Years Covered
Types of Sources Covered
Geographic Coverage
Language Coverage
A lot of the studies look at qualitiative features rather than quantitative (Gavel & Iselid, 2008)

16. Web of Science - Introduction
From Thomson Scientific
Pioneer in field
Includes:
Science Citation Index (1900-present)
Social Sciences Citation Index (1956-present)
Arts & Humanities Citation Index (1975-present)
Can search across all or limit to one or more

17. Web of Science - Scope
Best source where historic coverage is important
Citation data available from 1945 onwards for Science, 1956 for Social Sciences and 1975 for Arts and Humanities
Limited to citations from some 9,300 scholarly journals
No citations from books and few conference proceedings
Covers predominantly North American & English language titles
Offers coverage in all of the academic disciplines although is strongest in Science
Source: Jasco, P
Only game in town for a while but that’s changed significantly…

18. Scopus - Introduction Launched Nov. 2004 Elsevier product
Includes its own web search engine Scirus
Draws on Medline, Embase, Compendex, World Textile Index, Fluidex, Geobase, Biobase
Uses all of their controlled vocabularies making searching more powerful

19. Scopus - Scope Best source where international coverage is important
Citation data available from 1996 onwards (bibliographic data back to 1966)
Covers over 15,000 titles total
Broader range of types of publications than WoS
Covers more serials
Very heavy weighting toward coverage of Biomedical literature, though some social sciences and physical sciences coverage
Source: SCOPUS Help File 2008

20. Scopus - scope
Source: SCOPUS Help File 2008
Covers mainly Western European, North American, & English language titles

21. Google Scholar - Scope
Unknown number of records, unknown number of sources, unknown year coverage
Various studies conducted; some general conclusions are possible:
Over 30 different document types – especially noted for much better coverage of conference proceedings and grey literature
Coverage of social sciences, arts and humanities generally better than WoS and Scopus
Coverage of international, non-English journals better than WoS and Scopus
Sources: Meho and Yang, 2007

22. Brief Demonstration Sources of Citation Data
Web of Science
&
SCOPUS

23. Google Scholar
Citations are not of the same quality as in Wos and SCOPUS because there is no selection or indexing by subject specialists
Search functionality is very basic and limited:
Not possible to limit to scholarly high impact journals
No reliable way of searching or limiting searches
Can sort by « recent articles » but this does not result in most recent publications
Significant improvement in results display needed

24. Introducing Harzing’s Publish or Perish
Like a metasearch gateway
Draws on citation data in Google Scholar
Freely available for download on the web:
Attractive interface features

25. So which citation source should I use and when?
No one or best source (Meho and Yang, 2007, Bakkalbasi et al., 2006, Gavel and Iselid, 2008)
Overlaps vary with discipline, date range, type of publication, place of publication, etc.
Depends on the kind of analysis and searching you need to do

26. Source: Gavel & Iselid, 2008
Based on a study done in 2008 in Sweden