STIMULATE 5 Citation analysis and research performance Ronald Rousseau ronald.rousseau@khbo.be Web page: users.telenet.be/ronald.rousseau

Talk based on: Introduction to Informetrics (Elsevier) Leo Egghe and Ronald Rousseau

Citation analysis: Part I History and development of the field Structure of citation indexes Use of citations in research evaluation Problems

A reference list (example) References Drijvers, P. and Gravemeijer, K. (2004). Computer algebra as an instrument: examples of algebraic schemes. In: The didactical challenge of symbolic calculators: turning a computational device into a mathematical instrument, (D. Guin, K. Ruthven & L. Trouche, eds.). Kluwer: Dordrecht, pp. 171-206. Feller, W. (1968). An introduction to probability theory and its applications. Vol.1 (third ed.). Wiley: New York. Frandsen, T.F. and Rousseau, R. (2005). Article impact calculated over arbitrary periods. Journal of the American Society for Information Science and Technology 56(1),58-62. Ingwersen, P., Larsen, B., Rousseau, R. and Russell, J. (2001). The publication-citation matrix and its derived quantities. Chinese Science Bulletin, 46(6), 524-528. Also (in Chinese): Kexue Tongbao, 2001, 46(8), 700-704. Jiang, G., Rousseau, R., and Wu, Y. (2003). Proceedings of the 9th International Conference on Scientometrics and Informetrics. Dalian University of Technology Press, Dalian (P.R. China).

Citation analysis Leading scientist in the field: Eugene Garfield Application areas: Information search and retrieval Quantitative evaluation of scientists, institutions, countries Study of the historical development of science and technology

Citation indexing A citation index is a structured list of all references in a given collection of documents First in legal practice: Shepard’s citations Science Citation Index (SCI), Social Science Citation Index (SSCI), Arts & Humanities Citation Index (A&HCI): Web of Science (WoS)

History of citation indexes First one (SCI): covering 1961, published (in print) in 1963 Social Science Citation Index since 1972 (in print) Online versions (DIALOG) CD-ROM versions (since 1988) Nowadays: on the Web There also exist retrospective SCIs

Why is a citation index better than a subject index for retrieval? Citation indexes indicate subject relationships Authors make generally better judgments than indexers They are independent of topic descriptors Timeliness Multidisciplinarity Avoiding semantic difficulties

Citers’ motivations: ‘good’ reasons Giving credit for related work Identifying methodology, equipment, etc. Identifying original publications in which an idea was discussed or introduced Providing background reading Correcting work of others or of oneself Disputing priority claims

Citers’ motivations: ‘bad’ reasons Hat-tipping citations Citations can be selected to support one’s point of view Self-serving citations You scratch my back and I’ll scratch yours Expectations: citing works because it is felt that it is expected to do so Political considerations

Bad publication habits Serial publications (LPU’s, or salami slicing) Multiple publications Publication of luxurious reports in order to attract attention Searching out grant funding by publishing what is fashionable

Classification of citations Moravcsik and Murugesan conceptual vs. operational organic vs. perfunctory evolutionary vs. juxtapositional confirmative vs. negational + redundancy

Citers’ motivations: the work of Brooks Result: persuasiveness (Gilbert) No negative credit

Assumptions for the proper use of citation analysis in research evaluation Citation of a document implies use of that document by the citing author Citation reflects merit Citations are made to the best possible works A cited document is related in content to the citing document

Objections and problems a Self-citations Multiple authorship Homographs Synonyms Types of sources Implicit citations Fluctuations in time

Objections and problems b Field variations Incompleteness of the ISI-databases Domination of English American bias Sex bias Errors

Co-authorship First-author count (straight count) Normal count Fractional or adjusted count More elaborated counting schemes Influence of alphabetical name ordering

Some remarks Ethics of citation practices Citation amnesia Use of eponyms Problem of superspecialization Invisibility of work from developing countries

Conclusion Citation analysis is not a substitute or shortcut for critical thinking. It is a point of departure for those willing to explore the avenues to thorough evaluation. Peer review is also necessary