Presentation is loading. Please wait.

Presentation is loading. Please wait.

On Standards in Science Metrics and Classifications Henk F. Moed Elsevier, Amsterdam, Netherlands Workshop on Science Metrics, Classifications, and Mapping.

Published byDenis George Modified over 9 years ago

Similar presentations

Presentation on theme: "On Standards in Science Metrics and Classifications Henk F. Moed Elsevier, Amsterdam, Netherlands Workshop on Science Metrics, Classifications, and Mapping."— Presentation transcript:

1 On Standards in Science Metrics and Classifications Henk F. Moed Elsevier, Amsterdam, Netherlands Workshop on Science Metrics, Classifications, and Mapping Standards, August 11-12, 2011 School of Library and Information Science, Indiana University, Bloomington, USA

2 Workshop on "Bibliometric Standards", River Forest, Illinois, USA (11/06/1995) SubjectTopic Metrics (mathematical- statistical aspects) Macro indicators; journal impact factors; relative citation rates; ClassificationsDocument type; subject classifications; years; author and institutional names Terminology, concepts (theoretical aspects) Quality; impact; visibility; productivity; activity index; Research methodologyModels; normative principles; ‘Good’ analysis practices

3 Workshop on "Bibliometric Standards", River Forest, Illinois, USA (11/06/1995) 1.Standards can not be set by committee but must evolve through an on-going debate 2.Perhaps, the Scientometric community needs a refereed forum more dedicated to methodological issues 3.Often technical controversies reflect theoretical issues

4 Standardisation is (also)........ Providing frameworks in which various approaches can be positioned and compared with one another Example 1: Research assessment indicators Example 2: Subject classifications

5 Example 1: The Multi-Dimensional Research Assessment Matrix Expert Group on the Assessment of University-Based Research (AUBR, 2010)

6 Multi ‐ dimensional Research Assessment Matrix (Part) Unit of assessment PurposeOutput dimensions Bibliometric indicators Other indicators IndividualAllocate resources Research productivity PublicationsPeer review Research group Improve performance Quality, scholarly impact Journal citation impact Patents, licences, spin offs DepartmentIncrease multi-discipl. research Innovation and social benefit Actual citation impact Invitations for conferences InstitutionIncrease regional engagement Sustainabi- lity & Scale Internat. co- authorship External research income Research field Promotion, hiring Research infrastruct. citation ‘prestige’ PhD com- pletion rates

7 Multi ‐ dimensional Research Assessment Matrix (Part) Unit of assessment PurposeOutput dimensions Bibliometric indicators Other indicators IndividualAllocate resources Research productivity PublicationsPeer review Research group Improve performance Quality, scholarly impact Journal citation impact Patents, licences, spin offs DepartmentIncrease multi-discipl. research Innovation and social benefit Actual citation impact Invitations for conferences InstitutionIncrease regional engagement Sustainabi- lity & Scale Internat. co- authorship External research income Research field Promotion, hiring Research infrastruct. citation ‘prestige’ PhD com- pletion rates Read column- wise

8 Multi ‐ dimensional Research Assessment Matrix (Part) Unit of assessment PurposeOutput dimensions Bibliometric indicators Other indicators IndividualAllocate resources Research productivity PublicationsPeer review Research group Improve performance Quality, scholarly impact Journal citation impact Patents, licences, spin offs DepartmentIncrease multi-discipl. research Innovation and social benefit Actual citation impact Invitations for conferences InstitutionIncrease regional engagement Sustainabi- lity & Scale Internat. co- authorship External research income Research field Promotion, hiring Research infrastruct. citation ‘prestige’ PhD com- pletion rates

9 Multi ‐ dimensional Research Assessment Matrix (Part) Unit of assessment PurposeOutput dimensions Bibliometric indicators Other indicators Individual Allocate resources Research productivity PublicationsPeer review Research group Improve performance Quality, scholarly impact Journal citation impact Patents, licences, spin offs DepartmentIncrease multi-discipl. research Innovation and social benefit Actual citation impact Invitations for conferences InstitutionIncrease regional engagement Sustainabi- lity & Scale Internat. co- authorship External research income Research field Promotion & hiring Research infrastruct. citation ‘prestige’ PhD com- pletion rates

10 MD-RAM: Example 1  Individual  Hiring/promotion  Productivity & impact  Individual  Hiring/promotion  Productivity & impact  PhD date, place, supervisor;  Invitations for conferences  PhD date, place, supervisor;  Invitations for conferences  Publications in international jrnls;  Actual citation impact  Publications in international jrnls;  Actual citation impact

11 Multi ‐ dimensional Research Assessment Matrix (Part) Unit of assessment PurposeOutput dimensions Bibliometric indicators Other indicators Individual Allocate resources Research productivity PublicationsPeer review Research group Improve performance Quality, scholarly impact Journal citation impact Patents, licences, spin offs DepartmentIncrease multi-discipl. research Innovation and social benefit Actual citation impact Invitations for conferences InstitutionIncrease regional engagement Sustainabi- lity & Scale Internat. co- authorship External research income Research field Promotion, hiring Research infrastruct. citation ‘prestige’ PhD com- pletion rates

12 MD-RAM: Example 2  Research group  Allocate resources  Res. productivity & impact  Research group  Allocate resources  Res. productivity & impact  Competitive research income;  Ratio research active/total staff  Competitive research income;  Ratio research active/total staff  Publications in international jrnls;  Actual citation impact  Publications in international jrnls;  Actual citation impact

13 Positioning bibliometric products Unit of assessment PurposeAspect Product Research group Improve performance; Allocate resources Productivity/ impact A Department; institution Allocate resources; Stimulate multi- disciplinary research Research infrastructure and potential B InstitutionStimulate multi- disciplinary research; Improve performance; Social benefits; sustainability; Prod./impact C Group, Dept, Institution Improve performance & regional engagement; Allocate resources; Productivity/ impact; social impact; infrastr. D

14 Example 2: Journal subject classification systems

15 Journal subject classifications based on..... Semantic words in journal titles Journal-to-journal citations Journal co-citation analysis Journal bibliographical coupling Journal co-usage analysis Semantic words from article titles and abstracts Thesaurus subject terms in disciplinary databases

16 Three citation relationships A B A C A B B C A cites B B is a cited reference in A B and C are co-cited by A A and B are bibliographically coupled via C

17 Analogy Model CITATIONSUSAGE (Collections of) publishing authors (Collections of) users Citing a documentRetrieving the full text of a document ArticleUser session Author’s institutional affiliation User’s account name Number of times citedNumber of times retrieved as full text

18 Forming subject categories from journal titles Journal titleSubject category Journal of Modern Optics Optics Journal of Orthopaedic Science Orthopedics Legal and Criminological Psychology Law; Criminology; Psychology Journal of Social Work Social work

19 Subject categories: Examples “Linguistics” Nr Jrnls “Agriculture” Nr Jrnls BI- & MULTILINGUALISM12AGRICULTURAL ECONOMICS18 DIALECTOLOGY5AGRICULTURE138 DISCOURSE STUDIES10AGRONOMY26 LANGUAGE226AQUACULTURE12 LINGUISTICS217CROP SCIENCE18 LINGUISTICS, APPLIED17DAIRY SCIENCE10 PHILOLOGY38FISHERIES23 PHONETICS13FOOD S&T147 PRAGMATICS7HORTICULTURE19 SEMANTICS7PEST SCIENCE & PESTICIDE14 SEMIOTICS3POULTRY SCIENCE9

20 Proposal Use the classification based on journal title words as a benchmark....... as it represents a classification that everyone can grasp intuitively........ and illustrate the special features and added value of the new approach

21 Thank you for your attention

22 What is measured by citations and references? - 1 Principal author(s) References conceived asCitations measure Garfield Salton Descriptors of document content GarfieldManifestations of scholarly information flows Utility (quantity of information use) Small Garfield Elements in a symbol making process Highly cited items as concept symbols Merton Registrations of intellectual property and peer recognition of the knowledge clain Intellectual influence Cole & Cole Zuckerman Socially defined quality GilbertPersuasion toolsAuthoritativeness ZuckermanReferencing motives and their consequences are analytically distinct Citations are proxies of more direct measurements

23 What is measured by citations and references? - 2 Principal author(s) References conceived asCitations measure Martin & Irvine Manifestations of peer recognition, information flows and political pressures Impact (actual influence) CozzensReferences are at the intersect of the reward, rhetorical and communication system but rhetorics comes first Recognition, persuasiveness and awareness each generate a certain portion of variation in citation counts WhiteInter-textual relationships mainly reflect straightforward acknowledgement of related documents Co-citation maps provide an aerial view measuring a historical consensus as to impor- tant authors and works

24 What is measured by citations and references? - 3 Principal author(s) References conceived asCitations measure van RaanReferences are partly particularistic but in large ensembles biases cancel out The upper part of the distribution of a ‘thermodynamic’ ensemble of many citers measures ‘top’ research WoutersThe reference is the product of the scientist The citation is the product of the indexer. Validity of citations cannot be grounded merely in reference behaviour

Download ppt "On Standards in Science Metrics and Classifications Henk F. Moed Elsevier, Amsterdam, Netherlands Workshop on Science Metrics, Classifications, and Mapping."

Similar presentations

Usage statistics in context - panel discussion on understanding usage, measuring success Peter Shepherd Project Director COUNTER AAP/PSP 9 February 2005.

Usage statistics in context - panel discussion on understanding usage, measuring success Peter Shepherd Project Director COUNTER AAP/PSP 9 February 2005.
The Multi-Dimensional Research Assessment Matrix

The Multi-Dimensional Research Assessment Matrix
An application of semantic linkage technique within a CRIS Sergey Parinov, Central Economics an Mathematics Institute of RAS, Moscow.

An application of semantic linkage technique within a CRIS Sergey Parinov, Central Economics an Mathematics Institute of RAS, Moscow.
Assessing and Increasing the Impact of Research at the National Institute of Standards and Technology Susan Makar, Stacy Bruss, and Amanda Malanowski NIST.

Assessing and Increasing the Impact of Research at the National Institute of Standards and Technology Susan Makar, Stacy Bruss, and Amanda Malanowski NIST.
The building blocks What’s in it for me? Bibliometrics – an overview Research impact can be measured in many ways: quantitative approaches include publication.

The building blocks What’s in it for me? Bibliometrics – an overview Research impact can be measured in many ways: quantitative approaches include publication.
CURTIN UNIVERSITY LIBRARY Curtin University is a trademark of Curtin University of Technology CRICOS Provider code 00301J July 2014 Tell your impact story:

CURTIN UNIVERSITY LIBRARY Curtin University is a trademark of Curtin University of Technology CRICOS Provider code 00301J July 2014 Tell your impact story:
International Conference KRE-11 Prague, 9 September 2011 J.M. Verheggen, Elsevier Multi-dimensional research assessment and SciVal; a very brief overview.

International Conference KRE-11 Prague, 9 September 2011 J.M. Verheggen, Elsevier Multi-dimensional research assessment and SciVal; a very brief overview.
Lecture №2 State System of Scientific and Technical Information.

Lecture №2 State System of Scientific and Technical Information.
Scopus. Agenda Scopus Introduction Online Demonstration Personal Profile Set-up Research Evaluation Tools -Author Identifier, Find Unmatched Authors,

Scopus. Agenda Scopus Introduction Online Demonstration Personal Profile Set-up Research Evaluation Tools -Author Identifier, Find Unmatched Authors,
SCIENTROMETRIC By Preeti Patil. Introduction The twentieth century may be described as the century of the development of metric science. Among the different.

SCIENTROMETRIC By Preeti Patil. Introduction The twentieth century may be described as the century of the development of metric science. Among the different.
BIBLIOMETRICS Presented by Asha. P Research Scholar DOS in Library and Information Science Research supervisor Dr.Y.Venkatesha Associate professor DOS.

BIBLIOMETRICS Presented by Asha. P Research Scholar DOS in Library and Information Science Research supervisor Dr.Y.Venkatesha Associate professor DOS.
Disasters and Human Factors Literature Nestor L Osorio Northern Illinois University.

Disasters and Human Factors Literature Nestor L Osorio Northern Illinois University.
Www.gu.se Håkan Carlsson Gothenburg University Library Bibliometrics – A Tool in the Evaluation of Science.

Håkan Carlsson Gothenburg University Library Bibliometrics – A Tool in the Evaluation of Science.
Bibliometrics overview slides. Contents of this slide set Slides 2-5 Various definitions Slide 6 The context, bibliometrics as 1 tools to assess Slides.

Bibliometrics overview slides. Contents of this slide set Slides 2-5 Various definitions Slide 6 The context, bibliometrics as 1 tools to assess Slides.
About use and misuse of impact factor and other journal metrics Dr Berenika M. Webster Strategic Business Manager 23 January 2009, Sydney.

About use and misuse of impact factor and other journal metrics Dr Berenika M. Webster Strategic Business Manager 23 January 2009, Sydney.
Workshop on research assessment in CERIF Stephen Grace, Brigitte Jörg, Aija Kaitera, Maximilian Stempfhuber.

Workshop on research assessment in CERIF Stephen Grace, Brigitte Jörg, Aija Kaitera, Maximilian Stempfhuber.
College of Agriculture and Life Sciences WELCOME Associate Professor P&T Workshop Transitioning from Associate to Full Professor April 23, 2015.

College of Agriculture and Life Sciences WELCOME Associate Professor P&T Workshop Transitioning from Associate to Full Professor April 23, 2015.
Guillaume Rivalle APRIL 2014 MEASURE YOUR RESEARCH PERFORMANCE WITH INCITES.

Guillaume Rivalle APRIL 2014 MEASURE YOUR RESEARCH PERFORMANCE WITH INCITES.
Status of ICT structure, infrastructure and applications existed to manage and disseminate information and knowledge of Agricultural Biotechnology Innovations.

Status of ICT structure, infrastructure and applications existed to manage and disseminate information and knowledge of Agricultural Biotechnology Innovations.
SCOPUS AND SCIVAL EVALUATION AND PROMOTION OF UKRAINIAN RESEARCH RESULTS PIOTR GOŁKIEWICZ PRODUCT SALES MANAGER, CENTRAL AND EASTERN EUROPE KIEV, 31 JANUARY.

SCOPUS AND SCIVAL EVALUATION AND PROMOTION OF UKRAINIAN RESEARCH RESULTS PIOTR GOŁKIEWICZ PRODUCT SALES MANAGER, CENTRAL AND EASTERN EUROPE KIEV, 31 JANUARY.

Similar presentations

Ads by Google