CitEc as a source for research assessment and evaluation José Manuel Barrueco Universitat de València (SPAIN) May, й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

Citations and citation indexes Citations are indicators of the quality of a publication and therefore a key piece in any execise of research assesment Bibliometric indicators (impact factor, h-index) are used (and often misused) for research evaluation purposes Citation-based measures should complement expert peer-review and never supplant it Research evaluation requires citation indexes in order to provide current and accurate citation data For many years the only source of citation data were the indexes published by ISI (now Thomson Reuters) Fortunately the landscape has changed … 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

Citations and citation indexes The big players: Thomson’s Web of Science and Scopus by Elsevier Scholarly databases: PubMed or those from Cambridge Scientific, EBSCO, … Intermediate services: Mendeley, PlumX or KUDOS are using citation data extracted from Scopus, WoS or CrossRef in their products Social research networks: ResearchGate Repositories: arXiv in physics or SSRN in social sciences Atonomous citation indexes: CiteSeer in computer science CitEc in economics Google Scholar Citations 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

A few words about RePEc RePEc (Research Papers in Economics) is a digital library RePEc describes the whole discipline: Research outputs (articles, papers, chapters, books) Publication channels (journals, working papers series …) Researchers Institutions (departments, research institutes …) RePEc architecture: data providers vs. service providers RePEc in numbers: +2,000,000 research outputs Coming from more than 1,300 data providers University departments, international institutions, banks, publishers … +46,000 author contact and publication listings 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

CitEc : an autonomous citation index It is a RePEc service Processes, extracts and disseminates citation data for RePEc A citation dataset of: +22,000,000 references +8,000,000 citations +812,000 documents processed (about 40% of the available documents) CitEc produces open citation data As other RePEc services, CitEc is based on volunteer work No permanent funding Server sponsored by INOMICS Funds raised since : 525 EUR 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

CitEc : an autonomous citation index Data generation: All RePEc documents could become cited documents 40% of the available documents are citing documents: Full text not available: not in electronic format, not open access, not at the url provided … Unable to process full text to extract references (i.e. digitized documents) Data input: PDFs provided by data providers Need to convert PDF files to text in order to extract references Process prone to errors: references lost, incorrect interpretation of characters … Data providers need to do nothing about it XML structured metadata about references Good quality results Only large publishers (Elsevier or Springer) are able to cope with this format Nonetheless, around 60% of the references available come this way 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

Quality control: It relies on the reserchers community Involves detecting and reporting wrong or missing content Transparency and publicity of the data Reporting of errors as easy as posible at reference and document levels Registered authors may: Add references lists. Submitter data is logged and shown in web pages Identify missing citations to their works. Only applies to suspect citations identified by the system. Author may claim or refuse them Any CitEc user may: Add missing citations to any document. All data is checked for correctness 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016 CitEc : an autonomous citation index

reporting errors add citation references contributed by researcher

Research evaluation trougth CitEc Citation data cleaned in order to Remove self-citations Aggregation of cites between different versions of the same document Evaluation at three levels: Documents Researchers Institutions Two evaluation tools Citations profiles Rankings 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

Research evaluation Citation profiles Authors (46K profiles) Research production: number of works published and their distribution by year and document type; books and journals edited; journals or series where she publishes. Citation counts: total number of citations, unbundled number of citations by work, recent citing documents, total and percentage of self-citations. Relationship with other researchers: co-authors, who is citing him or to whom he cites. Basic bibliometric indicators like h-index and i10-index. Series (2K journals and 4K working papers series) Production and citation count data in time series that could be back to 1990 Bibliometric indicators: impact factor, immediacy index and h-index Institutions To come!! 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

Research evaluation The Martin Srholec’s case: 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

main data research outputs research activity additional info

details

main data

popularity citation data

Research evaluation trougth CitEc Rankings Allow easy comparation of individuals and institutions based in objective indicators Created by Christian Zimmermann Based in citations and popularity (abstract views and paper downloads) Rankings created according to 31 different criteria Aggregation of criteria based in the harmonic mean of the rank orders Additional rankings for: top women, young or deceased economists; top researchers by region, country or research field… 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016

That’s all! Thanks for your attention 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», мая, 2016