1. Research evaluation requirements José Manuel Barrueco Universitat de València (SPAIN) Servei de Biblioteques i Documentació May, 2011

2. Background Librarian at the University of Valencia (Spain) Manager of the institutional repository Co-Founder of the RePEc digital library Running an autonomous citation index for Economics (CitEc) since 2001

3. Contents 1. The current situation 2. What has been done? 3. A case study: CitEc 4. The challenge 5. The requirements

4. The current situation OAR are a new medium to distribute research results More than 1800 registered in ROAR Aim: increase the visibility and impact of the scientific literature Citation analysis used for research evaluation purposes Challenge: to apply citation analysis to contents distributed in repositories It will demonstrate the open access usefulness It will be an important added value for authors It will contribute to increase OAR contents

5. What has been done? Many research projects, not so many services Usually working with OA journals, not OAR Subject based repositories: CiteSeer arXiv: referes to + cited by RePEc: referes to + cited by + author/institutions rankings Institutional repositories Nothing has been done yet

6. A case study: CitEc It is a RePEc (Research Papers in Economics) service RePEc describes the whole discipline, not just research outputs: Institutions (departments, research institutes …) Authors Publication channels (journals, working papers series, books …) Research outputs (articles, papers) Every player in RePEc has a persistent identifier RePEc in numbers: +1,000,000 working papers + journal articles +27,000 author contact and publication listings +12,000 institutional contact listings Coming from more than 1,300 data providers

7. A citation dataset of: 6,222,142 references 2,517,198 citations 287,216 documents processed (about 30% of the available documents) CitEc produces open citation data which is used by other RePEc services to: add refence linking support Create rankings (authors, institutions … ) A case study: CitEc

8. Some examples: CitEc home page: http://citec.repec.org CitEc data used for reference linking: http://econpapers.repec.org/RePEc:ags:arerjl:31417 CitEc data used for research evaluation http://ideas.repec.org/top/ A case study: CitEc

9. Can citation analysis be applied to contents distributed in OAR? YES, WE CAN!! But …. It is not easy! If technology is not a problem (ParsCit, Xpdf), then what … ? The challenge

10. OAR are quite empty 43% of OAR listed in BASE have less than 1,000 items They are not limited to research outputs but any digital content (learning objects, cultural heritage … ) The size is not a problem We still have the subject repositories CitEc started with no more than 25,000 documents We may see the citation analysis as an opportunity to increase OAR contents. The challenge

11. Metadata issues No way to differenciate research outputs of other digital content in OARs Few OARs are providing list of references for each document E-LIS or MPRA Limited use of persistent identifiers (DOIs) No direct access to the documents full texts Documents in OAR are described as independent entities instead of representing relationship (isVersionOf, isPartOf … ). Research context gets lost The challenge

12. Researchers identification issues Researchers are the core of our efforts We need to provide them aggregated citation data from different OAR But OAR pay little attention to the identification of researchers (authors represented as not standarized strings) The big problem: author disambiguation!! Here we have an opportunity for CRIS+OAR integration The challenge

13. 1. Improve the interoperability of research OAR. 2. Improve the amount and quality of metadata 3. Guidelines for providers. But, don’t we have DRIVER? Not enought They haven’t been widely adopted, because there are not enought incentives to implement them. 4. Use of persistent identifiers for researchers in CRIS+OAR OpenID, ORCID A working alternative: AuthorClaim by Thomas Krichel The (basic) requirements

14. 5. Develop citation service providers!! Services like BASE should move forward to provide new added value services such as citation analysis There is ground for new players too The snake biting its tail!! They should provide open citation data so that it could be reused by research managers and administrators Open data != API But, such services should be expensive?! Not really, CitEc budget … 0 € The requirements

15. Thanks for your attention barrueco@uv.es