1. Competências Básicas de Investigação Científica e de Publicação Lecture 8: Global science and the need for metrics 05/06/2013Ganesha Associates 2013

2. Brasil’s economic growth creates a demand for scientists and technologists 05/06/2013Ganesha Associates 2013

3. Science drives the economy “The ability to create, distribute and exploit scientific and technological knowledge has become a major source of competitive advantage, wealth creation and improvements in the quality of life. Some of the main features of this transformation are; – the growing impact of information and communications technologies (ICT) on the economy and on society – the rapid application of recent scientific advances in new products and processes – a high rate of innovation across economically developed countries – a shift to more knowledge-intensive industries and services – rising skill requirements.” Source: Organisation for Economic Co-operation and Development (OECD) Policy document 2000Organisation for Economic Co-operation and Development (OECD) 05/06/2013Ganesha Associates 2013

4. 05/06/2013Ganesha Associates 2013

5. Global research investment 28 August 2012Ganesha Associates 5

6. Thomson Reuters announce strategic partnership to boost Vietnam’s output Thomson Reuters, a leading source of intelligent information for businesses and professionals, and the National Agency for Science and Technology Information (NASATI) of Vietnam, have announced a strategic partnership to advance the initiatives of the Ministry of Science and Technology (MOST) in accelerating Vietnam's science and technology (S&T) development. NASATI is Vietnam's leading S&T information organisation. As a subordinate body under the Ministry of Science and Technology (MOST), NASATI is responsible for the state management and implementation of Vietnam's S&T information, library and statistics for best practices and evaluation metrics, as well as other public services. It is implementing these national initiatives on behalf of MOST. The partnership will leverage Thomson Reuters capabilities of content, analytics and technologies to support the initiatives of MOST in developing Vietnam's potential in S&T, and the research and development (R&D) ecosystem. The scope covers decision-making tools; conducting workshops for government agencies, research institutions and universities on topics including best practices in research discovery, research evaluation metrics, knowledge commercialisation process, innovation lifecycle management; and supporting working groups on specific R&D areas to inform Vietnam's national R&D strategy and policy. Thomson Reuters

7. 05/06/2013Ganesha Associates 2013

8. 05/06/2013Ganesha Associates 2013

9. 05/06/2013Ganesha Associates 2013

10. Document output in 2011 05/06/2013Ganesha Associates 2013

11. Brasil: 05/06/2013Ganesha Associates 2013

12. Source: International Comparative Activity Performance – Elsevier 2011 BRIC output: documents 05/06/2013Ganesha Associates 2013

13. Citation quality is a problem 05/06/2013Ganesha Associates 2013

14. International collaboration is low 05/06/2013Ganesha Associates 2013

15. Share of articles by co-authorship type 05/06/2013Ganesha Associates 2013

16. EPI : English Proficiency Index by age

17. 05/06/2013Ganesha Associates 2013

18. Is " Science without Frontiers” the solution? Send 100,000 students abroad by end of 2015 <30% of students meet minimal English language requirements before departure Necessary to introduce crash English language courses abroad Ultimately, such a policy will work, but at what cost?

19. A more efficient solution One of the main reasons that English is so weak and under used in Brazilian universities is that there are few English speaking visitors If every lab had a foreign visitor then all lab meetings would have to be conducted in English It would be more efficient and cost effective to entice visitors to come here rather than send so many Brazilians abroad

20. Institutional ranking 05/06/2013Ganesha Associates 2013

21. O Qualis é um levantamento realizado pela Capes com o intuito de mensurar a qualidade da produção científica dos programas de pós-graduação. A classificação de periódicos é realizada pelas áreas de avaliação e passa por um processo anual e atualização. Esses veículos são enquadrados em estratos indicativos de qualidade; A1, o mais elevado; A2; B1; B2; B3; B4; B5; C, com peso zero. O seu programa de pós-graduação será avaliado segundo essa escala. 05/06/2013Ganesha Associates 2013 Brasil Qualis and the importance of Indexação

22. UK’s Research Excellence Framework 2014 Outputs: The panels will assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standards. This element will carry a weighting of 65 per cent. Impact: The sub-panels will assess the ‘reach and significance’ of impacts on the economy, society and/or culture that were underpinned by excellent research conducted in the submitted unit, as well as the submitted unit’s approach to enabling impact from its research. This element will carry a weighting of 20 per cent. Environment: The sub-panels will assess the research environment in terms of its ‘vitality and sustainability’, including its contribution to the vitality and sustainability of the wider discipline or research base. This element will carry a weighting of 15 per cent.

23. Global metrics summary Science and technology are too politically important for them to be left un-managed The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form Metrics will become more important as better standards are defined

24. Break

25. Portfolio analysis at NIH What is the state of the science? Are there gaps in our research portfolio? How do the gaps compare to research carried out in other agencies? How much do we spend on mitochondrial research? What is the average number of scientific papers generated per $100K spent per grant?

26. 28 August 2012Ganesha Associates 26 CAPES – scientific subdivisions –Ciências Biológicas I: Biologia Geral, Botânica, Genética, Oceanografia Biológica e Zoologia –Ciências Biológicas II: Biofísica, Biologia Molecular, Bioquímica, Farmacologia, Fisiologia, Morfologia, Anatomia, Histologia e Neurociências –Ciências Biológicas III: Microbiologia, Imunologia e Parasitologia –Saúde Coletiva –Ecologia e Meio Ambiente

27. 28 August 2012Ganesha Associates 27 Define: indexação A indexação de uma revista é o processo que confere o indicador de qualidade necessário para que esta possa pertencer a um ou mais bancos de dados de renome internacional, em uma área específica do conhecimento, como por exemplo, MEDLINE, ISI, LILACS, etc., segundo critérios preestabelecidos por essas instituições. É um fator importante para que a produção científica nela apresentada se torne conhecida e reconhecida internacionalmente. O periódico necessita, para isto, estar de acordo com os padrões básicos de apresentação formal e excelência de conteúdo científico, determinados internacionalmente.

28. 28 August 2012Ganesha Associates 28

29. 28 August 2012Ganesha Associates 29

30. 28 August 2012Ganesha Associates 30 As fontes de‘indexação’ em ordem de ‘qualidade ’ Thomson Web of Science PubMed/Medline Scopus ScieLo Google Scirus LILACS

31. 28 August 2012Ganesha Associates 31 Journal selection process - ISI To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI seeks to cover the best regional journals as well. However, rather than compare a regional journal with all journals in a particular subject category, the ISI editor considers it in terms of those journals in the category from the same geographic area. High journal publishing standards, especially timeliness, and English language bibliographic elements remain essential.

32. 28 August 2012Ganesha Associates 32 Impact Factor defined “The Impact Factor of a journal is probably the only quantitative way of assessing its worth and relevance to the academic community it serves.” Editorial, Journal of Computer-assisted Learning, 2001 Impact Factor 2011 = # citations received in 2011 to articles published in 2010/2009 ______________________________________________________ # number of source items published 2010 and 2009

33. Journal of Cell Science - IF

34. Nature citations per article

35. Citation statistics – PLoS Biology 5 July 2012Ganesha Associates35

36. 14 August 2013Ganesha Associates36

37. 28 August 2012Ganesha Associates 37 Impact Factors over time

38. 28 August 2012Ganesha Associates 38 Impact Factor and area

39. 28 August 2012Ganesha Associates 39 Impact Factors for Microbiology

40. 28 August 2012Ganesha Associates 40 Impact Factors for Neuroscience

41. 28 August 2012Ganesha Associates 41 Impact Factors and article type

42. 28 August 2012Ganesha Associates 42 Impact Factors and Authors Correlation between article citation rate and journal impact for four authors

43. 28 August 2012Ganesha Associates 43 Impact Factors and Journals Citation rates in 1986 or 1987 of articles published in three biochemical journals in 1983 or 1984, respectively. From Seglen, BMJ 1997;314:497

44. 28 August 2012Ganesha Associates 44 Alternatives ? The Hirsch Index is rapidly gaining favor as a performance measurement tool for scientific authors. A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np - h) papers have no more than h citations Thus, the h-index reflects both the number of publications and the number of citations per publication. The index is designed to improve upon simpler measures such as the total number of citations or publications. Like the IF, the index works properly only for comparing scientists working in the same field. For more information see Scopus or Wikipedia

45. 28 August 2012Ganesha Associates 45 H-index - weaknesses The h-index is bounded by the total number of publications. This means that scientists with a short career are at an inherent disadvantage, regardless of the importance of their discoveries. For example, had Albert Einstein died in early 1906, his h-index would be stuck at 4 or 5, despite his being widely acknowledged as one of the most important physicists, even considering only his publications to that date.

46. Declaration on Research Assessment Made during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco, CA, on December 16, 2012. Main recommendation: Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions. Alternatives?

47. New Usage Metrics Impact Factor Well-established, easily understood and accepted Endorsed by funding agencies and researchers Several competitors: H-index, Eigenfactor, SJR (Google Page Rank) COUNTER Usage Factor Journal/book usage-based alternative perspective Reflects value of journals to all categories of user Easy to understand PIRUS (Publisher and Institutional Repository Usage Statistics) COUNTER-compliant usage reports at the article level Will enable the usage of individual research outputs to be measured ORCID (Open Researcher and Contributor ID) Central registry of identifiers from individual researchers Will enable aggregation of multiple forms of research output

48. FundRef

49. Altmetrics, an alternative? 5 July 2012Ganesha Associates49

50. Many metrics, many uses… Development of national-level social contracts for science and technological investment Does investment lead to the achievement of national/international objectives for increased employment, economic growth, improved healthcare? At a national/institutional/departmental/individual level how do we compare with the competition? Management of human resources and tacit intellectual property in a multinational organisation Do I get tenure/promotion? How often has my article been downloaded/cited How well has my paper contributed to building the knowledge network in my field? Which journal should I publish in? Which articles should I read? Has my article been picked-up by the media? Standards: CrossRef, ORCID, DataCite, FundRef 14 August 2013Ganesha Associates50

51. Snowball metrics

55. Global metrics summary Science and technology are too politically important for them to be left un-managed The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form Metrics will become more important as better standards are defined