1 Making a Grope for an Understanding of Taiwan’s Scientific Performance through the Use of Quantified Indicators Prof. Dr. Hsien-Chun Meng Science and Technology Information Center (STIC) National Science Council 2002/12/

2 Outline Research Background Research Goals Methodology Results & Discussion Future Works References

3 Research Background Insufficient bibliometric indicators to evaluate scientific research performance in Taiwan Limited inquiry function of NSI and JCR Lack of international comparison of the performance with benchmarking countries STIC has aggressively involved in evaluating scientific activities

4 Research questions … Is Taiwan ’ s research performance world class? Has the performance improved or deteriorated? What are the relatively strong and weak fields?

5 Research Goals To assess the position of Taiwan ’ s S&T research system in an international comparative perspective To identify relative strengths and weaknesses of scientific fields To serve as a reference for technology policy formulation and for the future allocation of innovation resources

6 Methodology Methodology : Bibliometric Method Study Scope : Time interval : 1995~2001 Benchmarking Country Selection : 7 advanced countries 4 developing countries in Eastern Asia CWTS Classification system : 7 broad areas  43 fields

7 Benchmarking Country Selection Advanced Countries United States of America Japan United Kingdom France Germany Netherlands Israel Developing Countries in Eastern Asia South Korea Peoples Republic of China Singapore Taiwan

8 CWTS Classification system COMPUTER SCIENCES and MATHEMATICS MATHEMATICS COMPUTER SCIENCE STATISTICS ENGINEERING SCIENCES ELECTRICAL ENGINEERING MATERIALS SCIENCE CIVIL ENGINEERING MECHANICAL ENGINEERING INSTRUMENTS AND INSTRUMENTATION FUELS AND (NUCLEAR) ENERGY GEOLOGICAL ENGINEERING CHEMICAL ENGINEERING AEROSPACE ENGINEERING OTHER ENGINEERING SCIENCES NATURAL SCIENCES CHEMISTRY PHYSICS ASTRONOMY EARTH SCIENCES ENVIRONMENTAL SCIENCES MEDICAL SCIENCES AND LIFE SCIENCES BASIC LIFE SCIENCES BIOLOGY FOOD SCIENCES AND AGRICULTURE BIOMEDICAL SCIENCES CLINICAL MEDICINE HEALTH SCIENCES PHARMACOLOGY DENTISTRY

9 CWTS Classification system SOCIAL AND BEHAVIORAL SCIENCES POLITICAL SCIENCE AND PUBLIC ADMINISTRATION INFORMATION AND COMMUNICATION SCIENCES PSYCHOLOGY SOCIOLOGY ANTHROPOLOGY EDUCATIONAL SCIENCES SOCIAL GEOGRAPHY AND DEMOGRAPHY PUBLIC HEALTH AND SOCIAL WELFARE OTHER SOCIAL AND BEHAVIORAL SCIENCE LAW, ARTS, AND THE HUMANITIES LAW LANGUAGE AND LINGUISTICS HISTORY AND ARCHEOLOGY PHILOSOPHY RELIGION CREATIVE ARTS, CULTURE, AND MUSIC MULTIDISCIPLINARY SCIENCE MULTIDISCIPLINARY SCIENCES (Contains a set of broad general journal including Nature, Science, and the Proceedinf of the National Academy of Sciences)

10 Bibliometric Indicators (1) P: Publications An indicator of scientific productivity and specialization of a country Cx: Citations excluding self-cited a frequently cited paper has had a greater influence on subsequent research activities A measure of impact of research in the scientific community and recognition of scientific quality

11 Bibliometric Indicators (2) CPPx: Average number of citations per publication (excluding self-cited) CPPx/FCSm: normalized citation impact FCSm: World citation average in the subfields CPPx/FCSm score within the range of 0.8 and 1.2 is considered world average; above 1.2 indicates citation impact levels significantly above world average.

12 Data sources ISI database SCI (Science Citation Index) SSCI (Social Sciences Citation Index) AHCI (Arts & Humanities Citation Index) Specialty Citation Indexes: “ CompuMath ”, “ Biotechnology ”, “ Biochemistry ”, “ Chemistry ”, “ Materials Science ”, “ Neurosciences ”

13 Result and Discussion

United States1 (27.9%)1 (28.0%)1 (28.6%)1 (29.3%)1 (30.5%)1 (31.3%) United Kingdom2 (7.8%)2 (8.2%)2 (8.1%) 2 (8.5%) Japan3 (7.7%) 3 (7.8%)3 (7.7%)3 (7.5%) Germany4 (7.1%) 4 (7.3%)4 (7.0%)4 (6.8%) France5 (5.1%)5 (5.2%)5 (5.3%)5 (5.4%)5 (5.3%) Peoples Rep. China8 (3.4%)8 (3.1%)9 (2.5%)12 (2.1%)12 (1.9%)14 (1.6%) Netherlands12 (2.1%) 11 (2.2%) 10 (2.2%) South Korea15 (1.6%)16 (1.4%)16 (1.3%)16 (1.2%)18 (1.0%)20 (0.9%) Taiwan ROC18 (1.1%)20 (1.0%) 19 (1.0%)19 (0.9%) Israel21 (1.0%)19 (1.0%)18 (1.0%)18 (1.1%)16 (1.1%)17 (1.1%) Singapore34 (0.4%) 36 (0.3%) 37 (0.3%)40 (0.2%) Overview of publication output Country ranking in worldwide publication output (96 ’ ~01 ’ )

15 Overview of publication output Growth rates in total publication output

16 Research profiles of 4 Asian countries TAIWAN CHINASINGAPORE SOUTH KOREA

17 Research profiles comparison (Asian 4 countries V.S. Advanced countries) Advanced CountriesEastern Asia(4 countries) Including Taiwan Peoples Republic of China South Korea Singapore Including United States of America Japan United Kingdom, France Germany Netherlands Israel

18 Trend of research profiles in Taiwan

19 Research profiles in Taiwan (43 fields)

20 Research profiles in Taiwan (43 fields)

21 Scientific impact of selected countries COMPUTER SCIENCES, MATHEMATICS AND NATURAL SCIENCE

22 Scientific impact of selected countries ENGINEERING SCIENCE

23 Scientific impact of selected countries MEDICAL AND LIFE SCIENCE

24 Scientific impact of selected countries SOCIAL AND BEHAVIORAL SCIENCES

25 Scientific impact of selected countries LAW, ARTS, HUMANITIES AND MULTIDISCIPLINARY SCIENCE

26 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwan South KoreaSingaporePRC ANTHROPOLOGY(1) 0.41*(3) 0 (2) 0.3 PHARMACOLOGY(1) 0.52(4) 0.33(2) 0.48(3) 0.36 EARTH SCIENCES(1) 0.55(2) 0.52(4) 0.47(3) 0.48 ELECTRICAL ENGINEERING(1) 0.61(2) 0.6 (4) 0.41 PHYSICS(1) 0.64(3) 0.59(2) 0.6(4) 0.43 CIVIL ENGINEERING(1) 0.71(2) 0.62 PSYCHOLOGY(1) 0.71(3) 0.59(4) 0.58(2) 0.63 STATISTICAL ANALYSIS AND PROBABILITY(1) 0.72(3) 0.65(2) 0.7(4) 0.44 LANGUAGE AND LINGUISTICS(1) 0.74(4) 0.33(3) 0.65(2) 0.69 *(Ranking) CPPx/FCSmx

27 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwan South KoreaSingaporePRC EDUCATIONAL SCIENCE(2) 0.4(3) 0.32(4) 0.31(1) 0.42 ASTRONOMY AND ASTROPHYSICS(2) 0.48(1) 0.7(4) 0(3) 0.35 BASIC LIFE SCIENCES(2) 0.54(3) 0.34(1) 0.99(4) 0.28 COMPUTER SCIENCE(2) 0.61(3) 0.61(1) 0.67(4) 0.59 MATHEMATICS(2) 0.62(3) 0.59(1) 0.77(4) 0.56 ENVIRONMENTAL SCIENCES(2) 0.64(3) 0.63(1) 0.72(4) 0.62 CHEMICAL ENGINEERING(2) 0.74(3) 0.65(1) 0.77(4) 0.65 MATERIALS SCIENCE(2) 0.74(1) 0.83(3) 0.68(4) 0.55 CHEMISTRY(2) 0.76(3) 0.58(1) 1.01(4) 0.43 LAW(2) 0.84(1) 1.02(3) 0.5(4) 0.38 *(Ranking) CPPx/FCSmx

28 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwanSouth KoreaSingaporePRC RELIGION(3) 0 (2) 0.22(1) 0.82 POLITICAL SCIENCE AND PUBLIC ADMINISTRATION(3) 0.36(2) 0.38(1) 0.66(4) 0.33 PHILOSOPHY(3) 0.4(4) 0.27(1) 0.98(2) 0.74 BIOLOGICAL SCIENCES(3) 0.48(2) 0.64(1) 0.72(4) 0.34 INSTRUMENTS AND INSTRUMENTATION(3) 0.48(2) 0.63(1) 0.65(4) 0.42 FUELS AND (NUCLEAR)ENERGY(3) 0.49(2) 0.56(1) 0.79(4) 0.37 AEROSPACE ENGINEERING(3) 0.52(2) 0.65(1) 0.78(4) 0.23 BIOMEDICAL SCIENCES(3) 0.53(1) 0.59(2) 0.54(4) 0.53 CLINICAL MEDICINE(3) 0.55(2) 0.67(1) 0.7(4) 0.39 PUBLIC HEALTH AND SOCIAL WELFARE(3) 0.57(2) 0.81(1) 0.86(4) 0.43 OTHER ENGINEERING SCIENCES(3) 0.59(1) 0.77(2) 0.6(4) 0.47 FOOD SCIENCE AND AGRICULTURE(3) 0.62(4) 0.59(1) 0.95(2) 0.64 MULTIDISCIPLINARY SCIENCES(3) 0.62(2) 0.74(1) 0.96(4) 0.05 CREATIVE ARTS, CULTURE AND MUSIC(3) 0.63(4) 0.41(1) 1.05(2) 0.79 MECHANICAL ENGINEERING(3) 0.68(1) 0.69 (4) 0.54 DENTISTRY(3) 0.71(1) 0.96(4) 0.53(2) 0.8 OTHER SOCIAL AND BEHAVIORAL SCIENCES(3) 0.74(4) 0.47(2) 0.82(1) 0.85 *(Ranking) CPPx/FCSmx

29 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwanSouth KoreaSingaporePRC HISTORY AND ARCHAEOLOGY(4) 0(2) 0.73(1) 1.14(3) 0.61 GEOLOGICAL ENGINEERING(4) 0.3(2) 0.43(1) 0.47(3) 0.4 SOCIAL GEOGRAPHY AND DEMOGRAPHY(4) 0.4(3) 0.66(1) 1.15(2) 0.83 HEALTH SCIENCES(4) 0.53(3) 0.57(1) 0.95(2) 0.61 ECONOMICS, BUSINESS AND MANAGEMENT(4) 0.54(1) 0.86(3) 0.71(2) 0.83 SOCIOLOGY(4) 0.55(2) 0.78(1) 1.18(3) 0.61 INFORMATION AND COMMUNICATION SCIENCES(4) 0.66(2) 0.91(1) 1.2(3) 0.73 *(Ranking) CPPx/FCSmx

30 Future Works (1) Micro level bibliometric analyses of Taiwanese science Bibliometric data of 279 subfields in the period of Other bibliometric indicators are included: CPPx/JCSm: compare the impact factor with the worldwide average in the same journal sets JCSm/FCSm: to evaluate whether the country publishes in top journals Breakdown of output (P) and impact (CPPx/FCSm) into ‘ international collaboration ’ and ‘ national work ’

31 Example 1 Field (Impact Score= CPPx/FCSmx) Scientific Research Profile Output And Impact in Physics ＊ CPPx ： The average numbers of citations per publication (excluding self-citations) ＊ FCSmx ： The average citation rate of all papers in the fields in which the group is active. ＊ Impact Score (CPPx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High

32 Example 2 Field (Impact Score= JCSm/FCSm) Scientific Research Profile Output And Impact in Physics ＊ JCSmx ： The average citation rate of all papers published in the ISI-covered journals in which a group has published. ＊ FCSmx ： The average citation rate of all papers in the fields in which the group is active. ＊ Impact Score (JCSmx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High

33 Future Works (2) The research performance of Taiwan ’ s universities/institutions 55 universities and 20 institutions are included Breakdown of output and impact into scientific fields International collaboration will be analyzed

34 Example 1 Field (Impact Score= CPPx/FCSmx) ＊ CPPx ： The average numbers of citations per publication (excluding self-citations) ＊ FCSmx ： The average citation rate of all papers in the fields in which the group is active. ＊ Impact Score (CPPx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High Institutional Research Profile Output And Impact Per Institute Taiwan's most active Physics institutes

35 Example 2 Impact: Low Average High Field (Impact Score= JCSm/FCSm) Institutional Research Profile Output And Impact Per Institute Taiwan's most active Physics institutes ＊ JCSmx ： The average citation rate of all papers published in the ISI-covered journals in which a group has published. ＊ FCSmx ： The average citation rate of all papers in the fields in which the group is active. ＊ Impact Score (JCSmx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average.

36 Collaboration analysis Cooperation Profile:Output And Impact Per Type Taiwan ’ s 5 largest Physics institutes ＊ CPPx ： The average numbers of citations per publication (excluding self-citations) ＊ FCSmx ： The average citation rate of all papers in the fields in which the group is active. ＊ Impact Score (CPPx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High Field (Impact Score= CPPx/FCSm)

37 Impact: Low Average High Field (Impact Score= JCSm/FCSm) Cooperation Profile:Output And Impact Per Type Taiwan ’ s 5 largest Physics institutes ＊ JCSmx ： The average citation rate of all papers published in the ISI-covered journals in which a group has published. ＊ FCSmx ： The average citation rate of all papers in the fields in which the group is active. ＊ Impact Score (JCSmx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Collaboration analysis

38 Future Works (3) The links between scientific knowledge and patents To evaluate the contribution of science to technological development in Taiwan – academia- industry linkages To identify knowledge flows in the domestic S&T system (e.g., public-private R&D networks; role of patent citations) To develop a model to describe the relationships (e.g., links, flows, users, providers, feedbacks)

39 Reference New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications H. F. Moed, R. E. De Bruin, T. N. van Leeuwen, Scientometrics, 5(1995) Mapping the scientific performance of German medical research R. J. W. Tijssen, T. N. van Leeuwen, A. F. J. van Raan, 2002 Bibliometric mapping as a science policy and research management tool