Presentation is loading. Please wait.

Presentation is loading. Please wait.

Program Level Assessment of Outcomes and Impacts of Research of Centers Juan D. Rogers School of Public Policy Georgia Tech Project: Assessment of 15 Nanotechnology.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Program Level Assessment of Outcomes and Impacts of Research of Centers Juan D. Rogers School of Public Policy Georgia Tech Project: Assessment of 15 Nanotechnology."— Presentation transcript:

1 Program Level Assessment of Outcomes and Impacts of Research of Centers Juan D. Rogers School of Public Policy Georgia Tech Project: Assessment of 15 Nanotechnology Science and Engineering Centers’ (NSECs): Outcomes and Impacts: Their Contribution to NNI Objectives and Goals, NSF 0955089.

2 Objectives Publication output and impact of the NSECs Co-authorship patterns change over time Geographical spread or concentration of NSEC research collaboration Evolution of areas of research as reflected in keyword changes Multidisciplinary footprint of the centers The case for centers 11/5/2011AEA, Annaheim CA2

3 Methodology Outline Qualitative-quantitative concurrent design ◦ Qualitative component  Multiple embedded case studies of mechanisms for center outcomes  Follow up field level impact with quantitative analysis ◦ Quantitative component  Bibliometrics (productivity, citation, co-authors, etc.)  Personnel and funding data analysis  Intellectual property instances (patents, licenses, etc.)  Links with business and commercial applications ◦ Methodologically: Final results are generalizations to theory from qualitative analysis  We offer analysis of combined quant-qual data  Interpretive schemes for quantitative findings on centers 11/5/2011AEA, Annaheim CA3

4 Data and methods Acquisition of 85 center annual reports from all 15 NSEC centers Extraction and clean up (duplicate removal) of publication lists Extraction of NSEC articles from Web of Science (n=3,500) Look up and extraction of articles citing NSEC articles (n=75,000) Clean up and classification of collaborating organizations Clean up, look up, identification, and matching of NSEC authors in author listings Analyses of different type, multiple tools: ◦ Growth, shares, and overall trends (tabular) ◦ Networks and collaborations (Gephi) ◦ Geographical spread, GIS (ArcGIS) ◦ Keywords and topics (VantagePoint) ◦ Multidisciplinarity and science maps (Pajek) 11/5/2011AEA, Annaheim CA4

5 NSEC publication activity grows in three waves 11/5/2011AEA, Annaheim CA5 Notes: *Publication data not reported by all NSEC centers; last column reports average annual change for rows with change data. Source: ISI-WoS publication data based on NSEC annual reports by center. 200120022003200420052006200720082009-10*2001-10* NSEC publications (all centers) 66 133 221 262 499 515 715 737 361 3,870 ▪ Annual change 102%66%19%90%3%39%3%-51%34% Citing articles 48 391 1,164 2,619 4,595 7,415 10,469 15,243 19,149 94,484 ▪ Annual change 715%198%125%75%61%41%46%26%12% Publications 2001-042005-062007-08

6 One out of 10 NSEC publications has an industry co-author 11/5/2011AEA, Annaheim CA6 Total unique firms co-authoring articles with NSEC (2001-2010): 146 Total unique firms maintaining other types of collaborations with NSEC (as of 2010): 275** Notes: * Publication data not reported by all NSEC centers; last column reports average annual change for rows with change data. **The type of collaborations are not specified by centers (only number of industry partners was provided). Source: ISI-WoS publication data based on NSEC annual reports by center and lists of industry partners provided by NSEC centers. 20012002200320042005200620072008 2009- 10* 2001- 10* NSEC centers with publications36613 15 1315 NSEC publications (all centers)661332212624995157157373613,509 NSEC pubs. co-auth. with industry121316173552766534360 ▪ Annual change 8%23%6%106%49%46%-14%-48% 22% ▪ Share industry co-auth / all pubs.18%10%7%6%7%10%11%9% 10% Unique co-author firms11139163129504322146 ▪ Annual change 18%-31%78%94%-6%72%-14%-49% 20% Industry collaborations

7 Comparative Impact of NSEC Papers (Citations Cohort 2001) 200120022003200420052006200720082009 C_Mean 0.886.3014.4423.6435.8848.4861.1175.6190.17 C_Med 02469.51112.514.517.5 C_Max 1252148297528776106214091760 N_Mean 0.262.115.138.1811.6214.7817.8721.1124.04 N_Medi 0124568910 N_Max 3916337674712681803228629023484 11/5/2011AEA, Annaheim CA7

8 Comparative Impact of NSEC Papers (Citations Cohort 2002) 20022003200420052006200720082009 C_Mean 1.209.0621.3236.9251.7567.1385.15101.80 C_Med 038.51620.525.530.536.5 C_Max 198819640058580710631330 N_Mean 0.282.325.308.8312.1015.3618.7921.92 N_Med 01246789 N_Max 501533406611053149920422587 11/5/2011AEA, Annaheim CA8 NSEC papers appear to have higher impact measured by citations: median and mean citations grow faster than the cohort with window length

9 Rank in Cohort of Top 20 NSEC Papers 12345678910 2001 52998167179223313411465629 2002 6101720235669124127148 11/5/2011AEA, Annaheim CA9 11121314151617181920 2001 834134513832077299232023203328332843755 2002 175192228279322382430513559566 NSEC papers rank highly in their cohort of Nano papers. Total Cohort 2001: 30462 papers. NSEC Cohort 2001: 66 papers Total Cohort 2002: 34971 papers. NSEC Cohort 2002: 128 papers

10 Comparative Impact of NSEC Papers (Citations Cohort 2003) 2003200420052006200720082009 N_mean0.302.285.769.1712.6216.3219.77 N_median0134679 N_max462516741144167622992913 C_mean1.097.7519.2931.0843.1258.3573.32 C_median041015212630 C_max13802093775738551204 11/5/2011AEA, Annaheim CA10 The distributions parameters show NSEC high position in the field but the top paper is still not by the centers. N = 40813 CN = 222 12345678910 3141534363947658398 11121314151617181920 119126129132137159198211238245

11 Comparative Impact of NSEC Papers (Citations Cohort 2004) 200420052006200720082009 Xmean0.282.565.789.2112.9616.50 Xmedian013468 Xmax46299650107616832334 NXmean0.847.3216.1425.7437.3147.79 NXmedian04101520.528 NXmax1980154225306392 11/5/2011AEA, Annaheim CA11 12345678910 425255717379100105123147 11121314151617181920 248261300314472473539557588614 N = 48952; CN = 259

12 Comparative Impact of NSEC Papers (Citations Cohort 2005) 20052006200720082009 N_mean0.362.656.049.9713.74 N_median01357 N_max693035909741553 C_mean1.209.0720.5134.9649.12 C_median04101623 C_max251454059491553 11/5/2011AEA, Annaheim CA12 12345678910 15817182529323638 11121314151617181920 42508186100102105129130149 N = 55998; CN = 499

13 Comparative Impact of NSEC Papers (Citations Cohort 2006) 20062007200820092010 N_mean0.392.676.3710.1110.94 N_median01356 N_max39176396671756 C_mean1.067.5318.0429.4340.77 C_median0491418 C_max2094270574980 11/5/2011AEA, Annaheim CA13 12345678910 1356212530485170 11121314151617181920 84899599100103132146148190 N = 62351; CN = 512

14 Increasing JIF of Target Journals 11/5/2011AEA, Annaheim CA14

15 Increasing JIF of Target Journals 11/5/2011AEA, Annaheim CA15

16 Leveraging Support Reflected in Publication Support Acknowledgment 11/5/2011AEA, Annaheim CA16

17 NSEC co-authorship networks grow and become more widespread 11/5/2011AEA, Annaheim CA17 Co-authorships 2001-2006 Notes: Nodes represent authors. Node size represents number of publications for the period. Node colors represent 15 NSEC centers. Line colors are those of the centers that maintain each co-authorship as found in publication databases. Source: ISI-WoS publication data based on NSEC annual reports by center. Co-authorships 2007-2010

18 11/5/2011AEA, Annaheim CA18 Central and well connected authors Highly collaborative projects Multiple productive and collaborative authors NSEC collaboration networks present diverse patterns of co- authorship (2001-2010) Co-authorships

19 Wide geographic spread of NSEC research with concentration in specific locations/regions 11/5/2011AEA, Annaheim CA19 Note: Number of NSEC publications from 2001-2010 = 3509; number of citing publications = 75335. Citing publications, 2001-2010 exclude all NSEC publications. Collaboration and places

20 Co-authoring extends beyond the NSECs to nearly all US states 11/5/2011AEA, Annaheim CA20 Collaboration and places

21 Clusters of top keywords (21 cluster solution reported) More specialized terms More linked terms Research topics

22 Top 20 Terms Across 15 NSECs (relative position of 30 most common, 2001-2006 v. 2007- 2010) 11/5/2011AEA, Annaheim CA22 FOUNDATIONAL FABRICS THIN-FILM SELF-ASSEMBLED MONOLAYER SPECTROSCOPY CARBON NANOTUBE STABLE NANOCRYSTAL OPTICAL PROPERTY MOLECULAR-DYNAMICS GROWTH FIELD-EFFECT TRANSISTOR LITHOGRAPHY GOLD NANOPARTICLE QUANTUM DOT SENSOR WALL CARBON NANOTUBE IN-VITRO SEMICONDUCTOR MOLECULAR-DYNAMICS SIMULATION BIOSENSOR IN-VIVO THIN-FILM-TRANSISTOR GRAPHITE MATURE DNA SCATTERERS TRANSISTOR DIP PEN NANOLITHOGRAPHY SILVER NANOPARTICLE AQUEOUS-SOLUTION ATOMIC FORCE MICROSCOPY WAVE-GUIDE HETEROSTRUCTURE Research topics

23 Neurosciences Agriculture Chemistry Physics Engineering Environ Sci Matls Sci Clinical Med Computer Sci. Geosciences Ecology General Med Map of Science: Subject Categories within all Nano (2009) Source: Rafols, I. and Meyer, M. (2009) Diversity and Network Coherence as indicators of interdisciplinarity: case studies in bionanoscience. /Scientometrics/, 81(2), in print; Leydesdorff, L. and Rafols, I. (2009) A Global Map of Science Based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362. 11/5/201123AEA, Annaheim CA

24 Neurosciences Agriculture Biomed Sci Chemistry Physics Engineering Environ Sci Matls Sci Infectious Diseases Clinical Med Computer Sci. Geosciences Ecology General Med NSEC research is multidisciplinary with focus areas in materials science, chemistry and biomedical sciences Map source: Rafols, I., Meyer, M. (2009) Diversity and Network Coherence as indicators of interdisciplinarity: case studies in bionanoscience. Scientometrics, 81(2), in print; Leydesdorff, L., Rafols, I. (2009) A Global Map of Science Based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362. Multidisciplinarity Global Map of Science, 2006 175 SCI Subject Categories 11/5/2011AEA, Annaheim CA24

25 Neurosciences Agricultur e Biomed Sci Chemistry Physics Engineerin g Environ Sci Matls Sci Infectious Diseases Clinical Med Computer Sci. Geoscience s Ecology General Med A range of disciplines beyond focal areas cite NSEC works Multidisciplinarity Global Map of Science, 2006 175 SCI Subject Categories 11/5/2011AEA, Annaheim CA25

26 Implications for Center Policy NSEC publication growth rate indicates rapid take-off by new centers. NSEC mechanism allows for greater involvement of authors over time in diverse locations. NSEC research involves some foundational as well as some emerging (and some maturing) topical areas. NSECs work in multiple disciplines and their citation influence involves many more disciplines (especially in Biotechnology related fields). 11/5/2011AEA, Annaheim CA26 Conclusions

27 The Case for Centers Reporting by PIs on crucial role of centers: ◦ Unique incentive to go deeply across disciplinary boundaries  Significantly reduced cross-disciplinary transaction costs ◦ Unique research experience for graduate students  Rich and diverse research infrastructure enables more risk taking  Exposure to greater number of high quality scientific contacts  Exposure to unique industry contacts ◦ Accelerator of promising young researcher development  Center as recruiting tool of top talent  Diverse mentoring opportunities for rapid career development ◦ Unique infrastructure possibilities  Leverage of resources for shared new facilities  Design of new unique instruments and experimental arrangements 11/5/2011AEA, Annaheim CA27

28 Some Program Challenges Some scientific contributions are difficult to explain to the lay public ◦ Important for long term support of the enterprise The special benefits for development of faculty and graduate students should be scaled up to reach the rest of the university community The pre- and extra- university education efforts are dependent on the center program and are difficult to institutionalize without it. ◦ Their sustainability should be a program concern Similar sustainability issue is raised by specialized infrastructure that may go to waste if centers are discontinued 11/5/2011AEA, Annaheim CA28

Download ppt "Program Level Assessment of Outcomes and Impacts of Research of Centers Juan D. Rogers School of Public Policy Georgia Tech Project: Assessment of 15 Nanotechnology."

Similar presentations

Overlay Maps of Science (2010 update)

Overlay Maps of Science (2010 update)
Overlay Maps of Science Ismael Rafols 1,2, Alan Porter 2 and Loet Leydesdorff 3 1 SPRU, University of Sussex, Brighton 2 School of Public Policy, Georgia.

Overlay Maps of Science Ismael Rafols 1,2, Alan Porter 2 and Loet Leydesdorff 3 1 SPRU, University of Sussex, Brighton 2 School of Public Policy, Georgia.
Application of the Benefits Analysis Tools for MRC population health studies Professor Dipak Kalra Centre for Health Informatics and Multiprofessional.

Application of the Benefits Analysis Tools for MRC population health studies Professor Dipak Kalra Centre for Health Informatics and Multiprofessional.
Presented by: Jerry Legge Associate Provost for Academic Planning (Interim), and Professor of Public Administration and Policy (SPIA) Provost Advisory.

Presented by: Jerry Legge Associate Provost for Academic Planning (Interim), and Professor of Public Administration and Policy (SPIA) Provost Advisory.
EAC HIGHER EDUCATION POLICY

EAC HIGHER EDUCATION POLICY
Paris, May 2007 How good is the research base? New approaches to research indicators Colloque de l’Académie des sciences Évolution des publications scientifiques

Paris, May 2007 How good is the research base? New approaches to research indicators Colloque de l’Académie des sciences "Évolution des publications scientifiques"
Assessing and Increasing the Impact of Research at the National Institute of Standards and Technology Susan Makar, Stacy Bruss, and Amanda Malanowski NIST.

Assessing and Increasing the Impact of Research at the National Institute of Standards and Technology Susan Makar, Stacy Bruss, and Amanda Malanowski NIST.
Presentation at WebEx Meeting June 15, 2012 1.  Context  Challenge  Anticipated Outcomes  Framework  Timeline & Guidance  Comment and Questions.

Presentation at WebEx Meeting June 15,  Context  Challenge  Anticipated Outcomes  Framework  Timeline & Guidance  Comment and Questions.
Subcommittee on STEM Learning and STEM Learning Environments The Subcommittee on STEM Learning and STEM Learning Environments offers three overarching.

Subcommittee on STEM Learning and STEM Learning Environments The Subcommittee on STEM Learning and STEM Learning Environments offers three overarching.
Science of Science and Innovation Policy (SciSIP) Presentation to: SBE Advisory Committee By: Dr. Kaye Husbands Fealing National Science Foundation November.

Science of Science and Innovation Policy (SciSIP) Presentation to: SBE Advisory Committee By: Dr. Kaye Husbands Fealing National Science Foundation November.
IDENTIFYING HOT BRAZILIAN SCIENCE AND TECHNOLOGY: TECH MINING METHODS FOR RELATING SOURCES OF KNOWLEDGE AND EMERGING RESEARCH AREAS EU-SPRI CONFERENCE,

IDENTIFYING HOT BRAZILIAN SCIENCE AND TECHNOLOGY: TECH MINING METHODS FOR RELATING SOURCES OF KNOWLEDGE AND EMERGING RESEARCH AREAS EU-SPRI CONFERENCE,
Research evaluation at CWTS Meaningful metrics, evaluation in context

Research evaluation at CWTS Meaningful metrics, evaluation in context
Asha Balakrishnan Vanessa Peña Bhavya Lal Task Leader November 5, 2011

Asha Balakrishnan Vanessa Peña Bhavya Lal Task Leader November 5, 2011
Information Retrieval to Informed Action with Research Metrics Thomson Scientific Research Services Group 2007.

Information Retrieval to Informed Action with Research Metrics Thomson Scientific Research Services Group 2007.
Presented by: Charles Pallandt Title: Managing Director EMEA Academic & Governmental Markets Date: April 28 th, 2011 - Turkey “Driving Research Excellence.

Presented by: Charles Pallandt Title: Managing Director EMEA Academic & Governmental Markets Date: April 28 th, Turkey “Driving Research Excellence.
Shou Ray Information Service Co., Ltd.

Shou Ray Information Service Co., Ltd.
Disciplinary Diversity of Units These slides expand the article How journal rankings can suppress interdisciplinary research: A comparison between innovation.

Disciplinary Diversity of Units These slides expand the article How journal rankings can suppress interdisciplinary research: A comparison between innovation.
3. Challenges of bibliometrics DATA: many problems linked to the collection of data FIELDS – DISCIPLINES : various classifications, not satisfactory INDICATORS.

3. Challenges of bibliometrics DATA: many problems linked to the collection of data FIELDS – DISCIPLINES : various classifications, not satisfactory INDICATORS.
Introduction Breadth How it Works The Next Steps.

Introduction Breadth How it Works The Next Steps.
Bibliometrics overview slides. Contents of this slide set Slides 2-5 Various definitions Slide 6 The context, bibliometrics as 1 tools to assess Slides.

Bibliometrics overview slides. Contents of this slide set Slides 2-5 Various definitions Slide 6 The context, bibliometrics as 1 tools to assess Slides.

Similar presentations

Ads by Google