1. Research Evaluation Office, Evaluation Department 11 An Improved Approach of Research Unit Evaluation at the Beginning of the Third Research Term of AIST November 4 th, 2011 @ AEA 2011, Anaheim CA Takashi YOSHIMURA* 1, Yoshiaki Tamanoue 2, Masashi Suzuki 1, Shigeko Togashi 1, Hidenori Endo 1 and Kanji Ueda 1 (1. Evaluation Department, National Institute of A dvanced I ndustrial S cience and T echnology) (2. Ministry of Economy, Trade and Industry)

2. Research Evaluation Office, Evaluation Department 22 Missions of AIST Contribution to the following five missions for the development of society through advancement of industrial technology –A sustainable society –Industrial competitiveness –Local industrial development –Industrial technology policies –Human development for enhanced technical management

3. Research Evaluation Office, Evaluation Department 33 National network of AIST The headquarters are located in Tokyo and Tsukuba with nine regional centers throughout the country. AIST Chugoku AIST Shikoku AIST Hokkaido AIST Tohoku AIST Tsukuba Headquarters / AIST Tsukuba AIST KyushuAIST KansaiAIST ChubuAIST Tokyo Waterfront

4. Research Evaluation Office, Evaluation Department 44 AIST organization at the third research term (2010 to 2015 : five fiscal years) (As of April 1, 2011)

5. Research Evaluation Office, Evaluation Department 55 Research units by various industrial fields in AIST (As of April 1, 2011; no. of research units: 43)

6. Research Evaluation Office, Evaluation Department 66 AIST is enhancing the “open innovation hub” functions. Open innovation hub Integrations to realize the “Open Innovation Hub” strategy in AIST Contribution to society: “social outcomes”

7. Research Evaluation Office, Evaluation Department 7 Evaluation from the viewpoint of “social outcome” Input Output Direct Outcome Final Outcome Research & Development Collaboration Tech. Transfer ・ Innovative Materials ・ Innovative Knowledge ・ Innovative Software ・ Measurement Standards ・ Geological Maps, etc Plan/Action Management ・ New Products ・ New Processes ・ Knowledge Diffusion, etc ・ Developing New Industries ・ Creating Cultural/ Socioeconomic Values ・ Realization of Safe & Secured Society ・ Budget ・ Human Resources ・ Collaboration with Industry ・ Start-up Venture Business ・ Academic Association ・ Capacity Building for Industry ・ Contribution to Policy Making Propagation Infrastructures Know-how, Knowledge, Facilities, etc Evaluation from the viewpoint of value creation as outcome AIST’s Activities For Industry, etcFor Society

8. Research Evaluation Office, Evaluation Department 88 Research unit (center, institute, initiative) Report Res ults Report & Presentation Evaluation results Discussions Evaluation Research term plan Research term target President of AIST Evaluation department METI National Inst. Evaluation Committee (AIST branch) R&D evaluation system of AIST Framework of evaluation system in the 3 rd research term –Research unit evaluation –Research unit recommendation –Follow-up for research unit evaluation Report Research unit evaluation Follow-up for evaluation Resource allocation, Reorganization Research unit recommendation

9. Research Evaluation Office, Evaluation Department 99 Research unit evaluation of AIST Purposes –Activation and efficient management of research units –Reference for the top management of AIST –Accountability to the Japanese nation Evaluation indexes –Roadmaps of the research unit –Outputs of each research theme in the unit –Contributions to society –Research unit management

10. Research Evaluation Office, Evaluation Department 10 The improvements in research unit evaluation at the third research term We attempted to –Increase the number of external reviewers of the evaluation committee –Reduce the burden of the evaluation process –Strengthen the recommendation function of the evaluation committee

11. Research Evaluation Office, Evaluation Department 11 Role of reviewers in evaluation External reviewers –Specialists and intellectuals outside of AIST –5 to 7 for each research unit –Main role: evaluation of roadmaps and outputs for each research theme from the viewpoints of “social outcomes” Internal reviewers –From evaluation department (two for each research unit) –Main role: evaluation of research unit management 11

12. Research Evaluation Office, Evaluation Department 12 Background of the external reviewers Reviewers from the industry sector have been highly recommended at the 3 rd research term. ⇒ increase of discussions toward product realization research in the evaluation committee Research term1st2nd3rd FY2001-2004FY2005-2009FY2010- Academia70358.3%74554.9%9137.4% Industry34428.5%41630.6%12551.4% National Institute13811.4%1329.7%124.9% Journalism211.7%654.8%156.2% Total1206100.0%1358100.0%243100.0%

13. Research Evaluation Office, Evaluation Department 13 Comments of reviewers for innovation promotion External reviewers: various comments, in particular, advices for collaboration with industry Internal reviewers: relatively more comments for collaboration with academia and industry

14. Research Evaluation Office, Evaluation Department 14 Summary of improvements in evaluation Increase of useful comments from the viewpoints of social outcomes –By reviewers from industry and journalism Feedback of evaluation results to presidents of AIST for recommendations to the management –Before research unit reorganization and resource allocation for the next fiscal year Evaluation guideline have been reconsidered to reduce the burden of the evaluation process. –Sufficient promotion of evaluation system to reviewers before a meeting of the evaluation committee –Rearrangement of evaluation indexes

15. Research Evaluation Office, Evaluation Department 15 Our approaches for evaluation system are now proceeding... We welcome any comments or suggestions, and please visit to our website. http://www.aist.go.jp/index_en.html