1. GO WOTRO 2011-2012 Impressions from a selection jury member by Fred Roozeboom WISE event, Sept. 5, 2013

2. GO WOTRO 2011-2012 Impressions from a selection jury member by Fred Roozeboom Who am I -2007- present Part-time professor in Applied Physics dept. of Plasma & Materials Processing) at TU/e - 2009 - present Senior scientific advisor at TNO -1983-2009 Research Fellow at Philips and NXP -1980-1983 Scientist at Exxon-Mobil (USA) -1976-1980 PhD student at Univ. Twente (Chem. Engng.)

3. NWO’s request ~ 6 months in advance -Phone call referring to my previous VENI jury membership, 2 years earlier, requesting to be available to review max. 10 proposals on 2-3 meeting days for the “Cross-Border” VIDI proposals, 2012 edition GO WOTRO -Quite some insistence on the part of NWO -Jury of 12 pre-advisors (all Dutch) with very different backgrounds Shortly after my acceptance -E-mails that you will get ~5 envelopes with all (~44 !) proposals

4. Initially.. it seemed that each member had to take only ~ 10 proposals > 3.49 no granting Key ranking criteria Quality of researcher CV, h-index, international experience, experience with coaching students, independent research line Quality, innovative nature and scientific impact of the proposal Relevance research, well written, feasibility of the proposal Knowledge utilization Elaboration knowledge utilization, applicability results

5. Soon.. came the request sent afterwards to take and rank all proposals.. NWO’s argument: to ensure that all members have gone through all the pieces well and can compare the sessions

6. GO WOTRO: a wide range of programs A selection from http://www.nwo.nl/en/about-nwo/organisation/nwo-divisions/wotro Food & Business Research Agriculture beyond Food Conflict and Cooperation over Natural Resources in Developing Countries (CoCooN) Global Health Policy and Health Systems Integrated Programs Joint MFS II Evaluations of development interventions Migration, Development and Conflict Urbanising Deltas of the World I recall: Bio-medical, medical-psychological (very often) Cognitive recognition, RF-transmission

7. GO WOTRO 2011-2012 list of submitted proposals

8. Overall jury scores before 1 st meeting in March ‘12 > 3.49 no granting

9. After case-by-case reconsideration at 1 st meeting > 3.49 no granting

10. Done deal for interview part ? Good to be a woman ? Mostly: yes…, but an interview can make or break a proposal ! N ABESPREKING Commissielid ….. merkt op dat de kandidaat op een hoge positie stond voor het interview. De voorzitter prof. ….. merkt op dat de referenten 3 x A+ hebben gegeven, maar wel inhoudelijke vragen hadden. Commissielid …..merkt op dat de kandidaat erg zenuwachtig was. De voorzitter prof. ….. merkt op dat er niet voldoende onderbouwing is voor type D en de oorzaken van type D en dat het goed is dat commissielid ….. de genetische studie had uitgezocht. Commissielid ….. merkt op dat het genetische gedeelte niet haalbaar is. De voorzitter prof. ….. merkt op dat de kandidaat vastliep in haar antwoord op de vraag over causaliteit. Dit is geen mechanistisch onderzoek. One triple A + -case on medical-psychological as a painful exception:

11. 2 months after interviews, when you think it’s done… An unexpected appeal letter sent to NWO, requesting a revision

12. Very educational for assessing jury members and the assessed candidates The number of proposals for the interview selection, 45 pieces, is (too) much It's a lot of work to give all applications in the selection interviews a score. Logical that this method ensures that all members have gone through all paper work. The interview has a major impact on the overall assessment. Not all the proposals had a clear cross-border nature. How to filter out these requests? The selection of referees has a large influence on the entire assessment process. An expert was not always present in the committee for each relevant application area. Applicants should be trained in presenting for a wide committee. Remarkably, the utilization of knowledge was not a mandatory section in the proposal / process. This makes a good comparison impossible. It is advisable that NWO better makes the utilization of knowledge a mandatory part included in the application, or leaves it out. The assessment of the CVs of candidates is complicated. Not in all areas, impact factors are not universally used, making comparison of CVs complicated. It is not clear what position journals hold in different areas. Guidelines for the assessment of the CV can be submitted more explicitly to the referees; an example may be the form of the ERC. Some other notes on the GO Vidi procedure:

13. The diversity of topics is inherent to the cross-border nature of this VIDI call Too specialized proposals were referred to other domains The remaining proposals were often bio-medical, medical-psychological, etc., of nature, and (too?) many jury members had just such a background Very instructive for me as a jury member Sometimes random factors cannot be completely avoided, as the proposals are first reviewed by referent-experts and provided with a score before the plenary review by the pre-advisors committee In retrospect : the cognitive recognition candidate (with A +, A +, A) had made a minitious study on all interview committee members in an attempt what to expect, how to approach them during the interview My own impressions and final remarks (her interview )