1. Personal Comments on the NSERC ICT Panel’s Decision-Making Process Carl McCrosky

2. Panel Evaluation: Quality & Originality These aspects are the hardest to define, but often the easiest for the panel to judge. Based on the external reviews and those panel members with related expertise, the panel comes to an appreciation of the degree of Q+O. Without sufficient Q+O, the application does not merit further detailed evaluation (although it does continue through the review process, just with less in-depth examination). One factor lumped in with Q&O is the quality of the applicant(s). It is important that at least one applicant be perceived as being a very high quality researcher. While an established researcher is easier to judge positively, more junior ‘stars’ can certainly be considered high quality researchers.

3. Panel Evaluation: Relevance & Impact The proposal must be relevant to, and have a beneficial impact on, industry. This R&I must fall within one of the targeted areas. The key issues are whether some substantial or promising Canadian industry –fully understands, and hopefully contributed to, the proposal, –is willing to work with and track the progress of the proposed work, and –is in a position to benefit from the work and bring some result to a successful commercialization. Cash from industry is nice, but not essential. Their time and interest is vital. Some research entities (e.g. TRLabs) do not directly count as industry support. Check with NSERC if you have questions. NSERC officers can be contacted to discuss R&I issues.

4. Panel Evaluation: Effectiveness It is necessary to convince the panel that the work will be carried out effectively. This is accomplished in two ways: –The work plan must reflect a detailed understanding of exactly how the research will proceed. –The budget must reflect the requirements of the work plan. In both work plan and budget, vagueness is very damaging to an application’s chances. Beware of “level” work plans and budgets!

5. Panel Evaluation: Training Potential The proposed project must include some training of students (MSc, PhD). –Other training may add some minor value, but student training is vital. –The more students trained, the better, so long as it makes sense. NSERC officers can be contacted to discuss this issue.

6. Panel Evaluation: Application Ranking To be seriously considered, an application must meet minimum requirements on each of the four preceding general areas (Q&O, R&I, E, T). Seriously considered applications are then ranked on a ‘sum’ of their values on the four general areas. The ranking is reviewed in detail, with close neighbours on the ranking being given detailed A vs B comparisons. Many adjustments to the rankings are made during this lengthy adjustment process. Finally, as many applications at the top of the list as can be funded are recommended for funding.

7. When Preparing your Application Think and write clearly! –The panel is stressed by the application load number of applications, range of areas of applications, poor quality of some applications. –Few can be funded –So poor communications can be an easy reason to not fund. Seek advice from –Colleagues, –Previous grant winners, –Panel members and previous panel members. Check that you’ve not missed a major requirement (e.g., student training), as missing requirements can kill otherwise great applications.

8. General Quality of Applications 10% of the applications are first rate and without significant defect. These obviously deserve funding. 10% of the applications are quite poor. The remaining 80% is a more-or-less continually improving continuum. –The lower 40% typically has some significant flaw. The panel would like to fund the top 10% and the upper 40% of the middle cohort. As you know, funding does not allow us to go that deep.

9. If Your Application is Rejected Ask the NSERC officer if it makes sense to try to move the application to another program. The committee often considers this possibility, so the NSERC officer may be primed for the possibility. Consider reapplying the next competition. –But do not simply resubmit the same application! Remarkably, the panel has a good collective memory. This, coupled with the irritation that comes with unnecessary overwork when weak applications are re-reviewed, leads to repeated rejection and a bad reputation with the panel. –While you might have been funded if more funds were available, there is definitely some weakness in the application that caused it to be ranked too low for funding. Fix the problem or weakness! –It is possible to discuss weaknesses with panel members.

10. Is the Panel Process Strong? I have served only one year, so had much to learn in Ottawa last fall. My impressions from year 1: –The panel has some very strong members. –The NSERC officers are very bright, knowledgeable and dedicated. –The review process is very difficult, due to the volume of applications, the range of subject areas, and the shortness of time. –The quality of the top half of the applications was very strong. –The quality of the funding decision process was high, and makes me confident in the process. –If an application is not funded, it is generally because the competition is quite tough, and perhaps you made a mistake in your application.