Bottom up Strategy Reform of Open Suite of Programs Full spectrum of CIHR mandate Top Down Strategy Strategic Reform Targeted to specified areas of health research and knowledge translation. These programs and initiatives are intended to: Focus on gaps in specific research areas and research communities or Leverage existing strengths for impact Open to all areas of health research and knowledge translation. This suite of programs is intended to: Capture excellence across all pillars Capture innovative/breakthrough research Improve sustainability of long-term research enterprise Integrate new talent Reform to the Peer Review System 4 To support the strategic directions outlined in Roadmap, three interrelated reforms have been identified.
Programs and Peer Review “Rapid growth, particularly of new strategic initiatives and peer review panels, has led to excessive complexity. This complexity needs to be reduced to enable opportunities and activities to be both focused and manageable.” 6
Facts and Figures The Research Portfolio currently: Handles over 6,500 grant applications per year for review. Relies on the work of 123 review panels and over 2,000 reviewers. Reports unevenness in number of grants reviewed by each committee (between 3 and 60!). Carries out over 200 competitions a year. 7
Facts and Figures (continued) Awards per Program Code For Competition Year 2007-08, EIS data indicates that there were 218 programs (measured as unique program codes in the EIS), leading to a total of 2948 grant awards. The data indicates that a significant percentage of funding opportunities (66%, or 145/218) result in 5 or less grant awards, and that only one program (Operating Grant) resulted in more than 500 grant awards. 145 program codes (66% of codes) resulted in 5 or less grant awards (58PAs, 87RFAs) 8
The objective of the Strategic Reform is to attain greater focus in the use of strategic investments. This new strategic investment process is a key component of this reform. It responds to feedback from CIHR’s community to have fewer more targeted initiatives and is designed to achieve greater impact. It is intended to concentrate limited resources on fewer, but better funded, initiatives and to simplify the interface between CIHR and its partners and stakeholders. It relies on a common sense of strategy and purpose within the organization as well as on a tight collaboration between Institutes. It implies establishing clear differences in objectives between targeted and open grants. Top Down Strategy - Strategic Reform 9
CIHR Signature Initiatives Enhance Patient-Oriented Care and Improve Clinical Results through Scientific and Technological Innovations Support a High-Quality, Accessible and Sustainable Health-Care System Reduce Health Inequities of Aboriginal Peoples and other Vulnerable Populations Prepare For and Respond To Existing and Emerging Threats to Health Promote Health and Reduce the Burden of Chronic Disease and Mental Illness Evidence Informed Healthcare Renewal Canadian Epigenetics, Environment and Health Research Consortium Community Based Primary Health Care Personalized Medicine Pathways to Health Equity for Aboriginal Peoples Inflammation in Chronic Disease Strategy on Patient-Oriented Research: Networks and SUPPORT Units International Collaborative Research Strategy for Alzheimer’s Disease CIHR Research Priority Areas Eight Signature Initiatives are now at varying stages of development and approval 10
Valley 1Valley 2 Basic Biomedical Research Clinical Science & Knowledge Clinical Practice & Health Decision Making Translational Continuum The Strategy on Patient Oriented Research is an umbrella strategy that will be supported by a number of initiatives…. Personalized Medicine Drug Safety Effectiveness Network Community Based Primary Health Care Some examples include….
Considerations for the reform of CIHR’s open suite of programs 12
CIHR’s mandate is to create knowledge and to translate this knowledge into benefits for Canadians through research across the full spectrum. There are currently both real and perceived barriers in the OOGP which limit the ability for this program to support CIHR’s full mandate. There are certain types of ideas that are not being well supported today (e.g. high risk – high impact). There are gaps in the current programming that limits CIHR from ensuring the long-term sustainability of the research enterprise. There is inconsistent application of criteria by peers which creates both real and perceived inequities. Current programs are causing peer reviewer fatigue and placing undue burden on applicants. Issues raised about the Open Programs at CIHR 13
CIHR Expenditures 2000-2010 ($ M Excluding NCE, CERC and CRC) Open and Strategic Expenditures
Objectives for reforming CIHR’s open suite of programs: Capture excellence across all pillars Capture innovative/breakthrough research Improve sustainability of long-term research enterprise Integrate new talent Any program design/change and implementation must take into consideration impacts on: peer review burden applicant burden program complexity cost-effectiveness and efficiency stability (regular and predictable competitions, stable program designs) Bottom Up Strategy - Open Programs 15
The New Open Suite of Programs We have initiated work on the reforming the open suite or programs but are still in the early stages of planning and development. An executive Task Force has been established to oversee the design and implementation of the changes. A dedicated team of CIHR employees has been assembled to provide subject matter expertise and ensure the right level of engagement and discussions are happening with our community. The work has focussed to date on assessing the current suite of funding mechanisms. 16
Programs and Peer Review “The peer review system that is responsible for handling most of the research funding is currently under strain and requires more academic leadership. A review of its processes and structure is necessary.” 18
% Peer Reviewers Applicants & Grantees Institutional Stakeholders Stakeholder Satisfaction – Peer Review Percent of respondents who provided an opinion Satisfied category includes: - Very satisfied; somewhat satisfied and neutral Dissatisfied category includes: - Very dissatisfied and somewhat dissatisfied 19 79 70 54 40 17 26 44 58 48 0 10 20 30 40 50 60 70 80 90 100 Efficiency of the peer review process Fairness of the peer review process Quality of peer review judgements Consistency of peer review judgements Consistency of peer review judgements SatisfiedDissatisfied
Issues raised about Peer Review at CIHR Unsustainable number of peers being currently used that leads to serious difficulties in proper peer recruitment Ad hoc mechanism of peer recruitment Not enough participation of seasoned researchers on review panels Not enough participation from international researchers Not good enough instruction of peers No systematic evaluation of peers performance 20
Enhancements will involve: Creation of a framework for organizing and managing a group of peer reviewers with expertise across the full spectrum of health research: CIHR College of Reviewers Chairs to playing a central role in ensuring that peers with the expertise to cover the full mandate of their respective committee are recruited to the College Enhancing Peer Review Objectives To strengthen peer-review quality in each of the four health research pillars; and To improve the breadth and quality of peer review panels 21
Enhancements to the Peer Review System: Three Streams The College will build the capacity for: Recruitment Training Incentives, Recognition and Performance Identify and mobilize a ready source of expertise to evaluate all applications submitted to CIHR for funding by using a systematic recruitment process. Inform, educate and support college members in their roles within the college which will strengthen organizational leadership in reviewer excellence and development. Establish meaningful incentives for members, recognize excellence and offer a framework for performance measurement of peers, committees and the peer review process. 22
2011 CIHR International Review Key review questions: Has CIHR been effective in fulfilling its mandate at outlined in the CIHR Act? How can CIHR improve at achieving its mandate? 24
Institute reviews - Expert review teams (ERT) Overall review – International Review Panel (IRP) March 30-31, 2011 International Review Panel (IRP) Dr. Elias Zerhouni (Chair) Prof. Rudi Balling Prof. Sir John Bell Prof. Christian Bréchot Dr. Marie-Françoise Chesselet Dame Sally Davies Prof. Victor Dzau Dr. Stephen E Hyman Dr. Jan Lundberg Dr. Chris Murray Prof. Fiona Stanley IGH Prof. Hilary Graham (Ch) Dr. Marianne Legato Dr. Marie-Françoise Chesselet INMHA Prof. T W Robbins (Ch) Prof. Charles P O’Brien Dr. Marie-Françoise Chesselet IAPH Dr. Jeff A Henderson (Ch) Prof. Linda Tuhiwai Smith Prof. Fiona Stanley IA Prof. Carol Brayne (Ch) Prof. Kyriakos S Markides Prof. Fiona Stanley ICR Dr. Anto J M Berns (Ch) Dr. Margaret Tempero Prof. Rudi Balling ICRH Prof. Stephen Holgate(Ch) Dr. Joseph Loscalzo Prof. Victor Dzau IG Prof. Han G. Brunner (Ch) Prof. Jim R. Lupski Prof. Rudi Balling IHSPR Prof. Sally Redman (Ch) Prof. Sally Macintyre Dr. Chris Murray IHDCYH Dr. Richard Johnston (Ch) Dr, Roberto Romero Prof. Fiona Stanley III Prof. Deborah Smith (Ch) Prof. Hidde Ploegh Prof. Rudi Balling IMHA Prof. Alan Silman (Ch) Dr. Matthew H Ling Prof. Victor Dzau INMD Dr. Garrett A FitzGerald (Ch) Prof. W Phillip T James Prof. Christian Bréchot IPPH Prof. Sally Macintyre (Ch) Prof. Don Nutbeam Dr. Chris Murray Feb 8, 2011Feb 9, 2011Feb 10, 2011 International Review Process 25
Sources of Evidence 1.Expert Review Teams and International Review Panel interviews with key informants (CIHR senior executives, Scientific Directors, senior and young investigators, senior government officials, funding agencies, stakeholders (charities, industry, provinces) 2.Institute reviews, self-assessment and response to 2006 review 13 Institute reports (Results of the Institute reviews) CIHR’s Response to the 2006 IRP and current Strategic Plan 3.Survey assessing satisfaction with CIHR performance in core function areas 4.Open invitation on website to provide feedback on CIHR mandate 26
Health Research Community Consultations Survey assessing satisfaction with CIHR performance in core function areas Examined several areas including grants application and peer review Census of grantees/applicants, peer reviewers and institutions 2461 responses to the survey (24% response rate) Noting that respondents could respond to more than one survey type, the distribution of responses is as follows: Open invitation on website to provide feedback on CIHR mandate –442 stakeholders completed the survey –The majority of respondents were affiliated with universities (70%) –The top three categories of respondents were researchers (76%), users of health research (15%) and health administrators (11%) (note: multiple responses possible) Survey TypeNumber of Completed Surveys Applicants/Grantees2198 Peer Reviewers1519 Institutional Stakeholders232 27
CIHR International Review – Next Steps IRP final report presented to CIHR Governing Council June 22-23, 2011 IRP final report posted on CIHR websiteJuly 2011 CIHR Governing Council retreat to discuss the IRP report. Management response to follow. August 23-24, 2011 CIHR International review webpage: www.cihr-irsc.gc.ca/e/31680.htmlwww.cihr-irsc.gc.ca/e/31680.html 28
Institution Engagement Universities Hospitals Research institutions where health research is conducted and applied Reform success depends on partnerships in Canadian health research: 30 CIHR is committed to a renewed approach for strategic communications with research institutions.
Institution Engagement CIHR is committed to strengthening relationships with the institutions through the establishment of a long-term partnership which includes: A CIHR which understands and values the perspective of Institutions and takes that perspective into consideration in its decision-making Institutions that understand CIHR’s programming from both an operational and strategic perspective A shared understanding of the values and beliefs that are embedded in CIHR’s programs and peer review system 31