Presentation on theme: "Patient-Centered Outcomes Research Institute What is it and what does it mean for health services researchers? Patrick S. Romano, MD MPH UC Davis Center."— Presentation transcript:
Patient-Centered Outcomes Research Institute What is it and what does it mean for health services researchers? Patrick S. Romano, MD MPH UC Davis Center for Healthcare Policy and Research March 28, 2012
Speaker’s Verbal Disclosure Statement: Have you (or your spouse/partner) had a personal financial relationship in the last 12 months with the manufacturer of the products or services that will be discussed in this CME activity? _x_ Yes ___ No (If yes, please state disclosures and resolutions) I received a $200 honorarium and reimbursement of travel expenses for participating in a PCORI grant review meeting.
Educational objectives for this seminar: Please type between two and four educational objectives here. To describe the origin and mission of PCORI To differentiate Patient Centered Outcomes Research from Comparative Effectiveness Research (if possible?) To explain the PCORI grant review process and discuss future opportunities for UC Davis/CHPR research support
What is CER? Comparative effectiveness research is the conduct and synthesis of systematic research comparing different interventions and strategies to prevent, diagnose, treat and monitor health conditions. The purpose of this research is to inform patients, providers, and decision- makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations… – Federal Coordinating Council for Comparative Effectiveness Research, definition; [cited 28 July 2010].definition
What is PCORI? Prior to several years ago, no coordinated effort for comparative effectiveness research (CER) 2009 stimulus package provided $1.1 billion for CER (most went to AHRQ and NIH) 2010 Affordable Care Act created Patient Centered Outcomes Research Institute Funded by tax on government and private insurers for each covered life (~$500 million per year)
What is PCORI? Independent, nonprofit, quasi-governmental organization – AHRQ almost eliminated in 1990s after published guideline that surgery for back pain was unnecessary – Provide political protection – PCORI funding does not depend on annual Congressional appropriation No infrastructure for determining funding priorities, issuing opportunities, reviewing and administering grants – Outsourcing (NIH, AHRQ, CMS) Peer review, contract management May receive $ from PCORI to originate grants – PCORI will be its own entity
1. Identifying national priorities for research. 2. Creating a research agenda based on identified priorities. 3. Funding research consistent with these priorities and agenda. 4. Providing patients and their caregivers with useful research information.
PCORI Board of Directors Director of AHRQ (Carolyn Clancy) Director of NIH (Francis Collins) 17 members appointed by the Comptroller General of the United States as follows: – 3 members representing patients and health care consumers – 5 members representing physicians and providers, including at least 1 surgeon, nurse, State-licensed integrative practitioner, and hospital – 3 members representing private payers, including at least 1 health insurance issuer and at least 1 self-insured employer – 3 members representing pharmaceutical, device, and diagnostic manufacturers or developers – 1 member representing quality improvement or independent health service researchers (Harlan Krumholz or Arnold Epstein) – 2 members representing the Federal Government or the States, including at least 1 representing a Federal health program or agency
PCORI Leadership Eugene Washington, MD, MSc, (chair), Vice Chancellor of UCLA Health Sciences and Dean of the David Geffen School of Medicine Steven Lipstein, MHA (vice chair), President and Chief Executive Officer of BJC HealthCare Joe Selby, MD, MPH Executive Director Anne C. Beal, MD, MPH Chief Operating Officer,
PCORI Mission PCORI helps people make informed health care decisions and improves health care delivery and outcomes by producing and promoting high integrity, evidence-based information that comes from research guided by patients, caregivers, and the broader health care community.
PCOR vs. CER Why PCORI, not CERI? Political concerns that determining which treatments better → “rationing” – Restrict patients’ ability to choose inferior treatment Compromise focus on patient-centered outcomes research Legislation barred PCORI from – Developing or using QALYs – Considering cost in making coverage recommendations – In practice, PCORI will not fund cost-effectiveness analyses
Patient-Centered Outcomes Research Not just a semantic difference Focus on PCOR essential role in shaping funding priorities and “rules of the game” for getting funding Can’t just replace placebo control with active comparison group, do everything else the same Distinct elements – Outcomes should be patient-centered – Effectiveness, not efficacy – Heterogeneity of treatment effects – Involvement of stakeholders
Patient-Centered Outcomes Research Outcomes should be patient-centered – “…outcomes that people notice and care about such as survival, function, symptoms, and health- related quality of life” – Not biomarkers (BP, serum cholesterol, etc.)
Patient-Centered Outcomes Research Effectiveness – Research done in clinically relevant populations – Representative of patients – Representative of settings Heterogeneity of treatment effects – “Given my personal characteristics, conditions and preferences, what should I expect will happen to me?” – “What are my options and what are the benefits and harms of those options?”
Role of stakeholders Government payors and regulators; health plans; hospitals and other care providers; industry Consider how results will shape policy and practice through dissemination Stakeholders should be intimately involved in the study design process to identify important outcomes – Trials of anti-seizure drugs in children – Traditional measure – seizure frequency – Patient-centered measure – parent want to know how drug will affect school performance – Drug ↓ seizures but ↑ sedation may not be preferable
Implications for research design RCTs – Pragmatic - real-world settings; large N’s to assess subgroups – Focus on effect size, not just superiority – Bayesian and adaptive approaches to design, accounting for prior probabilities Observational studies – Causal inference modeling – Tension for studies using pure administrative data – powerful and low-cost; lack high-quality patient-centric data Cost and cost-effectiveness – PCORI not allowed to evaluate cost in making coverage recommendations, won’t fund “pure” CEA – Cost can be studied and used by decision-makers
Proposed National Priorities Assessment of Prevention, Diagnosis, and Treatment Options. The research goal is to determine which option(s) work best for distinct populations with specific health problems. Improving Healthcare Systems. Focuses on ways to improve healthcare services, such as the coordination of care for patients with multiple chronic conditions. Communication and Dissemination. Looks at ways to provide information to patients so that they, in turn, can make informed healthcare decisions with clinicians. Addressing Disparities. Assures that research addresses the healthcare needs of all patient populations. This is needed as treatments may not work equally well for everyone. Accelerating Patient-Centered and Methodological Research. Includes patients and caregivers in the design of research that is quick, safe, and efficient.
PCORI Funding Opportunities Developmental RFPs (Sept 2011) ~$30-$100K each Methods for setting research priorities Pilot Projects Grant Program (Dec 2011) $13 million in year 1 for ~40 projects Help develop national research priorities for PCOR Develop ways to engage patients and bring stakeholders together Develop measurement tools and data collection methods for novel patient-centered outcomes
PCORI Funding Opportunities $150 million in current FY; ultimately perhaps $500 million/year Mechanisms unknown, priorities under discussion – Likely NOT “traditional” research – Patient-centered outcomes Including developing methods – Real-world populations – Heterogeneity of treatment effects – Involvement of stakeholders
Other Funding Opportunities Existence of PCORI does not preclude government agencies or foundations from funding CER AHRQ has Effective Healthcare Program – Large evidence-synthesis and dissemination projects – EBPCs, DEcIDE network, CERTS – Does not support investigator-initiated projects Areas of CER explicitly outside PCORI’s mandate – CER with “traditional” outcomes – Cost-effectiveness analyses Remains to be seen
PCORI Pilot Research Projects $13 million in 2012 Approximately 40 awards 2 years maximum support – 2 nd year of funding must be well justified and will be evaluated for successful completion of the first year goals $250,000 per year (plus indirect costs) – Travel to annual PCORI meeting is an allowable expense Requires stakeholder involvement as co-investigators – unless not feasible as for an application on analytic methods
Principles of grant review Scientific and technical merit Fair, competent and thorough Conflict of interest Confidentiality Difference between overall impact and significance – Overall impact: Probability of whether the research will exert a sustained, powerful influence on the research field. – Significance: Does the project address an important problem or a critical barrier to progress in the field? If the aims are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved?
Structure of application Signed Face Page, Table of Contents Project Summary: single paragraph using plain language to summarize the – Broad objectives – Specific aims – Relevance to PCORI Objectives – Research Design and methods for achieving the goals Project/Performance Site(s) Key Personnel and Other Significant Contributors Biographical Sketches–Principal Investigator, Other Biographical Sketches – Personal statement allows each person to explain their role and goals for the project Budget and Justification Resources – Provide information on the facilities to be used in the proposed research project – Describe how the research environment contributes to the probability of success – Describe how project will benefit unique features of the research environment or community involvement or allow for useful collaborative arrangements – Describe institutional and community investment in the success of the research Checklist Addendum Form: Checklist to identify PCORI Areas of Interest
Research Plan Specific Aims (1 page) Research Strategy (10 pages) References Cited Protection of Human Subjects Consortium/Contractual Letters of Support No appendices are allowed and will be removed by PCORI before sending to reviewers
PCORI Review Criteria - Significance Does the project address an important problem or critical barrier to PCOR? Will the result produce new knowledge that can advance PCOR methods or infrastructure? Does the investigator demonstrate thorough knowledge of previous and ongoing work related to their proposed topic?
PCORI Review Criteria - Patient/Stakeholder Engagement Will the project make a unique contribution to learning about engagement of patients and stakeholders? Does the research team demonstrate authentic, feasible partnerships with the patients/stakeholders? Is there evidence that the stakeholders were involved in the preparation of the research proposal?
Is the research team well suited to the project? Is there appropriate scientific expertise? Does at least one member of research team have experience in patient/stakeholder engagement? Will the research team be able to achieve the study aims? Does the research team have complementary and integrated expertise? Is the approach and structure of the research team appropriate for the project? PCORI Review Criteria - Investigator
PCORI Review Criteria - Innovation Does the project address a new method or approach for the field of PCOR? Does the project apply a proven method or approach in a novel way to the field of PCOR? Is there a new application or refinement of theoretical concepts, approaches, methodologies, instrumentation or interventions proposed?
PCORI Review Criteria - Approach Are the overall strategy, methodology, and analyses appropriate to accomplish the specific aims? Are potential problems, alternative strategies, and benchmarks for success presented? Will the strategy establish feasibility and will particular risky aspects be managed Is the budget and timeframe appropriate for the research plan? Are there plans for dissemination among key PCOR stakeholders in education, practice and policy?
PCORI Review Criteria - Environment Will the environment contribute to the probability of success? Are the institutional support, equipment and other physical resources adequate for the project? Will the project benefit from unique features of the research environment, community involvement, patient populations, or non-traditional stakeholder collaborative arrangements?
33 PCORI Review Template Based OVERALL IMPACT SECTION SCORED REVIEW CRITERIA SECTIONS: The six review criteria, Significance, Patient/Stakeholder Engagement, Investigators, Innovation, Approach, Environment, with a separate score for the first five criteria. Overall Impact 1. Significance Strengths Weaknesses
When Writing Critiques Overall Impact paragraph – – requires a summary narrative (paragraph) of the strengths and weaknesses. Do not cut and paste. – Focus on major strengths and weaknesses that impacted your overall rating of the application – Overall impact score is not an average of the criterion scores; weights can differ across reviewers Use bulleted points to make succinct, focused comments (Be careful of being too brief – the reader needs to understand your comment) Any criterion score between 3 to 9 needs to have weaknesses listed 34
Scoring 9-point score scale is used to provide: Criterion Scores for each of the core review criteria Overall Impact/Priority Score based on but not an average of the core criterion scores plus additional criteria All applications receive scores: Not discussed applications (bottom half by pre-meeting scores) will receive only initial criterion scores from the 3 assigned reviewers. Discussed applications also receive an averaged overall impact score from eligible panel members (i.e., those without conflicts of interest). These scores will be averaged to one decimal place, and multiplied by 10. The 81 possible priority scores will thus range from
9-Point Score Scale Descriptors ImpactScoreDescriptorAdditional Guidance High 1ExceptionalExceptionally strong with essentially no weaknesses 2OutstandingExtremely strong with negligible weaknesses 3ExcellentVery strong with only some minor weaknesses Medium 4Very GoodStrong but with numerous minor weaknesses 5GoodStrong but with at least one moderate weakness 6SatisfactorySome strengths but also some moderate weaknesses Low 7FairSome strengths but with at least one major weakness 8MarginalA few strengths and a few major weaknesses 9PoorVery few strengths and numerous major weaknesses
Process During the Meeting Chair calls for preliminary overall impact scores Primary reviewer – Brief description – Discuss major/ score-driving strengths and weaknesses Second and third reviewers add only major/score-driving strengths and weaknesses Open discussion among all reviewers Human subjects issues Final scores from assigned reviewers Unassigned reviewers may vote outside the score range Record scores (in IAR and on hard copy) Budget (does not influence score!) minutes per application
Reviewer Voter Sheet (available only during the meeting)
Acknowledgments Michael Steinman MD, UCSF CTSI Director of Comparative Effectiveness Research Jonathan Arias, PhD, NIH Center for Scientific Review All of you