1. William E. Haley, PhD School of Aging Studies whaley@usf.edu

2. Thanks to Ardis Hanson, CBCS Research Coordinator (prepared a number of slides) Grayson Norquist, PCORI (shared slides)

3. My background with PCORI Reviewer for 5 cycles (back to early 2013) Ad hoc for Improving Healthcare Systems Standing Member, Addressing Disparities Investigator for successfully funded project (Susan McMillan, PI) “Patient outcomes of a self-care management approach to cancer symptoms: A clinical trial”. 2013-2016. ($1,984,020) Investigator for several other proposals submitted or in process

4. Goals for today What is PCORI looking for? Funding streams and priorities. What is different about PCORI reviews than NIH process? Steps to take before you submit a letter of intent Steps to have completed before you submit a proposal How to make your proposal shine Resources Questions and discussion

5. Authorized by the Patient Protection and Affordable Care Act of 2010 as a non-profit, nongovernmental organization to help patients, clinicians, purchasers, and policy makers make better informed health decisions by “advancing the quality and relevance of evidence about how to prevent, diagnose, treat, monitor, and manage diseases, disorders, and other health conditions.”

6. Funding Funded through the Patient-Centered Outcomes Research Trust Fund (PCORTF), which was authorized by Congress as part of the Patient Protection and Affordable Care Act of 2010 Two funding streams: the general fund of the Treasury and a small fee assessed on Medicare, private health insurance and self-insured plans. PCORI is expected to receive an estimated $3.5 billion from the PCOR Trust Fund to fund patient-centered outcomes research through September 30, 2019.

7. Funding per year For FY 2010, $10 million For FY 2011, $50 million For FY 2012, $150 million For FY 2013, $150 million from the general fund in appropriation plus an annual $1 fee per individual assessed on Medicare and private health insurance and self-insured plans. Estimated total is $320 million. For FYs 2014-2019, the PCORTF will receive $150 million from the general fund in appropriation plus an annual $2 fee per individual assessed on Medicare and private health insurance and self-insured plans and an adjustment for increase in healthcare spending. The combined estimated total averages $650 million per year.

8. Definition of PCOR Assesses the benefits and harms of preventive, diagnostic, therapeutic, palliative, or health delivery system features to inform decision-making, highlighting comparisons of outcomes that matter to people. Is inclusive of an individual’s preferences, autonomy, and needs, focusing on outcomes that people notice and care about, such as survival, function, symptoms, and health- related quality of life. Incorporates a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination. Investigates (or may investigate) optimizing outcomes while addressing burdens to individuals, availability of services, technology, personnel, and other stakeholder perspectives.

9. How does the research PCORI funds differ from Community Based Participatory Research (CBPR)? The main difference is the focus of CBPR on social action, which is not a requirement for patient-centered CER funded by PCORI. Both are similar in the inclusion of end users in the production of evidence.

10. Research Areas of Interest Compares interventions to reduce or eliminate disparities in patient- centered outcomes, including health, health care, and patient-reported outcomes. For example, by accounting for possible differences at the patient, provider, or systems level, we are interested in research to determine what interventions can be most effective for eliminating disparities in outcomes. Identifies and compares promising practices that address contextual factors (e.g., socioeconomic, demographic, or community) and their impact on patient- centered health outcomes. Compares benefits and risks of treatment, diagnostic, prevention, or service options across different patient populations, with attention to eliminating disparities. Compares and identifies best practices within various patient populations for information sharing about treatment outcomes and patient-centered research.

11. Priority Patient Populations Racial and ethnic minority groups Low-income groups Women Children (age 0–17) Older adults (age 65 and older) Residents of rural areas Individuals with special healthcare needs, including individuals with disabilities Individuals with multiple chronic diseases Individuals with rare diseases Individuals whose genetic make-up affects their medical outcomes Patients with low health literacy/numeracy and/or limited English proficiency Lesbian, gay, bisexual and transsexual (LGBT) persons.

12. Sample Research Questions NOTE: All questions must have a comparative component Compare which characteristics of the patient-centered medical care are most critical to implement to improve patient-centered outcomes, reduce disparities, and promote health equity. How does the availability of a patient navigator for patients and/or caregivers improve patients’ health outcomes compared to usual strategies? Under what circumstances, or for what conditions, are patient navigators most effective? Compare best options, materials, and venues for patient education materials that take into consideration patient and caregiver culture, beliefs, literacy, and numeracy to reduce disparities and improve outcomes. How do the practices of the top-performing facilities that primarily serve racial or ethnic minority groups, low-income populations, or other groups at risk for experiencing disparities compare with lower performing facilities? Which of the promising practices are replicable? Given that effective interventions to improve care in vulnerable populations often require a multipronged approach, compare what options work best under different circumstances?

13. Research Studies Should Be based in comparative effectiveness research. Study the benefits and harms of different interventions and strategies that can be delivered in actual settings. Compare at least two alternative approaches. Compares health outcomes that are meaningful to the patient population under study. Address questions about conditions that lead to high costs to the individual or to society. Examine differentials in healthcare resources or costs as a determinant of, or barrier to, good outcomes. Evaluate interventions to reduce health system waste or increase health system efficiency.

30. The Review Process Competitive Letter of Intent PI chooses panel (e.g. Addressing Disparities) Panel members likely NOT experts in condition you are studying Online reviews by two scientific reviewers, two stakeholder reviewers Triage Discussion at face to face meeting—2 minutes per reviewer Vote

31. The review process is unconventional Stakeholder reviewers Scientific reviewers Neither will know the science in your area Relatively little emphasis on preliminary data Relatively little attention to scientific track record of Investigators Heavy attention to stakeholder engagement and community partners

32. PCORI Rating Scale

33. Key points in writing a proposal Your proposal should be “dripping with patient centeredness” Provide evidence that you have engaged stakeholders Provide clear plans that you will engage stakeholders Stakeholders not just advisors—participants Write for your audience—not specialists in your disease condition

34. From the PCORI Funding Guide

35. Criterion 1. Impact of the condition on the health of individuals and populations Is the condition or disease associated with a significant burden in the US population, in terms of prevalence, mortality, morbidity, individual suffering, or loss of productivity? Alternatively, does the condition or disease impose a significant burden on a smaller number of people who have a rare disease? Does the proposal include a particular emphasis on patients with one or more chronic condition?

36. Addressing Criterion 1 Provide prevalence figures Provide figures on health disparities Include information on patient centered outcomes such as quality of life, daily functioning, emphasize these are important to patients

37. Criterion 2. Potential for the study to improve health care and outcomes Does the research question address a critical gap in current knowledge as noted in systematic reviews, guideline development efforts, or previous research prioritizations? Has it been identified as important by patient, caregiver, or clinician groups? Do wide variations in practice patterns suggest current clinical uncertainty? Is the research novel or innovative in its methods or approach, in the population being studied, or in the intervention being evaluated, in ways that make it likely to improve care? Do preliminary studies indicate potential for a sizeable benefit of the intervention relative to current practice? How likely is it that positive findings could be disseminated quickly and affect changes in current practice?

38. Addressing Criterion 2 Make the case that you have an intervention that is likely to have a significant effect size Make the case that it could actually be implemented in usual care settings Show plans for dissemination that involve engagement with organizations representative of stakeholders

39. Criterion 3. Technical merit A clear research plan with rigorous methods and key milestones clearly articulated A research team with necessary expertise, and an appropriate project organizational structure A research environment sufficient to support the conduct of the work with appropriate resources A diverse study population with respect to age, gender, race, ethnicity, and clinical status as appropriate for research A focus on a defined population for whom effectiveness information is particularly needed

40. Addressing Criterion 3 Justify your choice of comparator intervention well Reviewers vary on their like of Usual Care versus Attention Control or Rival Intervention—See FAQ for information on this Be sure that your outcome measures are patient centered and were endorsed by patients/stakeholders

41. Criterion 4. Patient-centeredness Is the research focused on questions that affect outcomes of specific interest to patients and their caregivers? Does the research address one or more of the key questions mentioned in PCORI’s definition of patient-centered outcomes research? How credible are claims that engaged patients and stakeholders will exert meaningful influence on the design and conduct of the research, to ensure patient-centeredness of the questions and outcomes addressed?

42. Addressing Criterion 4 Look at PCORI list of questions defining patient centered outcomes research Be sure that you have ALREADY engaged patients and other stakeholders in identifying research questions, methods Document that stakeholders are enthusiastic about your measures Don’t focus on outcomes such as cost, cost effectiveness, biomedical outcomes such as blood pressure Show how you changed your intervention and/or measures based on interaction with stakeholders

43. Criterion 5. Patient and stakeholder engagement Does the proposal describe how patients and stakeholders were or will be identified and engaged in the research? Are the roles of patients and key stakeholders significant in formulating the study’s research questions, hypotheses and design and in the study’s conduct and dissemination of results? Are the roles proposed for patients and stakeholders in any planned dissemination or implementation plans meaningful and likely to be effective? If engagement is not applicable to the proposed research, does the application justify why it is not?

44. Addressing Criterion 5 Propose an active Advisory Board—meet often Have a large number of diverse patients and family stakeholders-PAY THEM Include some front line practice stakeholders Include concrete plans for continuous engagement of Advisory Board

45. Other Considerations Methodological Studies must adhere to PCORI Methodological StandardsPCORI Methodological Standards Issues of possible heterogeneity of treatment effects must be considered and discussed. Observational comparisons must employ study designs and analytic methods that convincingly protect against selection bias and other threats to validity. Dissemination and Implementation Potential Also address any foreseeable barriers to dissemination and implementation PCORI has separate funding announcements for D&I.

46. Proposal development PCORI constantly revises its procedures Join the PCORI listserve Participate in their webcasts Include an Investigator or have a reviewer who knows the PCORI process Engage the PCORI expert before submitting LOI Plan to engage stakeholders before submitting proposal LOI should note process UNDERWAY to engage stakeholders

47. Before you submit Engage diverse stakeholders in review of your Aims, Methods, Measures, Dissemination Plan Treat stakeholders with respect as collaborators in the process Gain letters of support from stakeholders

48. Final comments and looking to the future PCORI’s approach to LOI unknown Percent screened out unknown Should increase funding rate if invited for full proposal Screening based on existing portfolio Funding beyond 2019 Potential for patient centered research movement to spread more broadly

49. Applicant Resources PCORI http://www.pcori.org/http://www.pcori.org/ Subscribe to PCORI http://www.pcori.org/content/subscribehttp://www.pcori.org/content/subscribe PCORI Funding Opportunities http://www.pcori.org/funding/opportunitieshttp://www.pcori.org/funding/opportunities PCORI FAQ http://www.pcori.org/http://www.pcori.org/ PCORI Funding Awards Past Awards Search: http://pcori.org/pfaawards/?viewby=priorityhttp://pcori.org/pfaawards/?viewby=priority Past Awards Map http://www.pcori.org/research-results/maphttp://www.pcori.org/research-results/map PCORI Methodology Standards http://pcori.org/assets/PCORI-Methodology-Standards.pdf PCORI Patient and Family Engagement Rubric http://www.pcori.org/assets/2014/02/PCORI- Patient-and-Family-Engagement-Rubric.pdfhttp://www.pcori.org/assets/2014/02/PCORI- Patient-and-Family-Engagement-Rubric.pdf