1. Stakeholder Engagement in Comparative Effectiveness Research Danielle Lavallee, PharmD, PhD Research Assistant Professor UW Department of Surgery July 13, 2015

2. Objectives Define stakeholders and stakeholder engagement for research Describe approaches to obtain stakeholder input to inform research Present an approach for planning stakeholder engagement in CER

3. “Failure to attend to the information and concerns of stakeholders clearly is a kind of flaw in thinking or action that too often and too predictably leads to poor performance, outright failure or even disaster.” ~ J. Bryson, Public Management Review (2004)

4. Background “Policy-makers and purchasers are interested in identifying sources of cost without benefit. Clinicians need to select effective treatments, and patients want to make informed treatment choices. Outcomes research—the study of the end results of health services that takes patients' experiences, preferences, and values into account—is intended to provide scientific evidence relating to decisions made by all who participate in health care.” ~Clancy CM, Eisenberg JM 1998

5. Purpose of CER The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels. Institute of Medicine

6. Why the focus? Constrained resources Change from acute health to chronic health management Rate of scientific discovery is increasing ─ More to know ─ More to do ─ More to manage (i.e. care coordination) Estimated 17 years for new evidence to reach practice

7. Increasing Demand

8. What topics are interesting to research? How do we measure change? Where do we publish? What issues are important to decision- makers? What outcomes are important to inform decision-making? How do we translate into practice? A Paradigm Shift Traditional New Approach

9. Stakeholders Individuals, organizations or communities that have a direct interest in the process and outcomes of a project, research or policy endeavor Patients & Caregiver s Clinicians Payers & Purchaser s Researche rs Policy- makers

10. Stakeholder Perspectives StakeholderPerspective Consumers, patients, caregivers, patient advocates Which aspects of an illness are of most concern? Which features of a treatment make the most difference? How are research results best presented and acted upon? Clinicians and Prof Societies What gaps in evidence regarding diagnostic or treatment choices are causing the most harm to patients? What evidence is required to support guidelines or practice pathways that would improve the quality of care? Purchasers and payersWhere does unclear or conflicting evidence cause difficulty in making the decision of what to pay for? Where is new technology or new uses of technology raising questions about what constitutes a standard of care? Health care policymakers at the Federal, State, and local levels Where is new technology or new uses of technology raising questions about what constitutes a standard of care? What research is or could be funded? What evidence is needed to help plan public programs? Industry & Industry Associations Provide unique information about treatments and devices Adapted from the Effective Health Care Program Stakeholder Guide: http://www.ahrq.gov/research/findings/evidence-based- reports/stakeholderguide/chapter3.html#Stakeholdershttp://www.ahrq.gov/research/findings/evidence-based- reports/stakeholderguide/chapter3.html#Stakeholders

11. Stakeholder Engagement A process of actively soliciting the knowledge, experience, judgment and values of individuals selected to represent a broad range of direct interests in a particular issue, for the dual purposes of: 1) Creating a shared understanding; 2) Making relevant, transparent, and effective decisions. Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, Tunis SR. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. Journal of Comparative Effectiveness Research, 2012; 1(2): 181-194.

12. Qualitative Approaches to Engagement Preskill H, Jones N. A practical guide for engaging stakeholders in developing evaluation questions. Robert Wood Johnson Foundation. Available at: http://www.rwjf.org/pr/product.jsp?id=49951http://www.rwjf.org/pr/product.jsp?id=49951

13. Where to Start? Not a traditional aspect of clinical research No standard approach exists Timelines can be tight so plan early!

14. Why? Why am I planning to engage stakeholders in my research? ─ Do I want help prioritizing research? ─ Do I want to understand what outcomes are important? ─ Do I want to understand how stakeholders seek/use information? What are the risks and opportunities for engagement? ─ Is this a contentious topic? ─ What are the risks of not engaging stakeholders? Why not? ─ If stakeholders are not included – understand and articulate why they are not directly involved

15. Who? Who are my stakeholders? ─ What perspectives are needed and why? Consider diversity in both populations represented & opinions present ─ Who from each group is best to engage? Which one(s) - advocates versus patients/caregivers versus consumers? What role/position does the person hold? Does the person need decision-making power?

16. Perspectives & Needs Vary Research Activity Why?Characteristics of Who? Topic Prioritization Protocol Development Translation & Dissemination

17. Research Activity Why?Characteristics of Who? Topic Prioritization Identify gaps in evidence Identify priorities Identify dissemination avenues Possess broad perspective Access to a larger community Funders?? Protocol Development Identify comparators for research Identify populations of interest Identify outcomes of interest Decision-makers Current topic experience/knowledge Translation & Dissemination Help craft messaging Identify venues for presenting Representatives of target audience Communication/editorial experience a bonus! Perspectives & Needs Vary

18. What? What are the objectives of engaging stakeholders? ─ Developing research guidance ─ Developing a research agenda ─ Designing a research plan ─ Disseminating information What are the methods for engaging stakeholders? ─ Members of research teams ─ Members of an advisory group ─ Qualitative approaches (Focus group, Surveys, interviews)

19. When and Where? When will stakeholder engagement occur in the process? How frequently will engagement occur? ─ One-time engagement vs. engagement throughout the life of a project ─ For long-term activities considers approaches to sustain and maintain involvement Where will activities be conducted? ─ In-person? Conference call?

20. How? How will we measure success? ─ What criteria will be used? ─ How will evaluations occur (survey, interview, etc) How will we incorporate findings for improvement? How will we inform stakeholders of progress?

21. Stakeholder Engagement in Practice: Working Examples

22. Research Proposal Development Engaged with Patients, Clinicians, Healthcare Administrators, Funders, and Researchers Multiple approaches to engagement utilized Planning over 7 months Non-funded work

23. Stakeholders as Advisors Back pain Outcomes using Longitudinal Data (BOLD) ARRA Funded Registry Multi-stakeholder Advisory Group ─ Receive updates on research ─ Advise on study issues/challenges ─ Discuss findings of research ─ Discuss future research opportunities ─ Assist with dissemination activities

24. Final thoughts Patient/stakeholder engagement plans should link to the research plan Allot time for planning and involving stakeholders Build relationships – collaboration becomes easier when you know the stakeholder landscape Evaluate the process from both the stakeholder and research team’s perspective Document activities and results

25. Take Home: Planning Tool Stakeholder Why is this stakeholder important to engage? What issues are important to this stakeholder? What are the expectations for engaging with this stakeholder? Are there any particular skills or experience that this stakeholder should have (i.e. personal experience, policy development, clinical practice) How might this stakeholder be engaged throughout the research continuum?

26. Questions?