3. Strategies for Engaging and Utilizing Stakeholders and “Gate Keepers” Donna Dent, MS, MISM Sally Honeycutt, MPH Associate Coordinator Evaluation Consultant Service to Science Lead Evaluation Team Lead, Emory Prevention Research Center SAMHSA’s Southeast Center for the Application of Prevention Technologies

4. Welcome to Connect Pro We will be using the following windows: PowerPoint Window Attendee List Chat Note We will also use the “Raise your Hand” feature Shannon Greer, Southeast IT Specialist

5. Introduction to Connect Pro Raise your hand by clicking You will see your status change in attendees list Un-mute your phone (press * #) when called upon Remember to mute your phone again once you have finished speaking Lower your hand by again clicking

6. Asking Questions in Connect Pro By writing your question in chat window Press enter Only the moderator will see the question

7. Facilitator and Presenter Donna Dent, MS, MISMSally Honeycutt, MPH CAPT Associate CoordinatorSTS Evaluation Consultant Southeast Service to Science LeadEmory University Professor

8. Objectives The participant should be able to: Define the term “stakeholder” Differentiate between a stakeholder and a “gate keeper” Explain at least three significant roles for stakeholders with regard to evaluation

9. Objectives (cont’d.) Identify at least three potential stakeholders for their program and explain the benefit they add to the evaluation process Discuss at least three strategies one might use for keeping stakeholders engaged over time Explain at least two reasons stakeholders might become involved and then stay engaged in a program

10. I What is a Stakeholder?

11. Stakeholders Have a vested interest in whatever is being evaluated Are in a position to use findings –May use findings in different ways

12. Types of Stakeholders Primary stakeholders –Are project beneficiaries Secondary stakeholders –Are directly involved with/responsible for primary stakeholders –Are affected by the project (jobs, lives) Key stakeholders –Are government officials/policy makers –Can influence others –Have an interest in the outcome of an effort (U. Kansas, 2008)

13. Gatekeepers vs. Stakeholders? Gatekeepers: Control access to people, resources, or information needed for evaluation May or may not overlap with stakeholders Can you give some examples?

14. II Why Engage Stakeholders?

15. Benefits of Stakeholder Engagement Increase the relevance and usefulness of the evaluation Improve evaluation process (RWJF, 2009) (U. Kansas, 2008)

16. Why Team Up? Share (mission/goals) – increase capacity, create unison Access – direct or indirect access to target audiences Credibility – Double-edged sword Nontraditional perspectives – creativity, invention, problem-solving, cultural inclusion Saboteurs Cast a wide net vs. focusing efforts

17. Challenges in Teaming Up Poll #2 Which of these challenges have you experienced in working with stakeholders and partners?

18. III How Do We Engage Stakeholders?

19. Five-Step Process for Engaging Stakeholders 1 Prepare for stakeholder engagement 2 Identify potential stakeholders 3 Prioritize the list of stakeholders 4 Consider motivations 5 Select an engagement strategy (RWJF, 2009)

20. 1. Prepare for Stakeholder Engagement Have a plan for recruitment, participation, and exit Beginner Plan – Need to identify and recruit a board/partners Intermediate Plan – You have a board or a few partners but they are not stable or large enough for the amount of work Advanced Plan – Have a functioning board, partners, workgroups, etc., just would like to improve working relations, efficiency and/or retention.

21. Stakeholder Engagement Plan Identify structures, roles and processes Identify representation needed Allocate resources, assign responsibilities Write down

22. Stakeholder Engagement Plan (cont’d.) Define and explain Recruitment strategies and activities Retention strategies and activities Exit/repurposing strategies and activities

23. 2. Identify Potential Stakeholders Community partnerships for evaluation (data collection activities, locating, and accessing existing data) Populations that directly benefit (participatory research) Engaging comparison groups who may help with evaluations Evaluators (Connecting with universities, graduate students, contractors)

24. Who are the Stakeholders and Gatekeepers? How to identify? Consider motives? Natural fits Unexpected surprises Creating alliances Changing regimes - do you have all key team members? Is there an exit strategy?

25. Who are some potential partners that you can think of? Traditional and Nontraditional Potential Partners

26. Gate Keepers: Captive Audience People at meetings People in restroom stalls People getting other services (beauty salons, nail salons) Waiting rooms (auto repair shop, lawyer office, cable television office, doctor office)

27. Think Outside the Box Captive Audiences –Waiting rooms, restroom stalls, train, planes, subways, post offices Heightened Sensitivity –Neighborhood clinics, emergency rooms, (shared experience) –Churches, funeral parlors, auto repair shops

28. Think Outside the Box (cont’d.) Targeted Audiences (demographic) –Age group specific (music store, nursing home, army retail store) –Gender specific

29. 3. Prioritize the List of Stakeholders (U. Kansas, 2008)

30. Considerations What does this stakeholder bring to the evaluation? How important is their perspective for the evaluation? –Vital –Important –Nice (RWJF, 2009)

31. 4. Consider Potential Stakeholders’ Motivations for Participating Commitment to program’s goals Personal stake in the program Professional benefit –Professional development –Networking opportunities Financial considerations –May be in-kind (RWJF, 2009)

32. Describe Potential Stakeholders Who are your stakeholders? (List potential stakeholders) How much influence and interest do they have? (Indicate High or Low for each) How important is their perspective to the evaluation? (Check one) What may motivate their participation? (Check all that apply) Influence Interest Vital Important Nice Commitment Personal Stake Professional Benefit Financial Other

33. 5. Select an Engagement Strategy Considerations Stakeholders perspectives and relationships Program characteristics Logistical

34. Stakeholder Engagement Considerations & Techniques Will you face these issues in engaging your stakeholders?YesSomewhatNoNotes Stakeholders perspectives and relationships Differing perspectives Lack of existing relationships Little familiarity with evaluation Little familiarity with the program Program characteristics Complex program/initiative Logistical Short evaluation timeline Limited budget Geographically dispersed stakeholders Large number of stakeholders Limited stakeholder availability

35. Stakeholder Engagement Considerations & Techniques Engagement Technique I will use this technique… (Check all that apply) To engage these stakeholders… (List individuals and/or groups) At this time… (List sequence and/or dates) With these resources (List funding, staff, other resources required) One-On-One One-on-One Interviews Group Meetings Logic Modeling Mind Mapping Appreciative Inquiry Role Playing Brainstorming/NGT Focus Group Interviews Discussion of Article/ Presentation Moderated Discussions (in person) Moderated Discussions (virtual) Surveys One-Time Survey Delphi Technique

36. IV What Will the Stakeholders Do?

37. CDC Framework for Program Evaluation in Public Health (U. Kansas, 2008)

38. Challenges in Teaming Up Poll How have stakeholders contributed to your evaluation?

39. Engage Stakeholders Assist in recruiting other stakeholders Provide input on engagement plan Develop common understanding of expectations

40. Describe the Program Develop (or review) logic model Ensure agreement on –Program activities –Intended outcomes Inputs Activities Intermediate outcomes Long-term outcomes Outputs Short-term outcomes

41. Focus the Evaluation Design Clarify evaluation purpose –Why are we doing this evaluation? Agree on intended users and uses? –Who will use evaluation findings? How? Assist in developing evaluation questions –What do we want to know? Provide input on appropriate methods –How will we find out what we want to know?

42. Gather Credible Evidence Review/provide input on data collection instruments –Literacy –Cultural appropriateness –Length Assist in data collection process –Access to data or populations needed –Additional data collectors –Data collection decision rules

43. Justify Conclusions Help interpret findings –Provide context for data –Understand meaning behind statistics/facts –Determine practical significance Generate recommendations –Data driven –Understanding of program and context

44. Ensure Use and Share Lessons Learned Stakeholders are evaluation users Develop useful products for different audiences Disseminate findings –May assist with distribution, marketing of results –Provide materials to relevant audiences –Communicate evaluation findings

45. Thank you! Donna Dent CAPT Associate Coordinator Southeast STS Lead ddent@edc.org 678-954-5822 Sally Honeycutt, MPH Evaluation Consultant Evaluation Team Lead, Emory Prevention Research Center sallyhoneycutt@gmail.com 404-997-2410 Additional resources are listed in the handouts