1. MetroHealth Cleveland, OH
Presented by: Josh Kratz, MA, Project Coordinator
PI: Ann K. Avery, MD Project Manager: Jen McMillen Smith, MSSA, LISW-S Evaluator: Mary M. Step, PhD

2. Implementation Timeline 7/27/16 – Alpha testing
9/12/16 – Submission to Apple Store & MetroHealth Server
9/29/16 – Ambassador enrollment
9/29/16 – 10/13/16 – Community building
10/13/16 – Begin criteria-eligible enrollment
10/31/16 – First application update
11/30/16 & 12/23/16 - Additional updates expected
Enrollment (as of 11/8/16)
9 ambassadors, 11 criteria-eligible users = 20 total
(Ambassador badges, private Tales of Triumph, app issue and abuse reporting, and bug-fixes)

3. Enrollment Target: N = 15 criteria-eligible participants per quarter
Current Enrollment (N = 20)
Characteristic n
Gender
Male
Female
Transgender 14 4 2
Race
African American
Multi-racial
White
Hispanic 3 1
Sexual Orientation
Gay
Heterosexual
Bisexual
Pansexual 11
Age
18-24
25-29
30+ 5
Target:
N = 15 criteria-eligible participants per quarter
N = 60 by 8/31/17
N = 100 for 18-mo enrollment period

4. Implementation process status
IRB
7/20/16 – Initial IRB approval
9/23/16 – Most recent revision approval
Staffing
Fully staffed – New: Project Coordinator, Biostatistician
Completed Protocols
1. Enrollment 4. Incoming Communication Flow
2. In-App Admin Posting 5.* Brand Book / Style Guide
3. Social Media Posting CAB
Next meeting February 2017 – Content feedback & Tales of Triumph
(Summary of revisions: additional visit for evaluation, updated recruitment, updated enforcement/study withdrawal protocol, added new participant handouts)

5. Implementation Challenges
SOLUTION(S)
Clear communication of expectations for app functions with tech developer (Blackbird)
Negotiating a written list of clear deliverables with timeline
Logistics of app (IOS) hosting & server
Using app store with password &
advocating for tech developer with hospital IT leadership
Sketchy wifi MetroHealth
Purchased a “hotspot”
Literacy, logistic & length challenges with ACASI multisite evaluation
Good feedback loop to UCLA, clear need for different voice options
Social media surprises: Twitter for pros, IG for PLWHA
Used Instagress boost based on specific #’s: ex: TCC2016, HIV, Safe on the Scene
Clinic location change within the hospital
Literally meeting with participants where they are

6. Recruitment Recruitment Strategies & Pools In Clinic Other Referrals
Community Outreach / Partners
Physician Referrals
Out of Care: 173 out of 576 were criteria eligible in Nov.
Newly Diagnosed: Metro usually receives 100+ newly diagnosed a year. Roughly 60% criteria eligible
Community Outreach: This is includes conferences (Transforming Care, etc.), regional meetings (HIV Case Management Meeting), outreach to partner agencies, and events (such as Pride)
Other strategies include: word of mouth, and targeting those in care but with high viral loads (no way to quantify this number)
Newly Diagnosed (~ 60 per year)
Out of Care (EMR list = 173 in Nov. 2016)
Social Media

7. Recruitment Tools & Where We’re At
Promotional Materials
Social Media
Instagram
90 posts
453 followers
1271 likes
Twitter
44 followers
6,791 impressions in last 28 days
53 Profile Visits in last 28 days
Facebook
- 90 likes 
- 547 reach in the last 28 days
- 2 new page likes in the last 28 days
- 7 page views in the last 28 days 
- 61 post engagements in the last 28 days 
Print Materials

8. Recruitment Challenges
SOLUTION(S)
Over scheduling (Due to ACASI)
Scheduling fewer enrollments daily
No shows / reschedules
Texting/Calling all participants on the day of enrollment appointment
Hours
Being flexible - extended hours
Lost to follow-up
Additional outreach attempts & saturation of app across social media/clinic/community
Saying yes to satisfy doctor (rare)
No pressure, accepting disinterest
External circumstances in patients’ lives
Staying in communication, offering support, & being understanding/patient

9. Discussion PI: Ann K. Avery, MD | aavery@metrohealth.org
Jen McMillen Smith, LISW-S |
Mary M. Step, PhD |
Josh Kratz, MA |