1. MIRA Trial Standard of Care (SOC) Impact Evaluation
Presented by Liz Montgomery, MHS
RTI International
19-20 June 2008,
SOC Consultation, Washington, D.C.
Naomi – I think it is weird to list your name and then presented by…I will say your name at the start, and then I have added it to another slide. If it is here, I think people will think I am Naomi. ok

2. MIRA Standard of Care (SOC)
Overall MIRA SOC goal: To actively facilitate linking trial seroconverters to existing, national care and treatment programs –
Explicitly, we did not plan to assume responsibility for care and treatment
AS NOTED AT CONSULTATION IN SOUTH AFRICA:
We initiated our SOC program in November/December 2006.
You can see here that our overall goal for the program was to …

3. MIRA Standard of Care: Process
Strengthen local referral sites
Formalized agreements with referral sites regarding service provision and tracking systems for HIV+ women
Introduced individual case management
Filled in service provision gaps where needed
AGAIN AS NOTED AT CONSULTATION IN SOUTH AFRICA:
We had a 4 part plan for achieving that goal.
1. Strengthen local referral sites : Where applicable, conduct needs assessments and respond within means of SOC Program
The first step was to find out how our local HIV referral sites were functioning, and if necessary to do what we could to strengthen them. The local SOC Coordinators visited referral sites and for some, conducted needs assessments. In some cases those assessments resulted in extra staff going to the site to help out. In other cases, the referral sites requested things that MIRA couldn’t provide. I don’t want to go too far into this topic. I will touch on it again in the lessons learned section. The take-away point here really is that this process seemed to be a good entry point for the SOC Coordinators as they introduced the SOC Program.
2. Formalization of agreements with referral sites regarding service provision and tracking systems
The second step for achieving our SOC Program goal was to formalize agreements with the referral sites. I mentioned before that before the SOC only in some cases did we have formal Memorandums of Understanding in place. For the SOC, we tried to formalize agreements with as many of the referral sites as possible.
3. Individual case management of seroconverters
Psychosocial support and counseling,
Monitoring of disease progression using CD4 counts,
Screening for opportunistic infections, and
Access to ARV therapy.
4. And finally, our fourth step was really a decision to fill in the gaps where needed.
So we decided to conduct CD4 tests or manage opportunistic infections if necessary until referral sites were prepared to handle each participant’s case independently. And we also decided to directly pay for services on behalf of the participants where needed. Usually that was done until we could link the participant with social welfare services or grants that would cover the costs of services. We did not pay for ARVs for any participants.

4. SOC Preliminary Results
We monitored each seroconverter case file to ensure that the processes described in the SOC plan were followed uniformly. We’re working on a publication that will include the outcomes of that work, challenges, etc.
Preliminary results: (ALL SITES combined.)
Seroconverters: 327
UZ-UCSF 125
MRC 153
PHRU 49
Received info about SOC and attended MIRA clinic to get info: 148
Received referrals to public facilities: 130
We have record of ppt having accessed care: 84
On ARVs by end of trial: 18

5. MIRA STANDARD OF CARE: IMPACT EVALUATION (SOC IE)
Led by Naomi Lince, Ibis Reproductive Health

6. Standard of Care Impact Evaluation (SOC IE)
Overall goal of SOC IE: To assess the effect of MIRA trial referrals and, where applicable, relationships established between the referral site and the local, MIRA coordinating institution on service provision, quality, capacity, etc. at the referral site.

7. SOC Impact Evaluation: Aims
Describe and evaluate process of establishing agreement;
Identify benefits and challenges resulting from relationship with MIRA trial;
Assess the effect of the relationship with MIRA on referral site service provision;
Elicit recommendations for future collaborations.
The specific aims of the SOC Impact Evaluation, were to:
Describe the process through which the MIRA researchers and the referral site established and maintained an agreement (i.e. verbal or written) regarding the care and treatment of MIRA trial participants;
Identify both actual and perceived, potential benefits and challenges presented by the existence of a study such as the MIRA trial in the referral site community;
Assess the impact of the referral site-MIRA trial relationship on referral site service provision as a whole and for MIRA trial participants specifically;
Elicit recommendations for future collaborations between researchers and the referral site (and possibly the larger community).

8. 13 SOC Referral Sites UZ-UCSF – Zim MRC - Durban PHRU - Soweto
Psycho-social support and counseling OR
Social services
Site 1
Site 2
No community-based providers. Provided by MIRA staff.
Site 11
Site 12
Monitoring of disease progression & screening/ treatment for OIs
Site 3
Site 4
Site 5
Site 6
Site 7
Site 8
ARV therapy
Site 3 (repeat)
Site 4 (repeat)
Site 5 (repeat)
Site 6 (repeat)
Site 9
Site 10
Site 13
There were 13 sites identified as referral sites for seroconverters across the three MIRA sites. You can see that we aimed to create a network of sites that could provide the various services that were key for seroconverters in the trial.
9 were selected for inclusion in the SOC IE. They are the ones in yellow here.
2 sites received 0 referrals or had a referral but no relationship with the MIRA trial researches. 2 additional sites at PHRu were not included only received referrals indirectly from the MIRA team.

9. SOC IE: Methods Sept ’07 – Jun ’08 Selected 9 of 13 SOC referral sites
Aimed to conduct 2 semi-structured interviews per site:
Manager/head administrator
Head clinician/nurse
Collected organizational information via survey at each facility
Analysis in SPSS
At each facility, we aimed to interview both a manager or program director and a lead clinician.
We also completed an organizational survey with the manager at each site to learn a bit more about their funding, overall service provision, history, etc. We’re not presenting this data in this presentation.

10. Results: Participants (n=13 from 9 sites)
Interviewed:
8 administrators, 5 clinicians
11 live in or are originally from the area that the clinic serves
7 worked at the facilities since before MIRA was initiated
12 heard about MIRA before SOC was initiated
5 had personally worked on or conducted a research project
11 are working with other researchers now (either MIRA researchers on different studies or other researchers)
We’re presenting the findings for managers/administrators and clinicians together since most managers/administrators reported that they had both types of responsibilities in the facility.
Almost all are currently living in the community or are from the community that the org serves
Almost all have been working at the facility since before MIRA was initiated and remember hearing about MIRA before SOC was initiated. This makes sense since we tried to establish referral networks at the start of the trial.
NB: In Zim, there was one respondent who said she’d never heard of MIRA, even though our key informants said they had worked with her regularly. We suspect that maybe she was nervous due to the political situation?

11. Results: Opinions of research
“What is your general opinion of research or health-related research?”
6 - Generally positive opinion
6 - Positive opinion, but qualified opinion (i.e. research good, but beware of false hope or exploitation)
1 – Question validity of results due to self-reporting
0 – Generally negative opinion
Here we asked, “What is your general opinion of research or health-related research?”
EXAMPLES OF THE FIRST TWO RESPONSE TYPES ON NEXT SLIDE…
REGARDING THE “VALIDITY OF RESULTS” COMMENT, THE RESPONDENT SAID JUST THAT – THAT “MOST RESEARCH IS BASED ON THE HONESTY OF THE PATIENT. ONE WONDERS HOW VALID THE INFORMATION IS IN THE END.”
NO ONE HAD A GENERALLY NEGATIVE OPINION.

12. Aim 1 Results: Knowledge of agreements with MIRA researchers
5 – Formal, written agreement
3 – Verbal only
3 – There was agreement but don’t know type
1 – Not sure if there was an agreement
1 – No agreement
Aim 1 was: Describe and evaluate process of establishing agreement;
Maybe discuss reports compared to actual.. E.g. We know that MRC had MOUs with all facilities. Naomi is investigating whether UZ-UCSF and PHRU had written agreements with all.
Generally agreements included information on referrals, tracking MIRA ppts, provision of services, etc.

13. Aim 1 Results: Feedback on process of establishing agreement
What are your opinions on the process of making the agreement with the MIRA researchers? (open ended)
8 – It was fine/ok
1 – Should have been more communication
1 – It should have been more formal
3 – No comment/opinion
0 – It should have been less formal
Here we asked, “What are your opinions on the process of making the agreement with the MIRA researchers?”
Almost all said that relationship with MIRA researchers was fine.
The 3 who didn’t comment weren’t included in the process/didn’t know what it was.

14. Aim 2 Results: Benefits
Any benefits of having MIRA ppts referred to this facility or working with MIRA researchers?
10 Yes, benefits to the facility
6 Reached target clients/increased enrollment in services
3 Got information (generally or about research)
1 MIRA funded additional staff/space
2 Benefits to clients, not facility
1 No benefits
Aim 2 was: Identify benefits and challenges resulting from relationship with MIRA trial;
Benefits to clients were 1) ppts got ARVS (at facility) and 2) women who were referred to facilities got services there.

15. Aim 2 Results: Challenges
Any challenges of having MIRA ppts referred to this facility or from working with MIRA researchers?
4 No challenges
8 Yes, challenges for facility (some gave >1 response)
3 Lack of resources/staffing problems
2 Workload high/difficult to add special cases
2 Ppts had expectations of reimbursement
2 Insufficient communication
1 Ppts don’t want ARVs/care close to home
1 No for facility, Yes for clients
Insufficient communication included:
Clients didn’t know who researchers were
MIRA staff didn’t meet with clinic staff enough
1 No for facility, Yes for clients – this one was “There’s a stigma attached to [this facility]. Clients referred here are stigmatized.”

16. Aim 3 Results: Impact on services
We asked if having MIRA trial ppts referred to facility or working with MIRA researchers had any effect on the following:
Way services are provided
Number of people facility could provide services to
Kinds of services provided
Quality of services provided
Aim 3 was: Assess the impact of the relationship with MIRA on referral site service provision;

17. Aim 3 Results: Impact of referral on services (cont)
Results were as follows:
9 No impact on services at all
3 Yes, impact on at least one aspect
Way services provided (Nurses had to list MIRA ppts in a book)
Number of people increased
“The community was more aware of services here; more people came and benefited.”
Kinds of services (MIRA ppts knew more re HIV, so counseling content for them was different.)
MIRA org allowed sharing/use of their space
“No change in services, but most ppts came well informed. That made it easy for us.”
Most said there was no impact at all which is good news for us.
3 of 13 reported an impact. 1 said having MIRA ppt referred impacted the way they provided services in that they had to list MIRA ppts in a book.
1 said it impacted the number of services they could provide services to, but the question here might have been misunderstood. She only commented that they saw more people. She didn’t explain how or why.
1 said that MIRA ppts required less info or time when counseling because they already knew quite a lot about modes of HIV transmission, preventing HIV, etc.

18. Aim 4 Results: Recommendations
We used open ended questions to ask about recommendations for future collaborations between the facility and the researchers:
To address the needs of the community
To address the needs of the organization
Or generally.
All 13 had recommendations. Multiple responses were allowed.
Aim 4 was: Elicit recommendations for future collaborations.
Multiple comments were allowed

19. Aim 4 Results: Recommendations (cont)
Comments included:
4 Increase communication
4 Incentives/reimbursement for ppts or facility
3 Consideration of workload at facilities
3 Consideration of needs of clients/community
1 Start processes sooner
1 Media/results release (Be careful not to stigmatize area during results release via release of prevalence statistics.)
Aim 4 was: Elicit recommendations for future collaborations.
Multiple comments were allowed

20. Aim 4 Results: Recommendations (cont)
“It is important to address the needs of the facility and the community. Sign the letter, etc, but also discuss space, etc with staff.”
“Let the referral list be long and holistic.”
“I would have wanted to do more for the clients – to assist with supporting them as research ppts.”
Aim 4 was: Elicit recommendations for future collaborations.
Paraphrase some of the recommendations on this slide

21. Aim 4 Results: Recommendations (cont)
“I went alone to the results meeting and was very interested in future research. For the next trial, could a rep come to the clinic – to a staff meeting – and give feedback?”
“Community meetings felt staged – fancy folders, long language. It felt like the community was only there because the MIRA researchers had asked.”
Aim 4 was: Elicit recommendations for future collaborations.
Paraphrase some of the recommendations on this slide

22. Discussion (1) Limitations Small n
Time since study or SOC initiation/closure
Small number of referrals at each site
LONG TIME SINCE END OF STUDY…
Small numbers referred to each site…though maybe this should be viewed not as a limitation of the study, but rather an indication of impact?

23. Discussion (2) Staff at facilities have personal biases
How to work with them?
Can we have parallel process to CABs?
Referring ppts can be seen as a benefit!
Challenges and impact seem to have been minimal
Process of follow-up may strengthen future and ongoing relationships
PERSONAL BIASES of STAFF. HOW CAN WE WORK WITH THEM? PARALLEL PROCESS TO CABS ?
We worry about referring ppts – and that being extra work for these sites. While some did say their workload was a problem, others said that by referring we helped them reach their enrollment or service provsion goals.
It seems that overall challenges or impact on services was minimal. In fact, the impact mentioned seemed to have been positive.
Small numbers mean that results cannot be statistically significant, but this impact assessment seems to be a case study of each of the referral facilities, which could have other advantages – as these sites will probably be used for referral in other studies conducted at these institutions, so provides important feedback for when they are used again.

24. Thanks! & More Information
Ibis Reproductive Health
Women’s Global Health Imperative
Cervical Barrier Advancement Society
Planned Parenthood
Diaphragm Renaissance
Here are a few websites in case you’d like to have more info about the trial or about diaphragms for contraception or dual protection.
OTHERS, LIZ? RTI? WGHI?