Soumya Alva, Sujata Ram, Anne Langston ICF International, May 15, 2012 Community Systems Strengthening Programs in Cambodia: Findings from a Recent Evaluation for the Global Fund Soumya Alva, Sujata Ram, Anne Langston ICF International, May 15, 2012

Motivation for the Evaluation The role of community health workers and CSOs is key Frameworks discuss ways in which community systems contribute to improving health outcomes Indicators to measure the role of CSS are relatively new A gap in measuring the success of CSS programs in relation to health outcomes

CSS Activities in Cambodia Health Center OD/PHD Supervisor NGOs OD Referral Hospital HCMC (Representatives of Commune Council, Health center Team, and VHSG) VHSG Community Representatives VMW MMW VHV CDOTS Watcher HBC/ Linked Response Population in Community

Evaluation Questions Program Design Program Implementation Program content aligned with national priorities/plans Inputs and processes to ensure proper steps and policies are incorporated in programs from the beginning Program Implementation Facilitating and inhibiting factors Coordination, communication, alignment Sustainability? Program Results Equity of services Synergies across HSS, HIV, TB, malaria GF grants Effect on diagnosis, referral, treatment, follow-up, linkages with other aspects of health system

Evaluation Framework Global Health Community Systems Framework Discusses how community systems contribute to improving health outcomes in conjunction with national health and social welfare systems 6 core components of community systems across 10 SDAs GF Monitoring and Evaluation Toolkit for Health and CSS programs Based on the 10 SDAs applied to the CSS context Presents indicators for health inputs, outputs (by SDA), health outcomes and impact

Linking to the GF CSS Framework Six Core Components of the framework Service Delivery Areas (SDA) Evaluation Themes 1. Enabling environment and advocacy SDA 1: Monitoring and documentation of community and government interventions SDA 2: Advocacy, communication, and social mobilization Community participation 2. Community networks, linkages, partnerships and coordination SDA 3: Building community linkages, collaboration and coordination Knowledge of community structures, roles and responsibilities, coordination 3. Resources and capacity building SDA 4: Human resources: skills building for service delivery, advocacy and leadership SDA 5: Financial resources SDA 6: Material resources—infrastructure, information, and essential commodities (including medical and other products and technologies) Human resources, capacity, sustainability 4. Community activities and service delivery SDA 7: Community-based activities and services—delivery, use, quality Access, demand and utilization of services, quality of services 5. Organizational and leadership strengthening SDA 8: Management, accountability, and leadership Adherence to work plan 6. Monitoring and evaluation and planning SDA 9: Monitoring and evaluation, evidence building SDA 10: Strategic and operational planning Standardization of guidelines and procedures, flow of information and data quality, feedback mechanisms

Overall Impact/Goal: Reduced Morbidity and Mortality CSS Inputs Enabling environment and advocacy Community networks, linkages, partnerships and coordination Resources and capacity building Community activities and service delivery Organizational and leadership strengthening Monitoring and evaluation, and planning CSS Processes Role of VHSG, HCMC, VMW, MMW, VHV, HBC, LR, C-DOTS watchers, and associated community programs Community based health education Referrals Early Diagnosis and Treatment Follow up care Monitoring health programs Conduit for community concerns Monitor demand for health services CSS Outputs No./% of villages with VHSG , No, of. VHSG members trained No. of HBC teams, No. of Health centers with HBC teams, No. of PLWHA supported by HBC teams, No. of PLWHA receiving care No. of C-DOTS watchers trained, No. of health centers supporting C-DOTS, No. of TB patients receiving DOTS from C-DOTS watchers No. of VMW, VHV trained, No. of fever cases presented/ treated by VMW CSS Outcomes Disease specific health outcomes (based on services received) TB (case notification, treatment success) Malaria (ACT) HIV (ART) MCH (ANC, SBA) Stakeholder perceptions of Access Coverage Equity Quality of services Coordination with the health system Research Monitoring and evaluation Cambodia Socio-Economic Context

Data Collection (6 ODs & MOH) Qualitative: Key Informant Interviews National level: MOH, PR, selected SRs, intnl. organizations OD level: SSR/NGO, OD/Provincial staff Focus Group Discussions at OD level Community members (including women) Community health workers – VSHG, HCMC Committee chair, VMW, C-DOTS watcher, HBC Quantitative: GF investments in health CSS outputs and health outcomes Examined Perceptions of program function and efficacy Barriers to implementation & program successes Information on health system and community system capacity Decentralization and governance at the local level Coordination across disease programs Increases in service utilization

Findings

Program Design: GF Disbursement

GF Expenditure by Round

Program Design Volunteers are the key to program success But, a top down approach Little involvement of health center staff, community members, volunteers, and PLWHA in program design Poor formal feedback mechanism Need to standardize guidelines for community participation in health Difficulty enlisting volunteers and low incentives (TB C-DOTS program: length of the treatment and the lack of incentives result in low motivation) Need for training and regular supervision

“In general, the community people never have any idea of designing or making up the project so we have our own strategy and just tell them the policy and method for them to follow.” —Provincial Malaria Supervisor “In case I don’t like any services in the health center, I just tell the village chief.” —Commmunity Health Worker

Program Implementation: Successes High community participation levels Overlapping of responsibilities Synergy between the programs - cadre of volunteers trained can increase education and access to treatment for other conditions Partnership among community entities, with collaboration between NGOs, the OD, the health department, and health centers Roles and responsibilities of various community structures clearly understood VHSG are key, mixed opinions re: Health Center Management Committee (HCMC)

“The HCMC are not really different from VHSG, they cooperate with each other to spread information as well as activities implementation. But we also have one more role; we are the coordinator between them to make a meeting at the health center. When there are some problems, we cooperate with the commune, village and health center to solve them. —Community Health Worker “The referral of patients from health center to the hospital is done on time and without discrimination between the rich and the poor.” —Community Member

Program Implementation: Concerns Reasons for inability to follow approved plans Disbursement of funds, low staff capacity, delays caused by partner organizations/in drugs and supplies, poor infrastructure, and problems with reporting due to poor data quality Hard to reach people living outside of the village, namely monks, rural community members, and migrant workers Community level data are weak Poor reporting and use for strategic decision-making Sustainability: volunteer activities would cease if incentives were removed

“…the groups who we haven’t fully reached include those living in the remote areas and mobile populations whom our programs could not meet. The groups who we should focus more on are only those who are poor and live in remote areas, the elderly people, and the pregnant women as they are the most vulnerable population.” —NGO Project Coordinator

Home Based Care: 32,000 PLWHA, 800 Health Centers Supported

HIV (Linked Response), TB (C-DOTS), Malaria (Village Malaria Workers) PLWHA support group successful Referral and testing of pregnant women through Linked Response program Linkages with TB not very clear TB: Increase in no. of health centers implementing C-DOTS: pilot in 2001/2002 to 839 health centers nationally in 2010 No clear pattern between case notification rates and no. of health centers offering C-DOTS across ODs Malaria: VMW play a key role in diagnosis and treatment Responsible for more than 42.5% of all malaria treatments in 2011

Program Results: Access, Coverage, Quality Improved service delivery Increase in no. of health centers and community-based services Volunteers referred patients to hospitals, more follow-ups HIV/AIDS diagnosis and treatment now easier - reduced time for referral and no payment required for treatment Increase in number of outpatient visits in 2011 – possibly because of role of VHSG Community perspective Increase in demand for services and caseload at health center Community participation through volunteer activities  improved quality of services Perception in the community of improved health Declines in the case fatality rate and number of malaria deaths

“I have seen that since the community health activities began, there are more people use the blood testing and treating for AIDS at health center because the health volunteers provide community people with health education every month, especially to the patients.” —Home-based Care Volunteer

Limitations Short timeframe Hard to extrapolate to other ODs Community members may not be a representative sample but are expected to provide information for program improvements Hard to determine impact and efficiency of interventions Only associations between CSS interventions and outcomes can be established