1. Soumya Alva, Sujata Ram, Anne Langston ICF International, May 15, 2012
Community Systems Strengthening Programs in Cambodia: Findings from a Recent Evaluation for the Global Fund
Soumya Alva, Sujata Ram, Anne Langston
ICF International, May 15, 2012

3. Motivation for the Evaluation
The role of community health workers and CSOs is key
Frameworks discuss ways in which community systems contribute to improving health outcomes
Indicators to measure the role of CSS are relatively new
A gap in measuring the success of CSS programs in relation to health outcomes

4. CSS Activities in Cambodia
Health Center
OD/PHD Supervisor
NGOs
OD Referral Hospital
HCMC (Representatives of Commune Council, Health center Team, and VHSG)
VHSG
Community Representatives
VMW
MMW
VHV
CDOTS Watcher
HBC/ Linked Response
Population in Community

5. Evaluation Questions Program Design Program Implementation
Program content aligned with national priorities/plans
Inputs and processes to ensure proper steps and policies are incorporated in programs from the beginning
Program Implementation
Facilitating and inhibiting factors
Coordination, communication, alignment
Sustainability?
Program Results
Equity of services
Synergies across HSS, HIV, TB, malaria GF grants
Effect on diagnosis, referral, treatment, follow-up, linkages with other aspects of health system

6. Evaluation Framework Global Health Community Systems Framework
Discusses how community systems contribute to improving health outcomes in conjunction with national health and social welfare systems
6 core components of community systems across 10 SDAs
GF Monitoring and Evaluation Toolkit for Health and CSS programs
Based on the 10 SDAs applied to the CSS context
Presents indicators for health inputs, outputs (by SDA), health outcomes and impact

7. Linking to the GF CSS Framework
Six Core Components of the framework
Service Delivery Areas (SDA)
Evaluation Themes
1. Enabling environment and advocacy
SDA 1: Monitoring and documentation of community and government interventions
SDA 2: Advocacy, communication, and social mobilization
Community participation
2. Community networks, linkages, partnerships and coordination
SDA 3: Building community linkages, collaboration and coordination
Knowledge of community structures, roles and responsibilities, coordination
3. Resources and capacity building
SDA 4: Human resources: skills building for service delivery, advocacy and leadership
SDA 5: Financial resources
SDA 6: Material resources—infrastructure, information, and essential commodities (including medical and other products and technologies)
Human resources, capacity, sustainability
4. Community activities and service delivery
SDA 7: Community-based activities and services—delivery, use, quality
Access, demand and utilization of services, quality of services
5. Organizational and leadership strengthening
SDA 8: Management, accountability, and leadership
Adherence to work plan
6. Monitoring and evaluation and planning
SDA 9: Monitoring and evaluation, evidence building
SDA 10: Strategic and operational planning
Standardization of guidelines and procedures, flow of information and data quality, feedback mechanisms

8. Overall Impact/Goal: Reduced Morbidity and Mortality
CSS Inputs
Enabling environment and advocacy
Community networks, linkages, partnerships and coordination
Resources and capacity building
Community activities and service delivery
Organizational and leadership strengthening
Monitoring and evaluation, and planning
CSS Processes
Role of VHSG, HCMC, VMW, MMW, VHV, HBC, LR, C-DOTS watchers, and associated community programs
Community based health education
Referrals
Early Diagnosis and Treatment
Follow up care
Monitoring health programs
Conduit for community concerns
Monitor demand for health services
CSS Outputs
No./% of villages with VHSG , No, of. VHSG members trained
No. of HBC teams, No. of Health centers with HBC teams, No. of PLWHA supported by HBC teams, No. of PLWHA receiving care
No. of C-DOTS watchers trained, No. of health centers supporting C-DOTS, No. of TB patients receiving DOTS from C-DOTS watchers
No. of VMW, VHV trained, No. of fever cases presented/ treated by VMW
CSS Outcomes
Disease specific health outcomes (based on services received)
TB (case notification, treatment success)
Malaria (ACT)
HIV (ART)
MCH (ANC, SBA)
Stakeholder perceptions of
Access
Coverage
Equity
Quality of services
Coordination with the health system
Research
Monitoring and evaluation
Cambodia Socio-Economic Context

9. Data Collection (6 ODs & MOH)
Qualitative:
Key Informant Interviews
National level: MOH, PR, selected SRs, intnl. organizations
OD level: SSR/NGO, OD/Provincial staff
Focus Group Discussions at OD level
Community members (including women)
Community health workers – VSHG, HCMC Committee chair, VMW, C-DOTS watcher, HBC
Quantitative:
GF investments in health
CSS outputs and health outcomes
Examined
Perceptions of program function and efficacy
Barriers to implementation & program successes
Information on health system and community system capacity
Decentralization and governance at the local level
Coordination across disease programs
Increases in service utilization

10. Findings

11. Program Design: GF Disbursement

12. GF Expenditure by Round

13. Program Design Volunteers are the key to program success
But, a top down approach
Little involvement of health center staff, community members, volunteers, and PLWHA in program design
Poor formal feedback mechanism
Need to standardize guidelines for community participation in health
Difficulty enlisting volunteers and low incentives (TB C-DOTS program: length of the treatment and the lack of incentives result in low motivation)
Need for training and regular supervision

14. “In general, the community people never have any idea of designing or making up the project so we have our own strategy and just tell them the policy and method for them to follow.”
—Provincial Malaria Supervisor
“In case I don’t like any services in the health center, I just tell the village chief.”
—Commmunity Health Worker

15. Program Implementation: Successes
High community participation levels
Overlapping of responsibilities
Synergy between the programs - cadre of volunteers trained can increase education and access to treatment for other conditions
Partnership among community entities, with collaboration between NGOs, the OD, the health department, and health centers
Roles and responsibilities of various community structures clearly understood
VHSG are key, mixed opinions re: Health Center Management Committee (HCMC)

16. “The HCMC are not really different from VHSG, they cooperate with each other to spread information as well as activities implementation. But we also have one more role; we are the coordinator between them to make a meeting at the health center. When there are some problems, we cooperate with the commune, village and health center to solve them.
—Community Health Worker
“The referral of patients from health center to the hospital is done on time and without discrimination between the rich and the poor.”
—Community Member

17. Program Implementation: Concerns
Reasons for inability to follow approved plans
Disbursement of funds, low staff capacity, delays caused by partner organizations/in drugs and supplies, poor infrastructure, and problems with reporting due to poor data quality
Hard to reach people living outside of the village, namely monks, rural community members, and migrant workers
Community level data are weak
Poor reporting and use for strategic decision-making
Sustainability: volunteer activities would cease if incentives were removed

18. “…the groups who we haven’t fully reached include those living in the remote areas and mobile populations whom our programs could not meet. The groups who we should focus more on are only those who are poor and live in remote areas, the elderly people, and the pregnant women as they are the most vulnerable population.”
—NGO Project Coordinator

19. Home Based Care: 32,000 PLWHA, 800 Health Centers Supported

20. HIV (Linked Response), TB (C-DOTS), Malaria (Village Malaria Workers)
PLWHA support group successful
Referral and testing of pregnant women through Linked Response program
Linkages with TB not very clear
TB:
Increase in no. of health centers implementing C-DOTS: pilot in 2001/2002 to 839 health centers nationally in 2010
No clear pattern between case notification rates and no. of health centers offering C-DOTS across ODs
Malaria:
VMW play a key role in diagnosis and treatment
Responsible for more than 42.5% of all malaria treatments in 2011

21. Program Results: Access, Coverage, Quality
Improved service delivery
Increase in no. of health centers and community-based services
Volunteers referred patients to hospitals, more follow-ups
HIV/AIDS diagnosis and treatment now easier - reduced time for referral and no payment required for treatment
Increase in number of outpatient visits in 2011 – possibly because of role of VHSG
Community perspective
Increase in demand for services and caseload at health center
Community participation through volunteer activities  improved quality of services
Perception in the community of improved health
Declines in the case fatality rate and number of malaria deaths

22. “I have seen that since the community health activities began, there are more people use the blood testing and treating for AIDS at health center because the health volunteers provide community people with health education every month, especially to the patients.”
—Home-based Care Volunteer

23. Limitations Short timeframe Hard to extrapolate to other ODs
Community members may not be a representative sample but are expected to provide information for program improvements
Hard to determine impact and efficiency of interventions
Only associations between CSS interventions and outcomes can be established