1. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Advocacy, Communication and Social Mobilization at Country Level Presentation to the ACSM Subgroup at Country Level

2. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Outline 1. What are the monitoring and evaluation challenges for ACSM? 2. What does the 10-year ACSM Framework prescribe? 3. What 5-7 indicators could be used in a country’s ACSM performance profile?

3. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level 2. What has the ACSM Sub Group at country level achieved?

4. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level …but M&E not yet standardized 2002 data as gathered from ACSM Needs Assessment Survey of HBCs 2006-2007 data gathered from HBC DOTS Expansion survey (self reporting) *Approved grants from 5 th round GFATM HBCs with GFATM funding via rounds 1-5 for ACSM activities Rapid Growth in Plans for ACSM utilization Momentum Building… $63 million over 5 years

5. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level 3. What are the monitoring and evaluation challenges?

6. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Currently, we have:  Minimal data  Old data  Unsystematic data  No centralized collection point

7. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Colombia

8. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level The Philippines' story Barangay Pangobilian, Brooke’s Point, Palawan Case Finding Activities

9. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level ACSM impact can be shown but the measurement can be disputed without standard measurement Consultant report on Kerala November 2005 There were increases in all but one of the 14 districts in the State with half the districts (7 of the 14) registering increases of 13% - 41%. The comparison between 4th Qrt, 2003 and 1st Qrt, 2004 showed an increase of 19%. The comparison between 2nd Qrt, 2003 and 2nd Qrt, 2004 showed an increase of 7%. RNTCP collected data

10. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level ACSM impact can be measured in carefully designed evaluation studies However we have challenges:  Validity, reliability, and repeatability  Contribution analysis is not straightforward  Contextualization  Lack of uniformity of indicators across countries  Evaluation of ACSM studies not part of routine TB M&E  Need for participatory monitoring and evaluation Lessons

11. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level These challenges can be overcome, BUT only with some retooling and investment in M&E

12. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level 4. What does the 10-year ACSM Framework prescribe?

13. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Five streams of evidence (at least) Country-level ACSM develop their own M&E mechanisms (using Guideline as a starting point) Regular ACSM evaluations commissioned by the Stop TB Partnership (akin to ACSM country case studies) 5-7 ACSM standard indicators for country-by-country reporting Regular reports from ACSM agencies partnered to NTPs through Technical Service Contracts Regular NTP reviews to incorporate ACSM measures

14. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Work in Progress Monitoring and Evaluating ACSM in National TB Programmes: A Guideline on Indicators ACSM capacity ACSM activity Sputum testing Treatment adherence Stigma and discrimination Monitoring and evaluating advocacy Measuring significant changes Measuring social change communication TB social change indicators

15. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level 5. What 5-7 indicators could be used in a country’s annual ACSM performance profile?

16. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Preface Suggested indicators are illustrative Any standard set of indictors will need country and partner consultation Based on more than 50 years of health communication experience and study

17. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Case detection InputsOutputs InputsCase cureOutputs Outcomes Impact Research, plans, resources, supplies, staff, etc. Research, plans, resources, supplies, staff, etc. Social mobilization and communication activities, knowledge, policies, laws, incentives Social mobilization and communication activities, knowledge, policies, laws, incentives Sputum-testing, reduced stigma, reduced discrimination, other significant social changes Treatment adherence, reduced stigma, reduced discrimination, other significant social changes Increase case detection rates Increase cure rates, decrease TB incidence, decrease TB mortality, reduce risk of MDR- TB, improve quality of life ACSM Indicator Flow

18. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level ACSM 10-year Framework Action Areas (Illustrative) Increasing capacity and technical assistance for ACSM Improving case detection and treatment adherence Combating stigma and discrimination Empowering people and communities affected by TB Mobilizing political commitment and resources for TB control

19. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level ACSM Capacity Country has an evidence-based, sufficiently resourced ACSM Strategic Plan Composite measures: % districts/provinces with TB ACSM plans based on recent formative research % national health budget devoted to TB ACSM (5- 15%) % districts/provinces with designated ACSM staff with appropriate qualifications Parks W. with T Pennas and J Deane. 2006. Monitoring and evaluating advocacy, communication and social mobilization in national TB control programs.: a guideline on indicators. Geneva, World Health Organization.

20. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Case detection and case cure % of population who know that going to a DOTS clinic is the best way to diagnose TB % of districts that have active pro-poor incentive schemes for TB patients on DOTS ACSM contribution to CDR and CCR could be produced through evaluation of “sentinel sites” (e.g. districts – pre/post, time series, intervention/control) Parks W. with T Pennas and J Deane. 2006. Monitoring and evaluating advocacy, communication and social mobilization in national TB control programs.: a guideline on indicators. Geneva, World Health Organization.

21. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Combating stigma and discrimination % of population expressing accepting attitudes towards TB suspects, patients and survivors Most Significant Change Stories (qualitative) Davies, R & J Dart. 2005. The most significant change 'MSC' technique: a guide to its use. Version 1.00. http://www.mande.co.uk/docs/MSCGuide.pdf International Centre for Research on Women. 2002. Understanding HIV-related stigma and resulting discrimination in sub-Saharan Africa.

22. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Patient empowerment # of TB patients or survivors trained to become educators, monitors or advocates Most Significant Change Stories (qualitative) The Global Fund to Fight AIDS, TB and Malaria. Monitoring and Evaluation Toolkit: HIV/AIDS, Tuberculosis and Malaria, Second Edition. 2006. www.theglobalfund.org. Davies, R & J Dart. 2005. The most significant change 'MSC' technique: a guide to its use. Version 1.00. http://www.mande.co.uk/docs/MSCGuide.pdf

23. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Political commitment Multi-sectorial involvement exists at the national level for TB control % national budget dedicated to TB control Singhal A and E Rogers. 2003. Combating AIDS: communication strategies in action. New Delhi, Sage Publications: 351-353.

24. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level 5. What is the Subgroup's counsel?

25. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level  To what extent is it desirable to develop a standard set of ACSM indicators that are applicable across all programs and contexts to inform on the quality and progress of ACSM interventions?  How can M&E of ACSM be better integrated into current reporting structures?  How should WHO/ACSM Sub-Group advance this M&E work?  What is the best way forward?

26. 2nd Meeting of the Subgroup on Advocacy, Communications and Social Mobilization at Country Level Recommendations Should there be standard indicators? Need to communicate in better ways Standardized surveys Link to other surveys (DHS and others) Diagnostic delay needs to be included Need simplicity Everyone is not starting at the same place. Population based may not be the best way to go. Country based M&E needs to be enhanced Options:  Throw away the guide  Circulate and comment  Revise the guide and circulate. Countries NEED some guidelines to base M&E plan. Reporting at global level and local level.  Develop a hierarchy for results or reporting  Can we look similar indicators across countries and measure those?  Look at the workplans to revise the guide to find common indicators  To look at the beneficiary of the information.  Develop circulation