1. 2015 1990 1 Module 9: Applications of Indicators Tools for Civil Society to Understand and Use Development Data: Improving MDG Policymaking and Monitoring

2. 2015 1990 2 What you will be able to do by the end of this module: Understand the place that MDG and other indicators have in developing and monitoring policy, and in advocacyUnderstand the place that MDG and other indicators have in developing and monitoring policy, and in advocacy Know how indicators can be used for these purposesKnow how indicators can be used for these purposes Understand how proxy indicators can be used when direct indicators are not availableUnderstand how proxy indicators can be used when direct indicators are not available Take their strengths and weaknesses into account when using indicatorsTake their strengths and weaknesses into account when using indicators Use indicators which are consistent with one anotherUse indicators which are consistent with one another Understand the properties of the different levels of indicators, and when to use the correct levelUnderstand the properties of the different levels of indicators, and when to use the correct level

3. 2015 1990 3 Why Use MDG Indicators? To set targets using empirical dataTo set targets using empirical data To ascertain and inform the public and policy makers on progress on MDGsTo ascertain and inform the public and policy makers on progress on MDGs To trigger action to meet time-bound targetsTo trigger action to meet time-bound targets To contribute to the design of policies based on evidenceTo contribute to the design of policies based on evidence To create public interest in MDG progress at home and abroadTo create public interest in MDG progress at home and abroad

4. 2015 1990 4 Use of Indicators Where are we today?Where are we today? How far away from the target are we?How far away from the target are we? What progress is being made?What progress is being made?

5. 2015 1990 5 Where are We Today? The value of an indicator tells us what the current level is in a given moment and place Example: MDG6, Indicator “Death rate associated with tuberculosis (per 100,000 people)” In Belarus in 2003 the rate was 10.4 In Moldova in 2003 the rate was 16.4

6. 2015 1990 6 How can this snapshot be used? Nationally: - Gives us a picture of the current status (at different levels of disaggregation if available) - Tells us the magnitude of the challenge of reaching targets - Provides hard evidence to motivate policy makers - Helps to quantify the intervention needed

7. 2015 1990 7 How can this snapshot be used? (2) Regionally - Shows us how we compare with similar countries - Feeds into regional policies and organizations Globally Globally - Information for international organizations and donors - Helps to build a global picture of current status

8. 2015 1990 8 How can this snapshot be used? (3) Source: Health for All Database, WHO Regional Office for Europe, 2006

9. 2015 1990 9 Position relative to the target MDG targets specify values or levels to be achieved by certain dates Examples. Trend Trend Target Target Sources: National MDG Reports of the Republic of Belarus and the Republic of Moldova

10. 2015 1990 10 What extra information does this give us? Shows whether change is happening, and in which directionShows whether change is happening, and in which direction Tells us how big the change has beenTells us how big the change has been Allows us to decide whether the intervention is having the predicted effectAllows us to decide whether the intervention is having the predicted effect - Direction - Direction - Magnitude - Magnitude

11. 2015 1990 11 What extra information does this give us? (2) Allows us to estimate trendAllows us to estimate trend Need to be very sure of underlying trendNeed to be very sure of underlying trend - Form, e.g., linear, exponential - Form, e.g., linear, exponential - Effects of natural limits, e.g., available resources - Effects of natural limits, e.g., available resources - Unforeseen outside forces - Unforeseen outside forcesWarning Prediction gets less reliable, the further you extrapolate

12. 2015 1990 12 Proxy Indicators Not always possible to measure the indicator we’re interested inNot always possible to measure the indicator we’re interested in Need to measure something closely related, that isNeed to measure something closely related, that is - Accessible - Accessible - Consistent with the measure we can’t get - Consistent with the measure we can’t get - Useful - Useful

13. 2015 1990 13 Proxy Indicators. Examples Environmental indicators are often proxies - Use of ozone depleting substances as proxy to ozone depletion - Use of ozone depleting substances as proxy to ozone depletion Literacy of men and women age 15-24 Literacy of men and women age 15-24 - Difficult to measure - Difficult to measure - In past educational attainment has been used as a proxy measure - In past educational attainment has been used as a proxy measure - These days, reading and writing tests are recommended instead of educational attainment - These days, reading and writing tests are recommended instead of educational attainment

14. 2015 1990 14 Proxy Indicators. Examples (2) Access to communication systems. Two proxy indicators used:Access to communication systems. Two proxy indicators used: - Telephone lines per 1,000 people - Telephone lines per 1,000 people - Personal computers per 1,000 people - Personal computers per 1,000 people They measure availability and not access or use

15. 2015 1990 15 Consistency Definitions should maintain consistency amongst related indicators, that is, use commonDefinitions should maintain consistency amongst related indicators, that is, use common - Definitions (e.g., of primary education) - Definitions (e.g., of primary education) - Denominators (e.g., defined populations) - Denominators (e.g., defined populations) - Data (data from the same surveys) - Data (data from the same surveys) - Time (collected at the same or very similar time points) - Time (collected at the same or very similar time points) - Analysis methods (where this is necessary to maintain common definitions) - Analysis methods (where this is necessary to maintain common definitions) Can provide checks on values and trends Can provide checks on values and trends

16. 2015 1990 16 Conditions for correct and effective use Selection of appropriate indicatorsSelection of appropriate indicators - Measures the correct thing - Measures the correct thing - Fit for the purpose (policy development or assessment, advocacy, target audience) - Fit for the purpose (policy development or assessment, advocacy, target audience) - Consistent with related indicators - Consistent with related indicators Available data, and information about the data (metadata)Available data, and information about the data (metadata) Correct and accurate analysis methodsCorrect and accurate analysis methods Some measure of reliability/variability/sample sizeSome measure of reliability/variability/sample size Specification of bias, if anySpecification of bias, if any TimelinessTimeliness Effective presentationEffective presentation

17. 2015 1990 17 Abuse of Indicators Wrong indicator - Telephone lines?? - Telephone lines?? Wrong interpretation Wrong interpretation Comparing unlike scales Comparing unlike scales - E.g., comparing poverty rates of two countries which use different welfare aggregates and poverty lines - E.g., comparing poverty rates of two countries which use different welfare aggregates and poverty lines Errors in data or analysis methods Errors in data or analysis methods - Rely on skills of data analysis teams - Rely on skills of data analysis teams Using out of date values Using out of date values Inappropriate extrapolation Inappropriate extrapolation Ignoring variability Ignoring variability

18. 2015 1990 18 Indicator Levels GlobalGlobal RegionalRegional NationalNational Sub-nationalSub-national

19. 2015 1990 19 Global Comparisons Varying sources for global MDG indicatorsVarying sources for global MDG indicators Some taken from national MDG reportsSome taken from national MDG reports Some produced by national agenciesSome produced by national agencies Some produced by international agencies, e.g., poverty measures produced by World BankSome produced by international agencies, e.g., poverty measures produced by World Bank

20. 2015 1990 20 Global MDGs vs. National MDGs National MDGs - Use only national data - Use only national data - Allow sub-national disaggregation - Allow sub-national disaggregation Global MDGs Global MDGs - Often adjusted to make comparisons possible - Often adjusted to make comparisons possible - May use different analysis methods - May use different analysis methods - May use different data (particularly UN population estimates) - May use different data (particularly UN population estimates)

21. 2015 1990 21 Policy and Advocacy Only national indicator values are useful for developing and monitoring national policyOnly national indicator values are useful for developing and monitoring national policy Global and regional indicator values don’t give a direct measure of how well the national policy is doingGlobal and regional indicator values don’t give a direct measure of how well the national policy is doing For advocacy, we need direct local evidence to support calls for intervention and action, so need local valuesFor advocacy, we need direct local evidence to support calls for intervention and action, so need local values

22. 2015 1990 22 Global and National MDG Indicators Both are valuableBoth are valuable GlobalGlobal - Position relative to region and world - Position relative to region and world - Internationally agreed basis for interventions - Internationally agreed basis for interventions - May be used in place of national estimates, or where none is available - May be used in place of national estimates, or where none is available - May not be directly applicable to national policies - May not be directly applicable to national policies

23. 2015 1990 23 Global and National MDG Indicators (2) NationalNational - Relate to national and sub-national issues - Relate to national and sub-national issues - Values used directly as input to national/regional policy development - Values used directly as input to national/regional policy development - Show national changes over time - Show national changes over time - Extra indicators can be used - Extra indicators can be used Though different, both can be right depending on contextThough different, both can be right depending on context

24. 2015 1990 24 Global and National MDG Indicators (3) Example. School enrollment Example. School enrollment Country, year Indicator National MDG Report World Development Indicators, World Bank 2008 Belarus, 2004 Net general secondary enrollment ratio 86.689.0 Moldova, 2006 Gross general secondary enrollment ratio 92.482.3

25. 2015 1990 25 Sub-National Indicators Based on a common national definitionBased on a common national definition Data from national datasets, which include sub- population identifiersData from national datasets, which include sub- population identifiers Allows for disaggregation of indicatorsAllows for disaggregation of indicators

26. 2015 1990 26 Use in Practice National examplesNational examples - Developing and monitoring development strategies - Developing and monitoring development strategies - Assessing the effectiveness of policies - Assessing the effectiveness of policies - Monitoring impact of economic and social change - Monitoring impact of economic and social change

27. 2015 1990 27 Use in Practice (2) Sub-national examples - Meeting regional and district needs for planning and service delivery - Meeting regional and district needs for planning and service delivery - Addressing the problems associated with pockets - Addressing the problems associated with pockets - Advocating for change - Advocating for change - Targeting interventions - Targeting interventions

28. 2015 1990 28 Strengths and Limitations Strengths of indicatorsStrengths of indicators - Simple - Simple - Well-defined - Well-defined - Comparability - Comparability - Roles in policy development/advocacy are clear - Roles in policy development/advocacy are clear LimitationsLimitations - Availability of data - Availability of data - Timeliness of data - Timeliness of data - Comparisons: global/national - Comparisons: global/national - Changes in external environment - Changes in external environment

29. 2015 1990 29 Summary Purpose and uses of indicatorsPurpose and uses of indicators Proxy indicatorsProxy indicators ConsistencyConsistency LevelsLevels Strengths and limitationsStrengths and limitations

30. 2015 1990 30 Practical 9 Year Ratio of girls to boys in secondary school Ratio of boys 10-14 to girls 10-14 19900.670.99 19950.720.98 20000.800.95 1. How would you interpret the following information? 2. Is there any further information that would help you interpret the indicators in question 1?

31. 2015 1990 31 Practical 9 3. Look at all of the MDG indicators included in your National MDG Report. Choose one indicator for which data are present in your report, and describe briefly how it could be used either to support policy development or to advocate for policy change. What are the strengths and limitations of this particular indicator for this purpose? How could it subsequently be used to monitor change over time?