Module 6: Properties of Indicators Tools for Civil Society to Understand and Use Development Data: Improving MDG Policymaking and Monitoring

What you will be able to do by the end of this module: Understand the difference between quantitative and qualitative indicators, and how to interpret themUnderstand the difference between quantitative and qualitative indicators, and how to interpret them Understand the role of quantitative indicators in monitoring MDGsUnderstand the role of quantitative indicators in monitoring MDGs Be familiar with the properties that good indicators should haveBe familiar with the properties that good indicators should have

Qualitative vs. Quantitative Targets Qualitative ( E.g., Belarus MDG Target 3.1: Eliminate practices of gender discrimination in employment and expand economic opportunities for women )Qualitative ( E.g., Belarus MDG Target 3.1: Eliminate practices of gender discrimination in employment and expand economic opportunities for women ) - Indirectly measurable - May be statement of opinion or feeling - Often multi-dimensional Quantitative ( E.g., Moldova MDG Target 4.1: Reduce infant mortality from 18.5 per 1,000 live births in 2006 down to 16.3 in 2010 and 13.2 in 2015 )Quantitative ( E.g., Moldova MDG Target 4.1: Reduce infant mortality from 18.5 per 1,000 live births in 2006 down to 16.3 in 2010 and 13.2 in 2015 ) - Directly measurable - One-dimensional - Types: counts, continuous

Quantitative Example Source: World Development Indicators, World Bank, 2008 Belarus – 100% Moldova – 98%

Qualitative Example

Indicators Require Values Indicators can only be used for assessment if some values are attached to themIndicators can only be used for assessment if some values are attached to them - No MDG3 target values for 2015 in Belarus and Moldova Quantitative indicators have numeric valuesQuantitative indicators have numeric values Qualitative indicators require well-defined, unambiguous descriptive or ordered valuesQualitative indicators require well-defined, unambiguous descriptive or ordered values

MDG Indicators These are all quantitative indicators For the purposes of the MDGs we need to be able to directly and objectively measure movement towards the targets For the purposes of the MDGs we need to be able to directly and objectively measure movement towards the targets

Other Indicators Other indicators, both quantitative and qualitative, are used at the national level to supplement MDGsOther indicators, both quantitative and qualitative, are used at the national level to supplement MDGs Examples:Examples: - Share of foreign investments in total investments, Programme of Social and Economic Development of the Republic of Belarus GDP growth rate, Economic Growth and Poverty Reduction Strategy Paper , Moldova Qualitative indicators are useful for assessing, e.g., satisfaction with government policies, services provided by different agenciesQualitative indicators are useful for assessing, e.g., satisfaction with government policies, services provided by different agencies

Properties of Indicators Indicators should: Be sensitive: direct, unambiguous measures of changeBe sensitive: direct, unambiguous measures of change - immunization coverage is less ambiguous than household expenditure on health Help us to observe different groups and areasHelp us to observe different groups and areas Different indicators will have different rates of change and may be suitable for different time intervalsDifferent indicators will have different rates of change and may be suitable for different time intervals

Properties of Indicators (2) Indicators should: Be chosen at the appropriate level of disaggregation, depending on country circumstances and on the intervention measuredBe chosen at the appropriate level of disaggregation, depending on country circumstances and on the intervention measured Disaggregation is typically done by:Disaggregation is typically done by: - Geographic areas: urban/rural, administrative units - Demographic groups: by sex, by age, by sex/age, by ethnicity

Properties of Indicators (3) Indicators should: Be valid/specific: measure only what it is intended to measureBe valid/specific: measure only what it is intended to measure - Vehicle operating cost depends on road quality but also on many other factors, such as international petrol prices. Vehicle operating cost is therefore not a good indicator to measure progress of the roads sector system Be relevant for policy making and/or advocacyBe relevant for policy making and/or advocacy - Proportion of parliamentary seats held by women

Properties of Indicators (4) Indicators should: Be reliableBe reliable - Scientific, objective indicators are more reliable than indicators which depend on the interpretation of the user Come packaged with a stated measure of variability and description of any bias, whenever this is availableCome packaged with a stated measure of variability and description of any bias, whenever this is available

Properties of Indicators (5) Indicators should be: Operational: measurable and quantifiableOperational: measurable and quantifiable Consistent with data available and the data collection capacityConsistent with data available and the data collection capacity - It is necessary to ensure that indicators will be measurable at the times and level selected and in line with the planned calendar of data collection Better to have few, but good, well chosen and measurable indicators than many unreliable, uncoordinated indicatorsBetter to have few, but good, well chosen and measurable indicators than many unreliable, uncoordinated indicators

Summary We have looked at quantitative and qualitative indicators, and how each of these are usedWe have looked at quantitative and qualitative indicators, and how each of these are used We have discussed the suitability of quantitative indicators for monitoring MDGsWe have discussed the suitability of quantitative indicators for monitoring MDGs We have listed the desirable properties of indicatorsWe have listed the desirable properties of indicators

Practical 6 1.Review all of the MDG indicators in your country 2.What time frame is covered by the MDG indicators in your dataset? 3.What sub-populations have been included? 4.Identify two indicators for each of the following categories of indicators: Input, Output, Outcome, Impact

Practical 6 (2) 5.Are there any qualitative indicators in your MDG report or mid-term development strategy? If so list two and explain whether they are examples of Input, Output, Outcome or Impact indicators 6.Choose one indicator and discuss which actions might trigger progress in the area monitored by the indicator