1. Monitoring and Evaluation: Maternal and Child Nutrition

2. Session Objectives
By the end of this session participants will be able to:
Apply basic M&E concepts to maternal and child nutrition interventions
Design and use M&E frameworks for nutrition programs
Identify nutrition interventions and common indicators for assessing their results
Describe M&E challenges of nutrition programs
This session focuses on monitoring and evaluating nutrition programs. By the end of this session, participants will be able to apply basic M&E concepts to maternal and child nutrition intervention programs; design and use monitoring and evaluation frameworks for nutrition programs; identify nutrition interventions and common indicators for assessing their results, as well as their data sources; and finally, describe measurement challenges in monitoring and evaluating nutrition programs.

3. Session Overview The problem of malnutrition
Interventions and strategies
M&E frameworks for nutrition programs
Common indicators & data sources
M&E challenges
We begin with a discussion of the problem of malnutrition and how malnutrition fits into global health and development goals. Then we proceed with a discussion of the key interventions that have been shown to reduce malnutrition. Next we present some examples of M&E frameworks for nutrition interventions before discussing the common indicators for assessing the outcomes of nutrition programs and their strengths and limitations. As with other modules, we go on to describe the main data sources for nutrition indicators and key M&E challenges for nutrition programs.

4. The Problem
Malnutrition contributes to over half of all child deaths, (60%)
Malnutrition is largely hidden, (mild, moderate, ?)
We will start with defining the nature of the problem of malnutrition in order to get a good understanding of the key interventions. Malnutrition contributes to at least half of child deaths worldwide. Malnutrition is sometimes called “the silent emergency” as it is not clearly visible and various tests may need to be administered to measure levels of malnutrition in a population. Most of the child deaths associated with malnutrition occur at the “mild” and “moderate” stages of malnutrition.

5. Importance of malnutrition as an underlying factor in under-five mortality in Ethiopian Children.
Others
Diarrheal Diseases
Malnutrition
58%
Perinatal Complications
Acute
Respiratory
Infections
Malaria
Measles
WHO ’98

6. Micronutrient deficiencies
Micronutrient deficiencies have severe consequences;
Iodine deficiency damages intellectual development,
50% of pregnant women and 40-50% of children < 5 in developing countries are iron deficient,
VAD affects > 100 million children, and is responsible for as many as one out of every four child deaths in places with Vitamin A deficiency,
Micronutrient deficiency, especially iodine, vitamin A and iron, represents a major threat to the health and development of populations worldwide, particularly to preschool children and pregnant women in low-income countries. For example, iodine deficiency is the primary cause of preventable mental retardation and brain damage, having the most devastating impact on the brain of the developing fetus and young children in the first few years of life. Iodine deficiency also increases the chance of infant mortality, miscarriage and stillbirth. Iron deficiency is the most pervasive nutritional problem in the world. Between 4 and 5 billion people suffer from iron deficiency and an estimated 2 billion are anemic.  Women and young children are most vulnerable: 50 per cent of pregnant women and 40 to 50 per cent of children under five in developing countries are iron deficient.  Vitamin A is an essential micronutrient for the immune system. At least 100 million children under five suffer from vitamin A deficiency (VAD). VAD greatly increases the risk that a child may die from diseases such as measles, diarrhea and acute respiratory infections. Severe stages of VAD can cause blindness.

7. How maternal and child nutrition are linked
The nutrition of mothers and children is closely linked. Malnutrition often begins at conception. When pregnant women consume inadequate diets, have excessive workloads or are frequently ill, they give birth to smaller babies with a variety of health problems. Children born to malnourished mothers are more likely to be low birth weight and to die as infants. If they survive, by the second year of life, they may have permanent damage. It is important, therefore, to prevent stunting in female children during the first two years to help break the cycle of malnutrition. Mother’s nutrition before and during pregnancy influences growth and development of the fetus and its birth weight; it also affects a woman’s chances of surviving the delivery. In this regard, micronutrient status, particularly iron, of the mother has a major contribution. Consequently, both maternal and child health strategies need to contain nutrition interventions in order for malnutrition levels to improve adequately.

8. Causes of Malnutrition: conceptual framework
There are many causes of malnutrition. These can be divided into immediate, underlying and basic. This chart shows how the underlying and immediate causes interact and has been used as a guide to plan effective actions to address malnutrition at the community, district and national levels. The necessary conditions for nutritional well being are access to food, adequate care of children and women, and access to basic health services, together with a healthy environment. The potential for fulfilling these three necessary conditions (food, care, and health) for nutritional security is determined by the availability and control of resources (human, economic, and organizational). The availability and control of resources are influenced, in turn, by political and ideological/cultural factors. Nutrition interventions and their measurement tend to focus on immediate and underlying causes of malnutrition, in addition to measuring nutritional status and outcomes.
Education is cross cutting the underlying causes of malnutrition.

9. Nutrition and Development
Nutritional status is a key indicator of progress in attaining MDGs;
Eradicate extreme poverty and hunger (Goal 1),
Achieve universal primary education (Goal 2),
Promote gender equality and empower women (Goal 3),
Reduce child mortality (Goal 4),
Improve maternal health (Goal 5),
Combat HIV/AIDS, malaria and other diseases (Goal 6),
Ensure environmental sustainability (Goal 7),
Develop a global partnership for development (Goal 8)

10. Nutrition is Critical in Achieving MDGs
#1. Poverty alleviation - an indicator is % children underweight
#2. Primary education - benefits can accrue when nutrition and cognition are adequate
#3. Gender equality- better nourished girls likely to stay in school longer
#4. Child mortality - 60% associated with malnutrition
#5. Maternal health - anemia, iodine deficiency, low BMI associated with health indicators
#6. Infectious diseases and HIV AIDS- malnutrition worsens and makes them more susceptible to adverse outcomes
Before we go on to talk about nutrition interventions, let us discuss how nutrition fits into the Millennium Development Goals that were unanimously adopted in 2000 by United Nations member states to dramatically reduce poverty and its many facets by The first such goal is to halve the proportion of those suffering from hunger and whose income is less than $1 a day by One of the indicators used to measure progress toward this goal is the percent of children who are underweight. Without addressing nutrition and food security, other goals—including universal primary education, promoting gender equality, improving maternal health, reducing child mortality, and combating HIV/AIDS, malaria and other diseases--would be compromised. For example, malnourished children have diminished learning capacity, which directly impacts on the goals of primary education. Better nourished girls are likely to stay in school longer, which would contribute to the promotion of gender inequality. Finally good nutrition helps protect natural immunity, which is important for health as resistance to drugs increases and new diseases emerge.

11. World Fit for Children Goals
Reduction of child malnutrition among children under five years of age by at least one third, with special attention to children under two years of age.
Achieve the sustainable elimination of iodine deficiency disorders by 2005
Achieve the sustainable elimination of vitamin A deficiency by 2010
Reduce the prevalence of anemia (including iron deficiency) by one third by 2010
More specific child nutrition goals were adopted at the Special Session on Children of the United Nations General Assembly in May One such goal was a reduction of child malnutrition among children under five years of age by at least one third, with special attention to children under two years of age. Micronutrient-related goals set out in the 1990 World Summit for Children were also renewed. Specifically, three micronutrients—vitamin A, iodine and iron—were singled out for attention. The renewed micronutrient goals set out in May 2002 aim to achieve the sustainable elimination of iodine-deficiency disorders by 2005 and vitamin A deficiency by 2010; to reduce by one third the prevalence of anemia, including iron deficiency, by An additional goal is to accelerate progress toward the reduction of other micronutrient deficiencies through dietary diversification, food fortification, and supplementation.

12. Interventions and Strategies

13. Interventions Proven to Reduce Malnutrition When Linked with Health Services (Essential Nutrition Actions)
Vitamin A and iron
Iodized salt
Breastfeeding
Mother’s nutrition
Complementary
feeding
Sick/severe
cases
Nutrition programs are broadly similar across countries. They generally involve one or a mix of the above interventions that have been proven to work at scale in diverse settings: (1) breastfeeding promotion; (2) improved complementary feeding practices (3) maternal nutrition; (4) vitamin A and iron/folic acid supplementation for women and children; (5) appropriate care and feeding of sick and malnourished children; (6) adequate intake of iodine by all members of the household. All these interventions need to be linked with basic health care and where food security is limited, household food security as well. These interventions are rarely implemented as single interventions. When combined, they are sometimes called the “minimum package for nutrition”. These interventions are often a broader part of reproductive maternal and child and infectious disease programs, or may be implemented even in the broader context of food security and poverty alleviation program. The multisectoral and interdisciplinary nature of nutrition programs has implications for monitoring and evaluation, particularly for demonstrating plausibly whether observed effects are due to a particular intervention.

14. Monitoring and Evaluation Frameworks
for
Nutrition Programs

15. Results Framework
SO: Vulnerable families achieve sustainable improvement in the nutrition and health status of seven million women and children by 2008
IR1 Service providers improve quality & coverage of maternal and child health & nutrition services & key systems
IR2 Communities sustain activities for improved maternal and child survival and nutrition
Speaker Notes
This slide presents an example of a results framework for a nutrition program.
Class activity:
Can you give us examples of appropriate nutrition-related indicators for each objective (SO), for each intermediate result (IR) or outcome, and for each sub IR or outcome? You may refer to the list of indicators on the handout provided.
EXAMPLES:
SO= weight-for-age (underweight) for children under age three year; weight-for-height (stunting) of children under age five years, and body mass index (BMI) of women yrs
IR1=coverage of nutrition actions in ANC for pregnant women and nutrition actions in immunization programs for children months (see supplemental slide);
- % of children aged 6-59 months of age who received a high dose of vitamin A in the last 6 months in immunization services
- % of women who receive who receive two high-dose supplements of vitamin A within six weeks of giving birth
- % of pregnant women who received the recommended number of iron folate supplements during pregnancy
IR2: Indicators measuring whether community health workers demonstrate knowledge and skills of nutrition actions (e.g. age and dose of vitamin A, iron folic acid, breastfeeding assessment and counseling) and make at least monthly contact with families of pregnant women and children under two years.
IR2.3: Indicators measuring whether a nurse from nearby health facility trains and supervises community workers, provides them supplies of iron folic acid, and assists families having difficulties with infant feeding or growth faltering; doctor in charge routinely checks monitoring data on vitamin A and iron folic acid coverage, and growth monitoring.
IR2.3 Stronger links between health systems and communities
IR2.2 Increase ownership and participation of community leaders and groups in monitoring health and nutrition services and behaviors
IR2.1 Increase awareness of households & other key audiences about desirable nutrition and health behaviors through multiple channels, e.g. ‘change agents’
IR1.1 Coordinate/converge services provided by the Dept. of social services (ICDS) and MOH, e.g. through Nutrition and Health Days, and Block planning
IR1.2 Build capacity of service providers, supervisors and managers in the dept. of social services (ICDS) and MOH
Source: Adapted from CARE/India INHP II,
DAP II

16. Logical Framework ASSUMPTIONS
- Stable political situation, sustained political commitment and financing
- Sufficient numbers of competent health care personnel and supplies in the government sector
- No natural disaster or disease epidemic
MEANS OF
VERIFICATION
1.Annual reports from MCH services, special surveys
2.Annual reports, special surveys
3.National / local tracking reports (surveillance) of high risk areas/ populations
PERFORMANCE
INDICATORS  
1.Proportion of children 6-35 months who are malnourished
2. Coverage of essential nutrition actions: exclusive BF, appropriate CF, vitamin A, iron supplements /fortified foods, iodized salt use, coverage of sick and malnourished in special programs
3. Proportion of households at risk of or vulnerable to food insecurity
PURPOSE
Sustainable improvement in the nutrition and health status of women and children through improved services provision and community participation
This is a logical framework for a program, the purpose of which is to improve the nutrition and health status of women and children through improved services and community involvement. The program planners identified three indicators and their data sources that correspond with this purpose.
Class activity:
Consider the performance indicators proposed for this nutrition program, and comment on their availability and reliability:
#1=young child malnutrition; should you use underweight or stunting? What cut-off points should be used?
#2=coverage of essential nutrition actions: If there is no separate or special nutrition program monitoring system in place, where could you get some of the data? (Answer: from an MCH program such as ANC registers or immunization tally sheets and reports, from a food aid program, etc.)
#3=The third indicator is the proportion of households that are at risk of or vulnerable to food insecurity. How could you distinguish between those chronically at risk and those occasionally or seasonally at risk?
NOTE: A logic model would allow a program to select indicators that monitor all stages (inputs, process, outputs)
of their activities e.g. funds and staff available (inputs), training sessions completed (process), number of skilled
workers or villages with trained volunteers (outputs).

17. Common Indicators and Data Sources

18. Categories of Nutrition Indicators
Nutritional status
Breastfeeding practices
Complementary feeding practices
Micronutrient supplements/fortified foods
Household food security; vulnerability to food and nutrition insecurity
Here you see five categories of nutrition outcome indicators. One group is certainly not enough, for example, nutritional status/anthropometry, or breastfeeding practices. All five groups, including the food security and vulnerability indicators are needed for monitoring and evaluating national-level nutrition programs. Each category of indicators answers a different set of M&E questions and includes different indicators. The last category, which is household food security, may require data at the community level, data on food/agriculture inputs and production, and weather and ecology. Household food security indicators may also need to be geographically disaggregated (e.g. using GIS if available).

19. Most Common Indicators
Nutritional status
Weight-for-age and/or height-for-age
Body Mass Index in women
Anemia prevalence
Vitamin A deficiency
Infant and young child feeding practices
Timely initiation of breastfeeding
Exclusive breastfeeding rate
Complementary feeding rate
Extra feeding for malnourished/recently sick children
This slide presents eight outcome indicators commonly used for evaluating nutrition interventions.. Each indicator is constructed from a set of questions around which there are conceptual and statistical considerations. The objective of the program and focus of nutrition interventions should guide the choice of indicators among these, as well as availability of good quality data. Low height-for-age reflects past under nutrition or past growth failure and is a measure of stunting in an individual or the extent of stunting in a population. Low weight-for-age is a measure of nutritional risk in a population and is recommended as the indicator for assessing change in the magnitude of malnutrition over time. Low Body Mass Index (ratio of weight to height (kg/m2))measures chronic energy deficiency or thinness in non-pregnant women. Anemia prevalence is used as a measure of iron deficiency but as we mentioned earlier, it is not a specific indicator of iron deficiency. In some settings, other nutritional deficiencies (vitamin C) and malaria and helminth infections may cause more than 50% of anemia seen in children. The second group of indicators measures appropriate infant and young child feeding practices. The first three indicators in this group reflect whether women have adopted behaviors consistent with the recommendations that women initiate breastfeeding within one hour of giving birth and that infants aged 0-5 months be exclusively breastfed and that starting at age 6 months, children should consume complementary foods in addition to breastmilk. The last indicator in this group measures extra feeding for malnourished or recently sick children to minimize the nutritional impact of illness in children.

20. Most Common Indicators
Micronutrient Interventions
Vitamin A supplementation
Iron supplementation
Coverage with iodized salt, other fortified foods
Household Food Security/Vulnerability
Daily meal frequency of family/individuals
Perceived inadequacy of food reserves in the home/community
Micronutrient interventions look at a different set of indicators. Vitamin A supplementation measures coverage achieved through national vitamin A supplementation programs in a specified period. The iron supplementation indicator that is commonly used measures whether women who gave birth in a given reference period received the minimum number of iron/folate supplements in the form of tablets, based on local policy or international standards. Coverage with iodized salt is used as a measure of the proportion of the population who may be receiving adequate amounts of iodine. Concerning the measurement of household food security or vulnerability, there are many indicators that can be used. The two most common nutrition indicators are daily meal frequency of families/individuals and perceived inadequacy of food reserves in the home or community. Other ways of measuring household food security include individual intakes, dietary diversity and indices of household coping strategies. We will not cover these indicators in this session. M&E specialists should consult nutrition experts to clarify nutrition concepts such as, exclusive versus full breastfeeding, inadequate food reserves and meal frequency or for questions such as how to verify if salt/a food sample is fortified.

21. Data Collection Systems
Routine
Sentinel food and nutrition surveillance
Institutional health records- clinics, schools
Feeding & cash or food transfer programs records- daily/weekly/monthly attendance
Non-routine
Population-based surveys
Emergency appraisals, rapid assessments
Experimental and operational research
What are the data sources for these indicators? Data collection systems can be routine or non-routine. Routine data collection systems include sentinel food and nutritional surveillance; institutional health records as in clinics or schools; records from feeding and cash or food transfer programs. Non-routine data sources include population-based surveys such as the Demographic and Health Surveys and Multiple Indicator Cluster Surveys, emergency appraisals; rapid assessments; experimental designs of efficacy of alternative treatments/delivery channels/processes; and disaster early warning systems.
Class Activity
Ask participants: What data sources do nutrition program use in your own country? (CRS growth surveillance program; CARE food security information system).

22. Anthropometric Measures (1)
Children:
Weight-for-age (underweight)
Reflects chronic or acute malnutrition or both
Height-for-age (stunting)
Reflect chronic (prolonged, cumulative) malnutrition
Weight-for-height (wasting)
Reflects acute and recent malnutrition
This slide presents common anthropometric measures for children. Anthropometry is the measurement of an individual’s body size e.g. weight, height, circumference and is often used in nutritional assessment. As we mentioned before low weight-for-age measures the proportion of children who are underweight and is a reflection of chronic or acute malnutrition or both. Low height-for-age measures the proportion of children who are stunted and is a reflection of chronic or prolonged malnutrition. Low weight-for-height is a reflection of wasting and reflects acute or recent malnutrition.
Class activity:
Are there certain age groups where any of these indices might be most sensitive to changes in program effectiveness? (Answers: Underweight rises quickly between 6-18 months and peaks in the under two’s or three’s. Wasting peaks between ages months).

23. Anthropometric Measurements (2)
Adults:
Body Mass Index (BMI)
Low weight-for-height ( kg/m2) reflects chronic &/or acute
Mid-upper arm circumference (MUAC)
Thin reflects chronic &/or acute
The two most common anthropometric measures for adults are body mass index and mid-upper arm circumference. As mentioned before, Low Body Mass Index (ratio of weight to height (kg/m2)) measures chronic and/or energy deficiency or thinness in non-pregnant women. Mid-upper arm circumference is often used for screening because it changes slowly in large populations. The percent of women with a middle upper arm circumference below 22.5 cm is what is measured. MUAC is correlated with pre-pregnancy weight and is useful for identifying women at risk of intra-uterine growth retardation, especially where scales are not available.
Class activity:
Are there certain age groups where any of these indices might be most sensitive to changes in program effectiveness? (Answer: MUAC is smaller for women than women 40-49; most sensitive age group for detecting intervention effects is likely to be years.)

24. Data Sources for Anthropometry
MCH programs/clinic records
School feeding- school heights.
Food and nutrition, epidemiological surveillance
Poverty mapping/school height census - heights for chronic, weights for current
Reports from emergency/refugee programs
Give examples from you country of these different data sources. Which have better coverage and which are not widely used or representative?

25. Detecting Low Weight-for-age
Option A
Growth Chart
Option B
Table of weight-for-age cut-off points
Cut-Off Points
Low Weight-for-Age
Girls
Boys
Age mths
Age mths
Concerning data sources for anthropometric indicators, let us take the example of low weight-for-age. There are several ways to detect low weight-for-age. The first option is to use a growth chart, which is commonly done in clinics and growth monitoring and promotion programs. The second option is to take height and weight measurements as in a clinic or population-based survey and then use a table of standard weights. With each option, a clinic or growth monitoring program can choose to count:
How many children fall below the lower line of weight for age (can be seen on a growth chart as deviation from the curve of the upper and lower lines)
How many children did not gain adequate weight (this is usually based on how much they weighed last time)
With Option A, growth faltering can be detected by identifying if the child gained the desirable amount of weight between 2 successive weighing sessions. With Option B, a child who is below -2 SD from the median of the NCHS/WHO reference population in terms of weight-for-age is classified as low-weight-for-age.
Class activity
Ask participants: What is the difference in the various options? Note that for option A you need a growth chart while for option B you need a table of weights.
(Answer: Using adequate weight gain captures children who are at risk of becoming malnourished but may not necessarily be in the malnourished category. This is useful for nutrition counseling and follow up, to prevent children becoming malnourished. Using below the line or below cut-off levels of weight for age counts only those who are already malnourished. This is more commonly used for monitoring and evaluation at the present time.)
Low wt/age
below this line
Low wt for age below this line

26. Statistical Presentation of Anthropometric Indicators
Prevalence
Percent below a cut-off, such as <-2SD or < -3 SD
Mean Z-score values (in SD units)
Z score refers to how far and in what direction the measure deviates from the median of the NCHS/WHO international reference standard
Now, we will turn to the handout with anthropometric indices of nutritional status for children under-five in the 2003 Kenya DHS. As you can see, three indices are shown: Height-for-age, weight-for-height, and weight-for-age. Each variable is presented as two types of indicator. The first indicator is prevalence, which is the percentage below a cut-off such as -3 standard deviations or -2 standard deviations. The second indicator is the mean Z-score in standard deviation units. The Z-score refers to how far and in what direction the measure deviates from the median of the National Center for Health Statistics/WHO international standard.
Class activity
Ask participants:
What are the different conclusions from each of the indicators?
What population groups and regions have what type of nutrition problem?
What are the limitations of these anthropometric indicators? (Answer: (1) reliability of weight measurements (variability between interviewers in performing the task of weighing but can be reduced through extensive training; (2) caretaker’s ability to report the correct age of the child; (3) weight-for-height highly susceptible to seasonality; (4) accuracy of height measurement – problems due to inadequate positioning of child’s head; (5) stunting is a long-term process so prevalence of stunting more responsive to interventions in children < 24 months than in older age groups.)

27. Exercise: Interpreting Standard DHS Nutrition Status Tables
If low HFA is 50%, WFA is 30%, WFH is 15%, which is the worst problem? Why?
Which child is more vulnerable to die: a -sd wasted or a -3sd stunted child? Why? In which age group?
Which characteristics are more important for program targeting: rural/urban, region, sex, age, or birth order?
Class activity
This slide is a follow-up to the class assignment on anthropometric indicators. Ask the first question at the top, and then probe with the second question. After discussion, then add the last question on background characteristics (education, geographic location, marital status, land size, etc.)
Then go on to the interpretation. Let the respondents work in small groups of 3-4 for some minutes. Then they report back to the entire class, each group giving one interpretation, and another group, then they continue until all the interpretations are reported. Then the instructor comments on the correct and incorrect ones. Then ask for the responses to the three questions. Vary procedures according to the level and dynamic of the groups, but leave enough time for each person to attempt to interpret before going onto the three questions

28. Feeding Practices: M&E Considerations
Proportion of infants aged 0-5 months who were exclusively breastfed in the last 24 hours,
Proportion of infants less than 12 months of age who were put to the breast within one hour of delivery,
Proportion of infants aged 6-9 months receiving breastmilk & complementary foods,
Mean number of food groups eaten in the last 24 hours by children 6-23 months of age,
This slide lists four common indicators for measuring infant and young child feeding practices. The first indicator is the exclusive breastfeeding rate. It is defined as the proportion of infants aged 0-5 months who were exclusively breastfed in the last 24 hours. The second indicator is a measure of the timely initiation of breastfeeding. It is defined as the proportion of infants less than 12 months of age who were put to the breast within one hour of delivery. The third indicator is the complementary feeding rate. It is defined as the proportion of infants aged 6-9 months receiving breastmilk and complementary foods. The fourth indicator is measuring dietary diversity among children aged 6-23 months.
Class Activity
Ask participants what the challenges are in measuring the first two indicators? (Answer: The use of a 24-hour recall period for the exclusive breastfeeding rate causes the indicator to overestimate slightly the percent of exclusively breastfed infants because some infants who are given other liquids such as water irregularly may not have received them in the 24 hours before the survey.)
With the timely initiation of breastfeeding indicator, mothers may have difficulty recalling correctly when they initiated breastfeeding for their youngest child. Because the indicator is looking at all infants 12 months of age, it can mask changes in population or health facility practices that may have occurred in recent months. If the timely initiation of breastfeeding indicator is measured at the facility level, it may be affected by selection bias since not all women give birth in facilities.)

29. Appropriate Complementary Feeding
Percentage of infants and young children months of age who receive appropriate complementary feeding
6 to 8 months of age : Breastmilk + other food at least 2-3 times per day + variety of food groups
9 to 11 months of age : Breastmilk + other food at least 3-4 times per day + variety of food groups
12 to 23 months of age : Breastmilk + other food at least 3-4 times per day + variety of food groups
Appropriate complementary feeding is also measured using a questionnaire about the past 24 hours. The indicator consists of continued breastfeeding plus two or more of the following:
Frequency of feeding (2-3 times per day for infants 6-8 months of age and 3-4 times during 9-23 months of age)
Diversity of foods given: this is usually measured as the number of food groups given and programs may choose to promote 3 or four different food groups, most commonly, animal foods, oil/fats, beans/lentils, green/yellow/orange fruits or vegetables in addition to the staple/cereal. Some programs are measuring the use of a specific fortified food.
Amounts that are sufficient to meet the gaps between breastmilk intake and energy and nutrient needs of the child (this is often difficult to measure, but some groups have used pre-measured local containers to determine volumes in order to derive quantities). There is evidence in the literature that frequency is correlated with amounts.
Class Activity:
What are the strengths and limitations of this indicator?

30. Coverage Indicators for Micronutrient Programs
Proportion of children aged 6-59 months who received a high dose of vitamin A in the last 6 months,
Proportion of households consuming adequately iodized (i.e. 15+ ppm of iodine) salt,
Proportion of pregnant women who received the recommended number of iron/folate supplements during pregnancy,
In this slide we present some of the indicators used to measure coverage of micronutrient programs. One of the indicators used to measure vitamin A supplementation coverage is the proportion of children aged 6-59 months who received high-dose of vitamin A in the last 6 months. The UNICEF-recommended indicator used to measure coverage of iodized salt is the proportion of households consuming adequately iodized salt. One of the indicators used to measure iron supplementation coverage is the proportion of pregnant women who received the recommended number of iron/folate supplements during pregnancy. There are several caveats to measuring each of these indicators but in the interest of time, we will focus on the first indicator.
Class activity:
What are the considerations in measuring the vitamin A supplementation indicator? (Answer: (1) defining what is a high dose supplement as this varies with the age of the child. Children 0-5 months of age are expected to receive 150,000 international units (IU) as three doses of 50,000 IU with at least a one-month interval between doses. Children aged 5-11 months are expected to receive 100,000 IU as a single dose every 6 months. Children aged 12 months and older are expected to receive 200,000 IU as a single dose every 6 months according to the Annecy Accord recommendations. (2) Many programs record Vitamin A in their immunization registers or with growth monitoring data. If this indicator is added to an immunization coverage survey, or an immunization tally sheet or register, then the age group for the indicator can be months. However the results will not show whether older children in the month age group are receiving vitamin A or not. (3) Using this indicator alone misses out two other components of a vitamin A supplementation program. What are they and how can they be included in M and E plans? [ Answer: Postpartum supplements for women can be measured from postpartum care registers/cards, and disease-targeted or therapeutic vitamin A from case management records/registers].)

31. Choices in Program M&E Design
Which age groups to measure
Anthropometry, infant and young child feeding,
How to obtain valid measurements
Anthropometry; micronutrients; infant and young child feeding
Timing
Trends; seasonality
Evaluation design
Based on our previous discussions, there are various choices that need to be made in the design of a monitoring and evaluation system or plan for nutrition programs. The first issue is which age groups to measure. This is most relevant to anthropometric measurements and in assessing infant and young child feeding. The second consideration is which methods can be used to obtain valid measurements and this is relevant to anthropometric, micronutrient, and infant and young child feeding indicators. The third issue is the timing of measurements. The fourth issue is the evaluation design. Now, let us look closely at each of these factors.
Class activity:
(1) Can you identify the various reasons for selecting certain age group for certain indicators? (Answer: when program effects are most likely to be measurable; nature of intervention/recommended protocol; ease of data collection; opportunity to obtain data from integrated monitoring/evaluation system).
(2) What are the most threats to valid data and its interpretation for each of the following: anthropometry, micronutrients (iron, iodine, vitamin A), infant feeding (breastfeeding, complementary feeding)?
(3) How does timing and seasonality influence the validity of results? (Answer: Seasonality can influence nutritional status and trends in nutritional outcomes if surveys are not conducted during the same season. Some indicators like wasting are sensitive to the time of the year in which the survey is conducted.)
(4) How can an evaluation plan strengthen its ability to attribute observed changes to the program and interventions? (Answer: by having a control group or comparison group)

32. Examples of Flaws in Nutrition Evaluations
No comparison groups
No pretest or baseline
No control for age, e.g. < 6 mo.,< 2 and 3+ yrs
Not accounting for confounding factors
Seasons not comparable
Not controlling for mortality reduction
Non-representative samples, small samples
Pilot projects, not replicable
Here are a few examples of flaws in the nutrition evaluations. In the first case, there is no comparison group. This type of design is not recommended if you are trying to determine the effect or impact of a program intervention but may be useful in diagnostic studies to determine the reasons why a problem exists. In the second case, there is no baseline. If there is no baseline, you can only measure the effect of a program on the intervention group by comparing that group to a control or comparison group. If there is no comparison group and baseline data are also unavailable, then you cannot determine the extent of change in nutritional outcomes at all. The third problem is not controlling for age and other confounding factors like mother’s education, etc. As we mentioned earlier, if seasons are not comparable from one data collection period to another, then it is difficult to establish trends. So if you wanted to have a time series design that involves repeated measurements before and after the nutrition intervention, that might be problematic. The next problem is not controlling for mortality reduction and the fourth is having non-representative or small samples. For example, if most of your anthropometric measurements are clinic-based, the data will not be representative of the general population in areas where use of health facilities is low. Finally, many pilot projects may not be replicable when interventions are delivered through health services in restricted areas under close supervision. Such pilot projects may answer the question of whether, given ideal circumstances, the intervention has an effect. But the interventions may not have an effect when you try to replicate them under “real-life” circumstances faced in other areas.
Class activity:
How can these problems be addressed in an evaluation plan? (Answer: whenever possible, try to create a control group. When random assignment is not possible, try to find a comparison group that is as similar as possible to your intervention group. If you cannot use a time series design, at least try to obtain baseline or pretest information before a program starts. If baseline or pretest information is unavailable, you should consider using multivariate analytic techniques. Always keep in mind the issue of validity. Are there threats to validity: history, selectivity, maturation, mortality or instrumentation) that might explain the results?)

33. Economic Analysis in Nutrition M&E
Cost-effectiveness analysis
compares two or more alternatives for achieving coverage or scale or behavior change, or a process outcome such as training to build capacity
Answers the question ‘which is the more efficient option?’
Used more in evaluations
Cost-benefit
compares the resources required to achieve impact and the monetary value of that impact
Answers the question ‘is the investment worthwhile?’
Based on many assumptions with limited empirical evidence
Nutrition programs are increasingly including some type of economic analysis in their monitoring and evaluation activities. In this slide we present two approaches to the economic evaluation of nutrition programs. The first is cost-effectiveness analysis and the second is cost-benefit analysis. We will discuss each of them in turn. Cost-effectiveness analysis compares two or more alternatives for achieving coverage or scale or behavior change or an output such as training to build capacity. It answers the question: which is the more efficient option? Cost effectiveness analysis is used more in evaluations than cost-benefit analysis. Cost-benefit analysis compares the resources required to achieve impact and the monetary value of that impact. It answers the question: is the investment worthwhile? Cost-benefit analysis is based on many assumptions, often with limited empirical evidence.
Class activity:
When is it useful to include cost considerations in an M and E plan? Give examples of situations where cost information was useful or can be useful.

34. Example: Use of Data to Assess Program Gaps
The figure shows current levels of six key Essential Nutrition Actions (ENA) indicators in developing countries. Six indicators are presented: (1) the exclusive breastfeeding rate; (2) weight/age in children under 36 months; (3) vitamin A supplementation coverage; (4) iron supplementation coverage; (5) continued feeding during diarrhea; and (6) iodized salt coverage. The lighter portion of each bar shows current coverage. The darker portion shows the unmet need. Countries included in the figure were selected from the 42 countries with the highest numbers of child deaths, as identified in Black et al, Countries with sufficient and available DHS data (1998 to 2002) for the key indicators were included: India, Tanzania, Uganda, Mali, Malawi, Zambia, Nepal, Egypt, Cambodia, Rwanda. Data for iodized salt were not available for Nepal and Tanzania. The data for vitamin A supplementation in Egypt are for children months rather than 6-59.
Source: Black, Robert E., Morris, Saul S, and Bryce, Jennifer Where and why are 10 million children dying each year? Lancet, 361:
Class activity:
(1) Why is this way of presenting results useful? (Answer: it is comparative across interventions, helps priority-setting, helps set targets/objectives. Although it shows no indicator for complementary feeding, it helps communicate key messages in one visual.)

35. Monitoring and Evaluation Challenges

36. Challenges of M&E Multisectoral programs (attributing outcome?)
Clinical Indicators
May need large samples (e.g. xerophthalmia)
May be sensitive to enumerator training (e.g. goiter)
Measurement of iron deficiency (lack of specificity)
Selection bias (institution based sample)
There are a number of challenges to monitoring and evaluating nutrition programs. The first challenge is introduced by the multi-sectoral nature of programs that address malnutrition. Because interventions are introduced from several sectors, for example, agriculture, education and livelihoods, and because programs are introduced as packages of services, it is difficult to establish that observed changes in nutrition outcomes are attributable to a particular intervention. The measurement of clinical micronutrient indicators brings it own challenges. For example, large samples are needed to measure accurately xerophthalmia. Xerophthalmia is an eye disease caused by vitamin A deficiency. The condition begins with night blindness and dryness of the cornea and, in the later stages, to softening of the cornea. The measurement challenge arises because xerophthalmia is relatively rare, with a prevalence of about one percent in preschool children in affected areas. The issue of large sample size needs is crucial for the measurement of changes in the prevalence of xerophthalmia over time. Assessment of the prevalence is goiter is sensitive to the training of enumerators. Differences in training to recognize goiter that is palpable but not visible may lead to variations in goiter prevalence rates and low inter-rater reliability. A third challenge arises when monitoring and evaluating the prevalence of iron deficiency. Anemia is commonly used as an indicator of iron deficiency but people can be anemic for reasons not due to iron deficiency, such as malaria and intestinal parasites (such as hookworm or whipworm). In addition, lack of vitamin C, dietary tannins as in tea, and phytates in cereal inhibit iron absorption. Selection bias is also an issue with clinical samples. Clinical samples are unlikely to represent the population in places where the use of health facilities is low.

37. Challenges: Comparisons & Trends
Sample design
Sample size
Cutoff points & standards
Seasonality
This slide deals with issues that must be considered when making comparisons of and assessing trends in nutrition outcomes. The first challenge pertains to the sample design. Samples for assessing nutritional status can come from population-based surveys, clinics, or schools. Sometimes schools and clinic samples are judged by those compiling and using data to be at least better than nothing. However, school- and clinic-based populations are unlikely to represent the general population. It is important that data from different sources not be aggregated and combined when assessing trends in nutritional status. Another issue is sample size. If the sample size is small, this would negatively affect the confidence intervals around the estimates. As we all may know, many school and clinic samples are generally small. A third issue is which cutoff points and standards are used to determine whether or not micronutrient deficiencies exist or whether women and children are malnourished. The cut-off points for indicators of a woman’s nutritional status can vary according to the period within the reproductive life cycle (i.e. during pregnancy and lactation). For example, it is recommended to use a prevalence of hemoglobin of < 120 g/L for non-pregnant women and 110 g/L for pregnant women. For child anthropometry, different standards may be used. The National Center for Health Statistics/WHO standard is recommended. However, in clinic-based growth monitoring, the Road-to-Health System is typically used instead of the NCHS/WHO system. When examining malnutrition patterns over time, it is important to use the same system to analyze and present data because the cutoff points for mild, moderate and severe malnutrition are different in each classification system. In the case of women’s nutritional status, applying the correct cut-off points depends on determining whether a woman is pregnant or not; such a determination is sometimes not possible. Finally, seasonality may be a source of considerable error in measuring the prevalence of underweight, stunted or wasted children in many environments. Seasonality can lead to differences in as much as 10 percentage points or more between the high and low values of underweight throughout the year. As a result, attempts should be made to repeat surveys in the same season.

38. References
Arimond, Mary and Marie T. Ruel Generating Indicators of Appropriate Feeding of Children 6 through 23 Months from the KPC Washington, D.C.: Food and Nutrition Technical Assistance Project, Academy for Educational Development.
Cogill, Bruce Anthropometric Indicators Measurement Guide. Washington, D.C.: Food and Nutrition Technical Assistance Project, Academy for Educational Development.
Wasantwisut, Emorn Recommendations for monitoring and evaluating vitamin A programs: outcome indicators. Journal of Nutrition, 132: 2940S-2942S.
Ruel, M.T., K.H. Brown, and L.E. Caulfield Moving Forward with Complementary Feeding: Indicators and Research Priorities. Food Consumption and Nutrition Division Discussion Paper #146. Washington, D.C.: International Food Policy Research Institute.
WHO a. Assessment of Iodine Deficiency Disorders and Monitoring their Elimination: A Guide for Programme Managers. Second Edition. WHO/NHD/ Geneva: World Health Organization.
WHO. 2001b. Iron Deficiency Anaemia: Assessment, Prevention and Control - A Guide for Programme Managers. WHO/NHD/01.3. Geneva: World Health Organization.
Wellstart International’s Tool Kit for Monitoring and Evaluating Breastfeeding Practices and Programs.
Suggest local facilitators obtain materials from country offices of WHO, UNICEF, university, NGOs like HKI and CARE. Also see the websites for USAID projects such as Linkages, FANTA, MOST and BASICS.

41. Madagascar Nutrition Case Study
During , Madagascar followed a comprehensive model, the “essential nutrition actions” (ENA) framework, which coordinated efforts from the community level through national policy making, and included both government and non-government entities. The model was first implemented in two districts in the Antananarivo and Fianarantsoa provinces. It focused on a set of proven interventions covering micronutrients and dietary practices for mother and young children. From 1995 to 1998, the overall focus was placed on designing mechanisms that linked nutrition interventions more directly with other child health and RH services, and national- and community-level actions. Further instructions are provided in the handout.
Class activity:
This case study focuses on the nutrition program in Madagascar. During , Madagascar followed a comprehensive model, the “essential nutrition actions (ENA)” framework, which coordinated efforts from the community level through national policy making and included both government and non-government entities.
First, we will divide the class into three groups. Each of you should have received a slip of paper with a group name written in it. The first group will focus on the behavior change component of the nutrition program. The second group will focus on the health worker training component, and the third group on the integration of ENA into maternal and child health services. Each group will develop a logical framework for monitoring and evaluating the assigned component of the nutrition program, propose a set of indicators for M&E of that component. Finally, for one proposed output and one proposed outcome indicator, each group will identify the indicator metrics, data sources, and the indicator’s strengths and limitations.