Presentation on theme: "Interim Monitoring & Evaluation Guidance for the BRACED Programme II"— Presentation transcript:
1 Interim Monitoring & Evaluation Guidance for the BRACED Programme II BRACED Interim Knowledge Management WebinarInterim Monitoring & Evaluation Guidance for the BRACED Programme II16 May 2014
2 Purpose of the second webinar To address key questions from Grantees submitted after the first webinarTo update Grantees on status of emerging guidanceTo explore key methodological issues in more detailTo present draft ‘Bronze, Silver, Gold’ standards for key aspects of project- level M&E (‘M&E Standards’)
3 Content of webinar Update on emerging guidance Measuring KPI1 – numbers supportedCollecting and using climate and related dataConstruction and using resilience indicators (KPI4)Attributing changes in resilience to project activities (KPI4)Use of control groups in attributionReport from Concern on Crisis Resilience Indexing System
5 Guidance available in May 2014 Methodological guidance for all 15 KPIs (note KPI4 being revised below)Full set to be shared with grantees ASAPAdditional guidance on definition of direct beneficiariesBRACED website – complements guidance to KPI1 (1 above)Revised KPI4 guidanceIn development – available on BRACED website by end of MayMeasuring Resilience reportFinal version to be on BRACED website before end of MayRevised log-frameNext iteration to be published on website shortlyProgramme monitoring planIn development – to be circulated by end of MayValue for Money for Adaptation frameworkFlyer will be on BRACED website next week, full version following week (end of May)
6 2. Measuring KPI 1 – numbers supported Martin Whiteside6
7 KPI 1 - Number of people supported by DFID programmes to cope with the effects of climate change Q – Is the “number of people supported” in the BRACED logframe (KPI 1) the same as our direct or indirect beneficiaries?Measuring an reporting KPI 1 is mandatory for all projects;Need consistencyCheck definitions in KPI 1 guidance notesDifferent projects/organisations have different definitions of ‘Direct’ and ‘Indirect’ beneficiaries – for KPI 1 reporting need to use KPI 1 definitions that are specific to BRACED (see the BRACED website);Report number of individuals disaggregated by gender (can be calculated from HH numbers as long as clear whole family are ‘supported;Counted and reported annually
8 DEFINITIONS KPI 1 - Number of people supported by DFID programmes to cope with the effects of climate change‘Support’ - direct assistance from the programme in question, with the explicit intention of helping people deal with climate change impacts. It could include for example financial resources, assets, agricultural inputs, training, communications (e.g. early warning systems) or information (e.g. weather forecasting). ‘People supported’ - identified by the programme in question with a direct relationship to it. ‘Effects of climate change’ - existing and future, sudden or gradual, arising from primary consequences of CC (changes to precipitation, temperature and sea level rise) - can include floods, droughts, storms, landslides, salination, coastal inundation, heat or cold waves and biodiversity loss.
9 Categories for reporting KPI 1 - Number of people supported by DFID programmes to cope with the effects of climate changeBRACED DIRECTA – High intensityBoth targeted and high intensity e.g. people receiving social protection cash transfers, houses raised on plinths, agricultural extension services, training of individuals in communities to develop emergency plans and use early warning systems.B – Medium IntensityTargeted & Medium intensity: e.g. people receiving weather information and text message early warnings.Not targeted & Medium intensity: e.g. people within the coverage of an early warning system, or catchment area of a large infrastructure project (e.g. flood defences), or living in a discrete community in which others have been trained in emergency responseBRACED INDIRECTThis does not contribute to the KPI 1 headline figure, but can be reported separatelyC – Low intensity: e.g. people falling within an administrative area of an institution receiving capacity building support, or catchment area of a Water Resources Management plan (can be captured through the programme’s own monitoring, or ‘institutional development’ scorecard).
11 3. Collecting & using climate data Martin Whiteside11
12 Collecting Shock and Stress (S&S) Data Q - How should we collect climate shocks/trend data, and how technical and detailed should it be?Key concepts:Concentrate on S&S being addressed by your projectBase focus on S&S as experienced by your target beneficiaries;Triangulate against S&S relevant ‘scientific’ data
13 S&S Data - STEPS Planning Phase Define the S&Ss your project is building resilience against;Participatory enquiry for each S&S:Beneficiary prioritisation of different S&S (understand differences)Severity category for main S&Ss – v. severe; severe, normal, good?Ground truth against known years/events in pastHistoric agri/met/hydro/well-being ‘scientific’ datae.g. crop prod., rainfall, flood levels, deaths from disasterReliability, specificity, relevanceTriangulationRelevance to community experience of S&S (e.g. days without rain in growing season may be more important than monthly rainfall)Data gaps – can we fill in implementation?Correlation between community perception and ‘scientific’ data?TrendWhat trends can be discerned? (Across perception and measured?)What are implications?
14 S&S monitoring - Draft Standards BronzeSilverGoldDefinition of shocks and stresses (S&S)S&S identified and described by programme staff or key informantsStaff/key informant identification of S&S supplemented by beneficiary information on priority and characterisationEach Shock or Stress prioritised and defined through participatory approaches across range of beneficiary types and contexts, ground truthed against specific events and years.‘Scientific’ data collectionLimited secondary data (e.g. mm rain/year) and/or not very congruent with project area or sub-areas and/or with doubts on reliabilityData relevant to some of the key shocks/stresses, reasonably congruent with project area and with fair reliability. Limited supplementary primary data collection.Relevant, reliable, geographically specific data covering most of key S&S, disaggregated if necessary across key zones of project area. Supplementary primary data collection organized as needed.Community shock and stress perception monitoringLimited ranking of very severe, severe, normal and good for year.Ranking disaggregated by key S&S and beneficiary type and or geographical area.Ranking disaggregated by S&S and beneficiary type and or geographical area. Clear participant explanation for each ranking and comparison with previous similar years/events.TriangulationLittle possibleSome correlation identified.Significant correlation.Trend dataWeak - scientific data of limited reliability, specificity and relevance supplemented by anecdotal views;Moderate – reasonably reliable, specific and relevant scientific data correlated in some cases with community perceptionsStrong - reliable, specific and relevant scientific data correlated cases with community perceptionsLearningLittleSignificantVery significant
15 S&S Data - STEPS Implementation Phase Beneficiary S&S perception monitoring – participatory, clear criteria (can include questions on how project outputs have affected these)Agri/met/hydro/well-being ‘scientific’ data - collectRelevant to S&SMay need to supplement/re-format/analyse/interpretTriangulation – annual/eventUse of information - learningBeneficiary S&S priorities and experiences during project,Contextualising project outputs and outcomes – did these address felt S&Ss?Deepening understanding of relationship between ‘scientifically measured’ and experienced S&SsEnabling explanatory correlation between resilience outcomes and well-being impactAdditional understanding of trendsFeed into continual project improvement.
17 4. Constructing & using resilience indicators Nick Brooks17
18 What should resilience indicators measure? Factors/attributes we think will make people better able to anticipate, avoid, plan for, cope with, recover from & adapt to stresses & shocksThese will be context-specific & should be identified during development or early in implementation phase through, e.g. participatory assessmentsFactors/attributes to be measured are those that will/may be influenced directly or indirectly by the project (positively or negative – potential for unintended consequences)Other factors/attributes identified as important for resilience but that are not related to project activities should be noted – these might be tracked to provide context, but will not contribute to calculation of numbers with improved resilience for reporting against KPI4
19 Resilience outcomes & project outputs Resilience measured at outcome level in BRACED log-frameChanges in resilience usually measured as project outcomesTwo broad types of resilience outcome indicator:Indicators that measure numbers of people sustainably adopting project outputs – essentially measures of uptake of project outputs, where there is good evidence that this ill improve people’s resilience (uptake indicators)Indicators that measure changes in factors/attributes/status that we believe (based on evidence) will make people more resilient that are not tied to project outputs & may be measured for non-beneficiaries (status indicators)Ideally, project will complement (1) with (2)
20 Types of indicatorsQualitative, e.g. based on beneficiary perceptions of how easily they can cope with particular stresses/shocks, access resources that make them more resilient, etc.Quantitative – binary, e.g. value of 0 for ‘no’ & 1 for ‘yes’ in answer to certain questions (e.g. does beneficiary have access to certain resource, meet a certain criterion that is important for resilience?)Quantitative – categorical/score based, e.g. assign score based on category of resilience as measured by a particular indicator (low, moderate, high, etc.)Qualitative indicators can be converted to quantitative score-based indicatorsQuantitative – continuous, e.g. based on measurement of a continuous variable such as household income, time to recover from previous shocks.Projects likely to employ a mixture of the above
21 Composite & individual indicators Projects might use:A number of individual indicators, each representing a different aspect of resilience as relevant to the projectMeasure changes in each indicator separately for individuals sampledSeveral composite indices, each representing a different dimension of resilience as relevant to the project:e.g. income & food access; safety nets, access to services, adaptive capacity, etcGuidance on dimensions under KPI4, but not prescriptiveA single composite index, representing resilience as relevant to the projectJudge whether such aggregation appropriate based on nature of indicatorsProject might measure only one dimension of resilience (cf (2))
22 Construction of composite indicators How to aggregate different types of indicator?Need to convert to common format, e.g. using:Discreet scores:Split range of continuous variables into divisions, with each division represented by a score, e.g. 1-3, 1-5;Define categorical indicators using same scoring systemAdd binary indicators (e.g. 3 or 5 indicators for same range as above)Scaled values: e.g. from 0-1Add values together or take average across valuesNeed to consider weightsAssign to constituent indicators based on relative importanceUsually done subjectively, not straightforwardThese issues are less pressing (although not necessarily irrelevant) for individual, disaggregated indicators
23 Numbers with improved resilience For panel data/ longitudinal studies (sampling same individuals)Single composite indexNo. showing increase in index value minus no. showing decreased resilienceMultiple composite indicesNo. showing improvement in 1 or more indices (and no decreases) minus no. showing decreased resilience in 1 or more indices (and no increases)Multiple disaggregated indicesFor ‘improved resilience’, require a minimum number (X) of indicators to show improvement, and a maximum number (Y) to show deterioration.X and Y to be set according to context, with X > YMight need to demonstrate improvement in a set of core related indicatorsVice versa for ‘decreased resilience’Number with improved resilience is number fulfilling (a) minus (b)Scale up from sample to beneficiary population, ensuring sufficient sample size and considering disaggregation (take statistical advice)
24 Numbers with improved resilience Where sampling does not involve the same individuals each timeDefine a threshold for improved resilience, based on e.g.Movement out of ‘low resilience’ category, to moderate or highFor disaggregated indicators need to decide how categories constructedMinimum value of a composite indexSample at different periodsBaseline data gathering period t0Subsequent sampling periods t1 , t2 … tn-1 , tnNumber with improved resilience is:No. in lowest resilience category at tn-1 minus number in this category at tnNo. above resilience threshold at tn minus no. above threshold at tn-1Sampling methodologies (e.g. panel survey or samples using different beneficiaries) will have implications for how indicators are constructed
25 Resilience indicators – Draft Standards BronzeSilverGoldCounting method (KPI4)No. with improved resilience, based on no. adopting project outputs, no. moving out of lower resilience category, or no. crossing res. thresholdAs Bronze, plus additional data on changes in numbers in multiple categories (e.g. low, med., high resilience) – panel data or other samplingAs Bronze, plus additional data on how much resilience has improved based on sampling of same individualsType of indicatorIndicators measuring uptake of project outputs supported by evidence that this improves resilience (can include indirect beneficiaries where uptake beyond direct beneficiaries, e.g. emulation)As Bronze plus one or more of:beneficiary perception-based indicators - better/worse than before; low, med., high for different aspects of res.More ‘objective’ criteria for different levels of resilience or position with respect to resilience thresholdMix of uptake, perception-based & objective indicators, with objective indicators based on more detailed categories and/or continuous variables. Assessment of consistency across different indicatorsTimingBaseline and endDuring implementation; 1 ex-post measurement;During implementation; ≥1 ex-post measurementsEvidence base for indicatorsQualitative evidence based on beneficiary feedback & good theory of changeAs Bronze plus case study evidence from past stresses and shocksAs Silver plus correlation of indicators with losses for past stresses & shocksDisaggreg-ationGenderGender + other pre-determined classes
27 5. Attributing changes in resilience to project activities Nick Brooks27
28 Attribution vs. Contribution Grantees (& DFID) want to determine whether (and to what extent) the intervention is making a differenceAttribution: Determining the degree to which the projects are ‘causing’ the observed resilience outcomes (and measured well-being impacts);Contribution: Determining the degree to which the projects are ‘contributing to or helping to achieve’ the observed resilience outcomes (and impacts);In practice this distinction not critical – projects will ask:Are resilience & well-being (as measured by indicators) changing?To what extent is the project responsible for these measured changes?To what extent can we say project contributed to or caused these changes?If evidence suggests changes would not have happened without the project, we can attribute changes to project; if evidence suggests project played a role but was not sole driver, we can say project contributed to changesIn practice there will be a continuum from no contribution to full attributionNotes from Euneka:The paper Broadening the range of designs and methods for impact evaluations, 2012 of Stern, et.al. for DFID provides a good summary. Other reliable source include papers by John Mayne (e.g., Addressing attribution through contribution analysis: using performance measures sensibly. Canadian Journal of Programme Evaluation 16; 1‐24), who developed this form of analysis.
29 How much of improved resilience is due to project? No. with improved resilience no. with improved resilience due to project supportAssess usingBeneficiary feedback (qualitative explanatory enquiry)Build questions on where/extent to which project helped build resilience into surveys used to gather data for resilience indicators (panel & other survey)Narratives from beneficiaries (panel survey: compare narratives over time)Comparisons of resilience (as represented using indicators)Before & after interventionBetween groups at different stages of a phased interventionBetween beneficiaries and comparison/control groupsComparison/control groups must have similar characteristics (e.g. livelihoods, economic/policy contexts) and be exposed to similar/same hazards
30 Characterising contribution Qualitative description of contributionIf X people have improved resilience, and there is good evidence* that project played a role, can say project contributed to improved resilience of X peoplee.g. demonstrable differences between project area and other similar areas coupled with feedback from key informants indicating project played significant role in these differences; resilience outcomes match theory of change and no other plausible explanations; etc.How much did project contribute, and how did this vary across different groups?Quantification of contributionBeneficiary perceptions from surveys – what proportion (X) of sample representing beneficiary population N, say project contributed?Number with improved resilience due to project = X*NCan also ask in what ways project contributed, and by how much (a little, a lot, etc.)“Difference in difference”1 - what proportion of people are more resilient in sample of beneficiary population (X) than in sample of control population (Y)?Number with improved resilience due to project = (X-Y)*NNeed large, representative samples – seek statistical advice1As described by Khan and Anderson (2014):
31 Attribution/contribution – Draft Standards BronzeSilverGoldQualitative explanatory enquiryProject level estimate of contribution of project based on combination of qualitative and quantitative reasoning.As Bronze, but informed by questions related to role of project built into surveys of beneficiaries. Include those ‘excluded’ from benefit.As Silver, but also informed by beneficiary narratives derived from panel studies. Include those ‘excluded’ from benefit.CounterfactualBefore/after intervention.Use of phased intervention approach to compare beneficiaries at different stages of intervention through panel survey.Use of control/comparison groups exposed to similar/same stresses & shocks.Assessment of contributionProject ‘contributed to’ improved resilience of X number of people.Qualitative description of extent to which project contributed, e.g. one of several factors, significant or major (but not sole) factor, resilience would not have been improved without project; describe for different groups of beneficiaries.Quantitative characterisation that indicates the % of the total numbers with improved resilience that can be attributed to the project and/or the degree of change that can be attributed to the project.31
33 6. Use of control groups in attribution or contribution Nidhi Mittal33
34 Role of Control GroupsRole of control groups in measuring attribution or contributionThe evaluation framework, including data sources/methods depends on whether the focus is on attribution or contribution.Attribution analysis will require more experimental type evaluations:Use of experiment (and control groups) tries to isolate one factor—the receipt of an intervention or a project , everything else being constant.Individual households or communities are randomly assigned to the control group (randomised control trials).Quasi‐experiments — similar groups of beneficiaries, or communities are used to create comparison groups ( no randomisation involved)Contribution analysis can be more theory-based and supported by a broader range of methods and sources of data to develop a consistent narrative.Quasi experiments, case studies, correlation studies, longitudinal studies, and sample surveysQuestions related to attribution vs contribution:Some questions addressing attribution might be: Are the resilience outcomes attributable to the project? Are the resilience outcomes changing as a result of the project? Did the project cause the resilience outcomes?Some questions related to contribution are: Is the program contributing to the resilience outcomes ? Are the resilience outcomes changing? Is there evidence that the project helped to achieve or was part of what caused the resilience outcomes?Choice of evaluation frameworks and methodologies: The evaluation framework and methods, including the data sources, need to match the evaluation questions and whether we are measuring attribution or contribution. Experiments and quasi‐experiments are used to show attribution, while quasi experiments, case studies, correlation studies, longitudinal studies, and sample surveys are used to show contribution.Use of control groups:Use of Experiments (or randomised control trials) try to isolate one factor—the intervention or a project while holding everything else constant. This involves random assignment of individual households or communities to the experiment or control group. Ideally, the experiment and control group will be identical except in receipt of the intervention or program. Experiments give the greatest amount of confidence that change is due to the project. Experiments, therefore are used to show attribution. For e.g. in a randomized control trial, the health effect of a drug on those who take it versus those who received a placebo can be tested effectively. Similarly the comparative resilience outcomes of beneficiaries who received BRACED project support or not can be tested to measure attribution.Quasi‐experiments can be used to show attribution or contribution. They do not involve randomization. Instead, similar groups of beneficiaries, or communities are used to create comparison groups.34
35 Effective use of Control Groups Beyond KPI4: using control groups to assess resilience impactsUse of control groups can be extended beyond measuring KPI 4.The approaches for assessing improved resilience outcomes are broadly applicable to assessing resilience impact indicators (e.g. well-being indicators , loss and damage).Stakeholder discussions:Engaging with stakeholders to ensure that the methods used are feasible, realistic, and responsive to needs, and appropriate to the intervention.Seeking expert advice to identify the types of questions to be asked, understanding fully the risks of leading questions.Stakeholder discussions:When approaching discussions with the stakeholders, it is important to understand and ask about the nature and characteristics of the intervention. This puts us in a better position to decide on appropriate models and methods, and determine whether or not attribution is the primary concern. It is important to be frank and realistic with stakeholders. We need to help them understand the value of showing contribution, as well as the challenges of attribution, and at the same time manage their expectations. Stakeholders need to be able to accept some uncertainty. Engaging stakeholders throughout the process helps ensure that methods are feasible, realistic, and responsive to their needs.Ethical Challenge: While there is no guidance specific to RCTs in development evaluations we can refer to DFID’s Ethics Principles for Research and Evaluation covers consent, harm avoidance, human rights, privacy and confidentiality, etc. Another commonly used guidance albeit specific to medical research is the Helsinki Declaration on Ethical Principles for Medical Research involving Human Subjects (http://www.wma.net/en/30publications/10policies/b3/), which can cover treatment of vulnerable groups and individuals. Both should require ethical clearance by local authorities (and internal review committees, if any) and safeguard privacy, confidentiality, and right to withdraw.Challenges in measuring attributionWe can often measure whether or not an intended outcome occurs, but it is difficult to determine what outcomes are attributable to a specific project. A broadly‐defined target population makes it difficult to show attribution in resilience evaluation. In addition, the complex and comprehensive nature of some resilience projects (as well as the strong link between adaptation and development initiatives) makes inferringcausation extremely difficult because there may be multiple projects or initiatives designed to support each other with multiple activities which may not have explicit, measurable objectives. In addition, projects may not be carried out in a short time frame, which also makes it difficult to show a causal relationship. Projects also operate in complex social environments. There are varied and dynamic variables affecting the environment within which the projects or multiple programs operate, such as socioeconomic, environmental, political, and cultural factors,which usually cannot be isolated, manipulated, or measured, which makes it difficult to show attribution. Change is seldom attributable to a simple factor and the causal explanation in evaluation may have too many influencing variables, especially when projects are aiming at long‐term outcomes, which increases the chance for confounding variables and makes it challenging to show attribution.Broadly defined target beneficiary groups or populationsAdaptation-development links and nexusDifficulties in inferring causation due to inexplicit or ‘hard to measure’ objectivesComplex environments : socioeconomic, environmental, political and cultural factors which cannot be isolated, manipulated or measured.A number of interlinked and influencing variables that drive changeLong timeframes to see visible demonstrable impactUncertain climate risks and impacts and shifting baselinesChallenges in measuring contributionWe would all like to be able to attribute success fully to our projects, that would be nice but it is also rare. Claiming to make a contribution to a desired result is oftentimes more likely. Evidence of contribution can be encouraging and provide credible information on the effects of program investment.Overcoming attributional challenges through a focus on contribution:Attributing success fully and completely to adaptation and resilience projects would be ideal but often ‘rare’Claiming contribution can provide encouraging and credible evidence ofContribution analysis is relatively easier to do vis-à-vis attribution.Ethical challenges and limitations of using control groupsThere are ethical questions on how/when to use control groupsWhen an actor interacts with beneficiaries, they expect certain actions.Managing expectations and helping beneficiary groups understand the value and challenges of contribution and attribution is key.DFID’s Ethics Principles for Research and Evaluation (2011) provides further guidance in relation to ethical use of control groups.35
36 Support Available from Interim and full KM Planning future KM support on control groupsProjects should be aiming to deliver their M&E plans as described in the concept notes and whereby they have planned or are planning to use control groups, they should flag support needs to the KM.Although encouraged, control groups are optional as a (gold standard) measure and not mandatory if projects did not originally plan this in their concepts.Guidance from the interim KMThe KPI4 guidance from the interim KM will provide some over-arching principles of using control groups for measuring ‘attribution’ of resilience outcomes or the ‘contribution to resilience of beneficiaries.Brief telephone or consultations/review of approaches to M&E project leads can be provided by the interim KM to the grantees.KM support are relevant to wider m and e (beyond KPI 4 and control groups, for example statistical advice on sampling sizes etc).36
37 Support Available from Interim and full KM Planning full KM supportThe full scope of the guidance and support from the KM cannot be finalised until the permanent KM is in place, the final projects are chosen and resourcing needs overall are assessed.It is likely to be a flexible and iterative process when the full KM pulls together the BRACED monitoring and evaluation approach overall on the basis of the final list of project’s M&E frameworks.Information needed from grantees to plan future KM supportProjects should indicate whether they are planning to use control groupsand which standard(bronze, silver, gold) best reflects their M&E approach.This information should be submitted to the interim KM within the Bronze, Silver, Gold grading standards template to be shared by end May.The interim KM will use this scoping assessment to map projects on the grading standards scale , assess their support needs and inform the tailored support plan for the full KM to provide to the grantees.37
39 6. Concern’s Crisis Resilience Indexing System Aine Magee39
40 Community Resilience Indexing System (CRIS) Aine Magee- Concern M & E Adviser
41 Measuring ResilienceDue to difficulties posed in measuring resilience, Concern has been developing a Community Resilience Indexing System (CRIS) which evaluates the assumed characteristics of a resilient community and comes up with an overall CRIS score per community.The CRIS score indicates the level of achievement of each community against specific indicators/characteristics. It is assumed that a high score, meaning a community has a lot of the necessary characteristics, is indicative of a resilient community. Concern could test this assumption through BRACED.
42 Objectives of CRISThe CRIS is designed to provide a semi-qualitative baseline/ endline comparison tool to assess the impact of work in Disaster Risk Reduction and building resilience, and to facilitate on-going monitoring by providing a comprehensive set of indicators that can be tracked over time.The intention is to investigate the correlation between these assumed determinant of resilience and actual resilience through BRACED, as well as in other contexts.We want to test whether the assumed characteristics of resilience (detailed in the CRIS tool) are correlated to communities who “bounce back better” after a shock.This work will be carried out with input from a research partner.
43 CRIS indicatorsThe CRIS Score is calculated on the basis of proxy indicators across 6 livelihood asset classes:Political AssetsSocial AssetsHuman AssetsFinancial AssetsPhysical AssetsNatural AssetsEach indicator is assigned a score between 1 and 5, where 1 is the lowest level of attainment, and 5 is the highest. An overall weighted score per asset, and per communities is calculated.Indicators have been developed based on DRR/resilience literature and Concern experiences.
44 Progress to date:Two pilots using CRIS to measure characteristics of resilience have been conducted in Zambia and Sierra Leone.Based on these pilots:the CRIS indicator set is being finalisedthe methodology for administering the tool is being refinedthe possibility of developing an app form of the tool for use on laptops, smart phones or tablets is being investigated
45 CRIS in BRACEDConcern hopes to use the revised CRIS tool to measure a related outcome level indicator for BRACED“Average Score across target communities on the Community Resilience Indexing System”.We would hope to see an increase in the average score throughout the programme indicating in an increase in the resilience characteristics of target communities.The suggestion is that the population in communities who show an increase in resilience characteristics over time, can be included in the measurement of KPI 4:“No. of people with improved resilience as a result of BRACED projects “
47 Opportunities for further knowledge sharing If you would like the opportunity to present your own learning regarding M&E tools and methodologies, please let us know via your Monitoring Officer.We would be happy to facilitate another webinar (given sufficient interest) to support the dissemination of your experience.Thank you for your participation in today’s webinar.Any further comments or questions should be sent to your Monitoring Officer and we will respond as soon as we are able.