IASC Inter – Agency Real Time Evaluations (IA RTEs)

Outline What are IA RTEs? What are they good for? Where do IA RTEs come from? How do IA RTEs work? In which ways are IA RTEs different from standard evaluations?

What are IA RTEs?

Definition IA RTEs are ‘Inter-Agency (IA)’ in that the RTEs are instigated, managed and used by a variety of international organizations The term ‘Real-Time (RT)’ refers to the early implementation stages of a humanitarian emergency operation, to the rapid feedback of IA RTE findings at the field level (within the two first month) and consists of remote monitoring and the IA RTE mission. ‘Evaluation’ (E), as applied in the term IA RTE, describes a participatory review process. IA RTEs differ from other humanitarian evaluations regarding speed, coverage, methods, and outputs.

What are IA RTEs good for ?

Purpose IA RTEs seek to: identify gaps, access constraints to the “humanitarian space”, assess the relevance, quality and timeliness of the response, unlock inter-stakeholder coordination problems or operational bottlenecks, propose appropriate strategic re-orientation provide real time learning to the Humanitarian Country Team. identify strengths and weaknesses of the response through the assessment of the response to the affected population.  The primary end users of IA RTEs are in-country agencies and staff engaged in the humanitarian response

SUM UP IASC mandated evaluation tool that is managed and used by a variety of international agencies for real time learning Focus is on inter-agency coordination and management. Carried out at the early implementation stages of a humanitarian operation Rapid feed back of findings at the field levels for immediate use Seek to identify gaps, access constraints & potential threats and assesses the quality of the response

Where do IA RTEs come from?

Towards IA RTEs Humanitarian Response Review Humanitarian Reform Clusters and CERF RTE

Humanitarian Response Review

Results of the HRR (excerpt from recommendations): The IASC and the donors should: establish a limited set of benchmarks (and indicators) to be implemented in the first period of a new emergency up to a maximum of 3 months and addressing in priority access and coverage of population in need identification of responsibilities in delivery of assistance and in coordination resources mobilization (human, assets, financial) The IASC should: identify and assign lead organizations with responsibility at sectoral level, especially in relation to IDP protection and care and develop a cluster approach in all priority sectors. (Action: ERC/IASC – immediately)

Humanitarian Reform 2005 / 2006 Cluster Evaluations IA RTEs

Inter-Agency Standing Committee (IASC) Unique inter-agency forum for coordination, policy development and decision-making involving the key UN and non-UN humanitarian partners. Established in June 1992 in response to United Nations General Assembly Resolution 46/182 on the strengthening of humanitarian assistance. General Assembly Resolution affirmed its role as the primary mechanism for inter-agency coordination of humanitarian assistance. Develops humanitarian policies, agrees on a clear division of responsibility for the various aspects of humanitarian assistance, identifies and addresses gaps in response, and advocates for effective application of humanitarian principles.

IASC WG requests IA RTE Pilot Phase IASC WG requests IA RTEs in 2007: At its 64th meeting held in March 2007, the IASC Working Group endorsed a one-year pilot to inform the developing and testing of approaches to inter agency real time evaluation (IA RTE), with the objective of eventually establishing a regular standby mechanism. At this time, the IASC also endorsed nine tenets / principles of IA RTE and committed to ensuring agency-specific and IASC-wide management response and action to the recommendations arising out of future IA RTEs.

1. Inter-Agency RTE should be used to address broader, cross-cutting and sector wide concerns. 2. The primary purpose of the IA-RTE is to support field management decision-making; a secondary purpose is to provide information to senior managers in agencies to help better understand and support the programmes they are in charge of directing. 3. The adopted approach should be light, flexible and responsive to the needs of field actors and the changing nature of the emergency response. 4. There needs to be a consistent and timely trigger for IA- RTEs – ideally IA-RTEs should already be indicated in a flash appeal/CAP. IASC WG formulates 9 Basic Principles

5. IA-RTEs should not replace agency-specific RTEs - the target audience is therefore the broader humanitarian community and not a single agency. 6. There needs to be a strong focus on the affected population which must inform the method of the IA-RTE. 7. IA-RTE teams should be lean and reflect a mix of agency expertise, methodological skills and sectoral expertise as is appropriate for the emergency. 8. IA-RTE are management intensive and need a full time lead management agency, supported by an inter-agency management group 9. IA-RTE need to result in immediate use and action. This also requires a strong commitment from all participating agencies from field upwards to headquarters to respond to and take seriously the recommendations made by the team. IASC WG formulates 9 Basic Principles

74th IASC WG agreed to move beyond the pilot phase to regular implementation and requested to develop a set of operating procedures:  Criteria for triggering IA-RTE  A Clear Purpose Statement for IA RTE  A standard ToR & methodlogies IASC WG request to move from pilot phase to regular implementation

IASC IA RTE Interest Group develops an IA RTE Procedures and Methods Guide IA RTE Procedures and Methods Guide is endorsded by the IASC during its WG meeting in July 2010!

How do IA RTEs work?

Who are the main Stakeholders in IA RTEs? How and when to trigger an IA RTE? What is the Evaluation Framework for IA RTEs?

IA RTE Stakeholders

Criteria Lists for Triggering IA RTE Essential IA RTE Trigger Criteria (Automatic) > 1 million people affected? AND Flash Appeal > than $50 million or CAP >than $350 million? Desirable IA RTE Trigger Criteria (By request) IASC WG, SG members, ERC, RC/HC or a majority of HCT members requested IA RTE Inter-agency coordination & management to be examined cover a specific period & a defined geographical area? Opportunities for RTE learning? Capacity exists to conduct an IA RTE?

Sheet I: Automatic Triggering IA RTE: Decision Making Process Who and When 5. ERC informs HC/HCT to elicit feedback & response TD: Does emergency meet automatic trigger criteria? More than 1 million people affected? AND Flash Appeal is > $50 MM ? or CAP Appeal > $ 350 MM? TD: OCHA Focal Point:  Posts results of criteria review on IA RTE website  IA RTE still might be triggered by request if it fulfils Desirable IA RTE Trigger Criteria (Consult Sheet 1) TD OCHA Focal Point:  informs Support Group (SG) of IA RTE launch  requests $ 80, ,000 in Flash Appeal  asks SG to establish an ad hoc Management Group (MG), & optional Advisory Group (AG)  posts criteria review & information on IA RTE web  along with MG, informs ERC of IA RTE TD: OCHA Focal Point (FP): reviews the IA RTE Trigger Criteria TD: Practical arrangements by MG (logistics, travel, arranging optional AG etc) TD: + duration of IA RTE 7. FP requests SG members begin remote monitoring and data collection TD: 0 until consultants deployed 8. FP//MG draft ToR, EoI, & publishes EoI on Relief Web & ALNAP TD FP /MG hires Consultant (s), contracts signed TD: Consultants receive remotely monitored data and start desk review TD: Briefing between consultants & MG face-to-face or via phone TD: End of Mission Debriefing in country TD: Consultants draft report, additional debriefings to MG TD: HC/HCT, Advisory Group, MG, SG, interviewees comment on report TD: Final Report Produced TD: MG: Approves Report Report is shared with SG TD: Report for follow up to HC via ERC TD: +109 ERC: Emergency Relief CoordinatorSG: Support Group MG: Management Group FP: OCHA Focal Point TD: Target Deadline HC: Humanitarian Coordinator HCT: Humanitarian County Team CAP: Consolidated Appeals ProcessAG: Advisory Group KEY YES NO 12. Consultants begin Mission to Country— Interview affected populations and actors TD: +52

IA RTE Framework & Bottom-Up Approach

IA RTE Standard ToR Focus on participatory approach Independent consultants team deployed during the initial response phase with arrival no later than two months after the disaster Assess initial response via desk reviews, field visits, interviews with key stakeholders, especially affected population & provide real time feedback & learning to the HCT Consultants serve as facilitators to encourage & assist field personnel to critically examine operations & to find creative solutions to problems Overarching questions include: What were the main operational results, and the positive & negative outcomes for all segments of the affected population, during each phase? Have appropriate common standards been developed within the coordination systems (globally & for each Cluster) & to what degree have these been met?

In which ways are IA RTEs different from standard evaluations?

IA RTEs are different in the following way: An attempt at system-wide assessment at an early stage Real-Time Learning for the HCT Bottom-Up Approach Specific Trigger Mechanism IA Stakeholders

IA RTEs ARE DIFFERENT FROM STANDARD EVALUATIONS DEFINITION: IA RTEs are ‘Inter-Agency (IA)’ in that the RTEs are instigated, managed and used by a variety of international organizations. These are represented at a global level in the IA RTE Support Group and for a specific IA RTE, through the IA RTE Management Group, the Humanitarian Country Team (HCT) and the in- country IA RTE Advisory Group, when established. The term ‘Real-Time (RT)’ refers to the early implementation stages of a humanitarian emergency operation, to the rapid feedback of IA RTE findings at the field level, and to the possibility of an iterative, multi-phased approach depending on the scale of the disaster. A one-phase approach of an IA RTE would be carried out within the two first month and consists of remote monitoring and the IA RTE mission […]. ‘Evaluation’ (E), as applied in the term IA RTE, describes a participatory review process. IA RTEs differ from other humanitarian evaluations regarding speed, coverage, methods, and outputs.

IA RTEs ARE DIFFERENT FROM STANDARD EVALUATIONS PURPOSE: Intended primarily for learning in emergency operations with field-level ‘ inter- agency coordination and management at the core, especially in the context of sudden- onset disasters, or protracted crises undergoing a phase of rapid deterioration or escalating violence. Defined as an evaluation carried out at the early implementation stages of a humanitarian operation which almost simultaneously feeds back its findings for immediate use by the broader humanitarian community at the field level. Seek to identify gaps, access constraints and potential threats to the “ humanitarian space ”, assess the relevance, quality and timeliness of the response, unlock inter- stakeholder coordination/ collaboration problems or operational bottlenecks, propose appropriate strategic re-orientation and provide real time learning to the field on same. Principle aim is to identify strengths and weaknesses of the response through the assessment of the response to the affected population. Primary end users of IA RTEs are in-country agencies and staff engaged in the humanitarian response, especially those involved in inter-agency coordination and management.

Challenges

Inter-Agency Challenges Funding Coordinating IA Stakeholders IA-RTE in Complex Emergencies Responsibility “Sharing”

Links Inter-agency Real Time Evaluations Inter-agency Standing Committee (IASC) OCHA & Inter-agency Evaluation Reports d/1325/language/en-US/Default.aspx Active Learning Network for Accountability & Performance in Humanitarian Action (ALNAP)

Andreas Schuetz Associate Expert in Humanitarian Affairs Policy Development and Studies Branch PDSB Evaluation and Guidance Section EGS Palais des Nations, Geneva Tel tel fax Claude Hilfiker Senior Evaluation Officer Policy Development and Studies Branch PDSB Evaluation and Guidance Section EGS Palais des Nations, Geneva tel fax Questions?