Presentation is loading. Please wait.

Presentation is loading. Please wait.

Evaluating Humanitarian Aid: November 2013 Harry Shannon McMaster University.

Similar presentations


Presentation on theme: "Evaluating Humanitarian Aid: November 2013 Harry Shannon McMaster University."— Presentation transcript:

1 Evaluating Humanitarian Aid: November 2013 Harry Shannon McMaster University

2 Working with David Sanderson Director, Centre for Development and Emergency Practice (CENDEP), Oxford Brookes University Charles Parrack, CENDEP

3 Outline of talk What is (humanitarian) aid Issues in evaluation of aid Shelter after disaster Proposed research agenda for evaluation Discussion Work in Progress!

4 Humanitarian imperative Helping fellow human beings in (desperate) need “How we think about each other, our relationships, our place in the ecosystem … are the most basic questions … We must recognise humility; trying and doing; governance … given them, don’t wait for the perfect answer.” – James Orbinski, former President of MSF

5 Maimonides: hierarchy of charitable giving 1. The lowest: Giving begrudgingly and making the recipient feel disgraced or embarrassed. 2. Giving cheerfully but giving too little. 3. Giving cheerfully and adequately but only after being asked. 4. Giving before being asked. 5. Giving when you do not know who is the individual benefiting, but the recipient knows your identity. 6. Giving when you know who is the individual benefiting, but the recipient does not know your identity. 7. Giving when neither the donor nor the recipient is aware of the other's identity. 8. The Highest: Giving money, a loan, your time or whatever else it takes to enable an individual to be self-reliant. charitywatch.org

6 Definitions (though boundaries fuzzy) (Overseas Development) Aid: flows to developing countries and institutions whose aim is to promote development and welfare of the recipient countries. Humanitarian aid/action (HA): aid that follows a disaster (complex emergency), typically on a more urgent basis than regular ODA Contrast with … Humanitarian intervention: the use of military force by a state to end human rights violations by a state (R2P)

7 Millennium Development Goals, 2000 By 2015: 1.Eradicating extreme poverty and hunger 2.Achieving universal primary education 3.Promoting gender equality and empowering women 4.Reducing child mortality rates 5.Improving maternal health 6.Combating HIV/AIDS, malaria, and other diseases 7.Ensuring environmental sustainability 8.Developing a global partnership for development

8 Millennium Development Goals Specific targets set, e.g., between 1990 and 2015, halve the proportion of people living on less than $1.25 Ban Ki Moon: ‘The Millennium Development Goals (MDGs) have been the most successful global anti- poverty push in history.’ However, progress has been uneven Being replaced in 2015 by Sustainable Development Goals

9 Development aid $140 billion / year Dambisa Moyo, William Easterly - sceptics Jeffrey Sachs – proponent Paul Collier - ?intermediate Esther Duflo, Abhijit Banerjee – RCTs (Randomized Controlled Trials) All support humanitarian aid (HA)

10 Duncan Green is Senior Strategic Adviser at Oxfam GB

11 Duncan Green, From Poverty to Power

12 Sceptical view of Canada’s NGOs

13 Humanitarian aid ‘system’ - some numbers Estimated 4,400 NGOs worldwide doing some HA (Haiti sometimes called a ‘Republic of NGOs’) 2010: estimated 274,000 humanitarian workers worldwide Approx $14 billion / year

14 Humanitarian aid ‘system’ UN humanitarian agencies International Movement of the Red Cross and Red Crescent Five largest International NGOs Médecins Sans Frontières (MSF) Catholic Relief Services (CRS) Oxfam International International Save the Children alliance World Vision International Increasing attention to ‘accountability’

15 Cluster approach – 2005 Inter-Agency Standing Committee (IASC)

16 CODE OF CONDUCT for the International Red Cross and Red Crescent Movement and NGOs doing humanitarian aid activities 1)The humanitarian imperative comes first. 2)Aid is given regardless of the race, creed or nationality of the recipients and without adverse distinction of any kind. Aid priorities are calculated on the basis of need alone. 3)Aid will not be used to further a particular political or religious standpoint. (Independence) 4)We shall endeavour not to act as instruments of government foreign policy. (Neutrality) 5)We shall respect culture and custom. 6)We shall attempt to build disaster response on local capacities.

17 7)Ways shall be found to involve programme beneficiaries in the management of relief programmes. 8) Relief aid must strive to reduce future vulnerabilities to disaster as well as meeting basic needs. 9)We hold ourselves accountable to both those we seek to assist and those from whom we accept resources. 10)In our information, publicity and advertising activities, we shall recognise disaster victims as dignified human beings, not hopeless objects. Emphases added CODE OF CONDUCT for the International Red Cross and Red Crescent Movement and NGOs doing humanitarian aid activities

18 What should humanitarian groups aspire to?

19 Some ethical dilemmas Do we respond in Japan, where it’s expensive, or in the Congo where we can do more for less? If we work with, e.g., the Syrian government, do we risk being co-opted into their strategy? If we bring food, do we drive down market prices and upset local agriculture? If we give out food at distribution points, do we put people at risk of attack when they walk home? Ctd …

20 What is the balance between paternalism and autonomy? Do we speak out about terrible things (only) we see, or stay quiet, do our work and save lives, and let others speak out? Relations with military? These decisions are made under (severe) time pressure, and possibly armed force and persuasion/coercion. Some ethical dilemmas (ctd) Source: Hugo Slim

21 The Guardian, 28 August 2013

22 On 14 August, Médecins Sans Frontières (MSF) announced that, after 22 years, it was closing all its activities in Somalia as “the situation in the country has created an untenable imbalance between the risks and compromises our staff must make, and our ability to provide assistance to the Somali people.” … Of the 160 aid workers killed, 82 wounded and 99 kidnapped in Somalia since 1997, the majority have taken place since 2007 (106 killed, 53 wounded and 57 kidnapped), when the overall security environment deteriorated for both humanitarians and Somali civilians. The Guardian, 28 August 2013

23 Report on aid worker security October 2013

24 Some practical concerns with HA Little argument about the principle Questions about how well HA is provided Speed of response Coverage of those in need Duplication Unnecessary help Doing more harm than good (cholera in Haiti)

25 Critique of humanitarian aid Use of HA for political purposes has been going on for a long time British soldiers in Afghanistan – giving food in exchange for information on Taliban hiding places Colin Powell to NGOs: You are our force multipliers In Darfur, aid workers have provided actionable information for ICC Secret services infiltrate NGOs Embassies keep an eye on what ‘their’ NGOs are doing Source: Antonio Donini

26 Other possible problems with aid NGOs attract talent from local governments Effect of internationals on local prices NGOs supplant government roles Plethora of NGOs; potential competition Government corruption Qualifications to provide help (Non) Involvement in politics Promised money may not arrive

27

28 Port-au-Prince, October 2011

29 Humanitarian/development aid or conscience money? Amira Hass in Israeli newspaper Haaretz, 6 Feb 2013

30 7 Golden Rules of Recovery Management 1.Trust survivors and avoid paternalism 2.Trust survivors and enable survivors to assess their own needs 3.Trust survivors and provide cash rather than kind 4.Think locally 5.Give priority attention to vulnerable groups 6.Think ‘process’ not product, ‘sheltering’ not shelters, ‘housing’ not houses 7.Adopt a long-term perspective while addressing short-term needs Ian Davis + Anshu Sharma, 14 Nov 2013

31 Toby Ord: “Aid Works (on Average)” Talk in Oxford, 13 February 2013 Cited Moyo: – Over 60 years, more than US$1 trillion in aid and not much good to show for it – $1 trillion is nearly $1,000 per person on the planet today Ord: – Nonsense, actually 1 billion people in Africa – $1 tn / 440 mn (average number of people in Sub-Saharan Africa over period) = $2,263 per person = $38 per person per year

32 Toby Ord: “Aid Works (on Average)” (ctd) Successes include: Smallpox eradicationLives saved since eradication: million Oral rehydration therapy for diarrheal diseases Annual deaths down from 4.6 million to 1.6 million ImmunizationMeasles annual deaths down from 5 million to 1.2 million MalariaAnnual deaths down from 3.8 million to 0.7 million

33 Why evaluating ‘aid’ is difficult Reason aid is given Reverse causation Non-developmental aid How/where aid is actually spent Fungibility Multiple donors Remittances Unpredictability of aid Short- vs. long-term impact of projects

34 Aid or …? The Guardian, 21 February 2013

35

36 Critique of methods in evaluating aid Stuckler, McKee, Basu 2012 Reviews are narrative not systematic Evaluations don’t consider bias, confounding Multiple outcome measures Time lags Data limitations Power, Type II errors They advocate: Experimentation Evaluation Replication

37 Process evaluation: how the intervention was done Impact evaluation: longer-term, intended or unintended Indicator – measure of the effect Evaluating aid - Definitions

38 Experiments Small-scale RCTs Examples: – paying parents if child attends school regularly – giving food when children vaccinated – microcredit

39 Increasing attention to accountability ALNAP - Active Learning Network for Accountability and Performance in humanitarian action Jim Kim, President of World Bank: “Science of delivery [of aid]” Various documents produced by different NGOs on how to evaluate

40 Evaluating the humanitarian system ALNAP Collates evaluations and produces summary reports State of the Humanitarian System: ‘assesses progress and performance in six areas: coverage/sufficiency, relevance/appropriateness, effectiveness, connectedness, efficiency and coherence.’ Interactive online evaluation training materials

41 Difficulties in evaluating HA impact No baseline / control (or poor data) Need for rapid information Insecure / volatile environments Surveys in settings with little information on the target population Impact of single intervention vs broader impact of programme Lack of capacity

42 Qualitative and quantitative methods “… much of the guidance for the evaluation of humanitarian action suggests that evaluations are more likely to provide robust evidence where they use ‘mixed methods’ approaches.” “However, in practice, humanitarian evaluation ‘uses mainly qualitative methods’ (Buchanan-Smith and Cosgrave 2012).” “… evaluations still tend to undervalue the experience of affected populations as a source of evidence …” James Darcy and Paul Knox Clarke, 2013

43 Focus on Shelter Seen as human right One of Clusters – incorporates settlement and non- food items Housing used for purposes other than purely shelter Relatively little on shelter evaluation ‘Providing adequate shelter is one of the most intractable problems in international humanitarian response’ ‘(Humanitarian Emergency Response Review, 2011) ‘Building Back Better’

44 Housing and health Shaw, Ann Rev Pub Health, 2004

45 Right to adequate housing Article 25 of Universal Declaration of Human Rights Adequacy includes – legal security of tenure – availability of services – affordability – habitability – accessibility – location – cultural adequacy UNHCR, 1991

46 Section in Sphere Handbook

47 Proposed timing of shelter after disaster Emergency shelter Temporary / transitional shelter Permanent shelter / reconstruction 0-2 weeks 2 years + 2 weeks – 2 years Charles Parrack Relief Recovery Reconstruction

48 Shelter damage in Port-au-Prince after 2010 Haiti earthquake Damage to housing%95% CI Completely destroyed Partly damaged No visible damage Kolbe, Hutson, Shannon et al., Medicine, Conflict and Survival, 2010

49 Shelter for displaced - refugees

50 Proposed agenda for evaluating shelter after disaster Step 1a: Systematically identify indicators Step 1b: Improve list by consulting past and potential beneficiaries Step 2: Develop (short) questionnaire Step 3: Conduct longitudinal study (RCT?) after disaster

51 Housing trade-off: durability vs comfort © David Sanderson; with permission

52 Step 3: Conduct longitudinal study/RCT after disaster Timing: – start 1-2 weeks after disaster – follow up at 6 months, 1 year and 2 years Allocate areas/communities/villages to different approaches to housing, i.e., cluster allocation/randomization E.g., ‘T-shelters’ vs. housing in camps vs. building materials and tools Mixed methods – quantitative and qualitative

53 Feasibility of RCTs?

54 Example of randomization at the individual level

55 Technical problems with RCTs include the need to deliver the treatment in a standardised way, the difficulties of maintaining stable membership in the treatment and control groups, and the expense and effort needed. These constraints led a review on evaluations of health promotion by the World Health Organization in Europe to find that the use of RCTs for this was ‘in most cases, inappropriate, misleading and unnecessarily expensive’ (WHO Regional Office for Europe, 1998, p. 5). RCTs also require that the evaluation be designed from the start of the project with random selection of control and treatment groups. For all of these reasons, RCTs are rare in humanitarian evaluation. (emphasis added) ALNAP EHA Pilot Guide p.113

56 Example of randomization at the community (cluster) level

57 Discussion


Download ppt "Evaluating Humanitarian Aid: November 2013 Harry Shannon McMaster University."

Similar presentations


Ads by Google