1. Decentralization Thematic Group/Public Sector Group Funding 2002-2003 “Assessment of the Impact of Decentralization: The Case of Colombia (1991-2001)” Presentation by: Jonas Frank (LCSPS) March 10, 2004

2. The Puzzle: What is the impact of decentralization and can this impact be evaluated? Better Services? Economic Efficiency? Democratic Government?  Evaluation is about finding causality

3. The Colombia Case: The Study:  Sample of 148 municipalities, out of a total of 1100  70 indicators  1000 pages  It took ca. 2 years to complete the evaluation Appropriate time-frame: process started in 1986, evaluation covers years following 1991 Integral reform: political, fiscal, administrative Typical sequence: political reform first, then fiscal and administrative decentralization; municipal level

4. Three important questions: Why evaluate? What should be evaluated? How can one evaluate?

5. Objectives of Evaluation differ by actors of decentralization:  Minister of Finance, Indigenous People, Legislators, Mayors, Regional Governors, Donors,... There is no single objective, but evaluations can...  (i) improve decision-making  (ii) create accountability among actors involved I. The “Why” of Decentralization Evaluation

6. Implications:  Evaluation is a continuous exercise  Transformation of data into information  A participatory exercise vs. purely technocratic I. The “Why” of Decentralization Evaluation

7. Objective of the Colombia Study: Crisis: fiscal pressures, weak budget constraints, bailouts Inefficient services Slow democratization Lesson: Evaluate as early as possible Ownership Too many objectives of evaluation I. The “Why” of Decentralization Evaluation

8. Some guiding questions: Where is decentralization supposed to have an impact? Were the initial goals of decentralization met and to what extent? Dilemas: Decentralization goals remain vague They are not expressively formulated and agreed upon Decentralization is a “moving target” I. The “What” of Decentralization Evaluation

9. Decentralization Goals in Colombia:  In 1986: democratization  In 1991: efficiency in services, redistribution, services for the poor, improve popular participation  In 2001: fiscal discipline, efficiency I. The “What” of Decentralization Evaluation

10. Venezuela Ecuador Peru Bolivia Colombia ARG Brazil Rio Grande Chile LAC: Decentralization Objectives (1970 – 2002)

11. Colombia Study: “Comprehensive evaluation” (not only fiscal): 1. Outcomes within eight sectors  Education  Health  Water and basic sanitation  Fiscal performance  Political decentralization  Management capability  Economic development 2. Municipal Progress: (i) sectoral outcome (ii) aggregate outcome, II. The “What” of Decentralization Evaluation

12. 1. Sectoral Evaluation: Results in Education “Decentralization has improved and contributed to:  Increase in coverage  Improved teacher/student ratio  Higher schooling levels  Reduction in illiteracy rate” II. The “What” of Decentralization Evaluation

13. Results in Education

14. Sectoral Evaluation: Results in Education

15. 1. Sectoral Evaluation: Results in Health “Decentralization has improved and contributed to:  Increased coverage  Higher public spending  Greater equity  Lower infant mortality rates  Greater ratio of physicians per inhabitant”. II. The “What” of Decentralization Evaluation

16. Sectoral Evaluation: Results in Health

17. 2. Performance Evaluation of Municipalities (i) Municipal Performance in Six Sectors (Health, Education,...) (ii) Aggregate Performance: Most municipalities have achieved an average performance; only 4% achieved and acceptable standard But:  What were the starting conditions?  Where there several observations in the past and when? II. The “What” of Decentralization Evaluation

18. Results in Economic Development

19. Lessons: Some important areas are excluded:  Allocative efficiency  The regional level: departmental government Conclusions about the causality of decentralization are primarily hypothetical “Less is more” II. The “What” of Decentralization Evaluation

20. 1. Base-line of evaluation 2. Finding causality 3. Indicators 4. Selection of samples of local government 5. Periodicity of evaluation III. The “How” of Decentralization Evaluation

21. 1. Base-line Usually no such base-line has been established before decentralization was implemented   The use of proxies   Select indicators that are available and mirror the starting conditions III. The “How” of Decentralization Evaluation

22. 2. Finding Causality To date, impact studies of decentralization are either: (i) Evidence-based (ii) Subjective (iii) Hypothetical All of these three types of studies provide a useful understanding and appraisal at various stages of the decentralization process. III. The “How” of Decentralization Evaluation

23. 2. Finding Causality The Colombia study is primarily hypothetical Statistical analysis is not sufficient to prove causality Lessons:  Discerning between decisions that are in hands of local government and which are not  Clarifying the value added of local government in carrying out new responsibilities?  Continuous monitoring III. The “How” of Decentralization Evaluation

24. 2. Finding Causality: Example of the Education sector Local government defines curriculum content:   ”Instruction Indicator” Local government is responsible for construction and maintenance of school buildings   “Infrastructure Indicator” Local government develops education improvement plan   “Planning Indicator” Local government allocates budget   “Budget Indicator” Is this operationally possible? III. The “How” of Decentralization Evaluation

25. 3. Indicators 70 indicators were used in the Colombia study Lesson:  Selection of indicators depends on “what” will be used, but:  use only a minimum set of indicators, and...  indicators that can be monitored regularly and with low cost III. The “How” of Decentralization Evaluation

26. 4. Comparability of data and selection of local government Colombia: “representative sample” = most diverse municipalities were selected Lesson:  Use only most similar cases because diversity increases during decentralization process  Split them into groups: eliminate some structural factors that lie outside control of municipalities (population size) III. The “How” of Decentralization Evaluation

27. 5. Periodicity of evaluation Colombia: evaluation after 10 years of time Lesson:  Time period sufficiently large, but...  Regular monitoring is necessary  It is important to separate between short-term, medium-term, and long-term goals  helps create accountability  helps better decision-making III. The “How” of Decentralization Evaluation

28. Conclusion: Risks and Opportunities Evaluation as a purely bureaucratic exercise (i) Evidence and (ii) subjective evaluation are still important: the only “early-warning system” that is operational Ready to propose and implement corrective measures: “exit” of decentralization process? Giving “erroneous” answers to the “wrong” questions: “Decentralization has not worked well enough because there was not enough of it”