1. Assessing Public Expenditures on Social Protection: Some Methodological Suggestions Kathy Lindert, World Bank Qualidade do Gasto Publico no Brasil June 26-27, 2003

2. Outline: Key Questions Sector-Wide View What are the goals of the sector? Where does the money come from (financing)? How much is spent? Where does the money go? (composition of spending) Basic inventory Mapping against key vulnerable groups Gaps, duplications, horizontal inequities across programs Program View (what do you get for the money?) Institutional aspects Performance indicators

3. Sector-Wide View

4. Goals of the Social Protection “ Sector ” SOCIAL ASSISTANCE - To provide support to the extreme poor (structural poor) - To help households cope with shocks (transient poor) - To reduce inequality (redistributive goals) - To reduce future poverty via human capital conditions (CCT) - To provide social services to particularly vulnerable groups (Generally in form of non- contributory transfers – cash or in-kind; conditional or not) SOCIAL INSURANCE - To help individuals, households mitigate the adverse effects of risks and shocks, such as: old age, disability, health shocks, unemployment, etc. (Generally in form of contributory payments) To assist individuals, households, and communities to better manage risks, and to provide support to the critically poor

5. Goals, continued Goals depend largely on profile of poverty, risk and vulnerability: Profile of Poverty: Poverty headcount, gap, trends Characteristics of the poor Degree of chronic vs. transient poverty (what share are structurally poor vs. poor due to shocks?) Profile of risks and vulnerability Sources of risk and vulnerability Specific vulnerable groups (e.g., disabled, street children, child laborers, indigenous, etc.) Goals also depend on the extent to which private transfers, insurance are available and accessible to the poor (formal, informal)

6. Where do funds come from? (sources of financing) General Revenues Other sources (mainly for social insurance): Payroll taxes/contributions Employer/employee contributions Self-employed contributions Penalties from employers Interest income Other – fees for service Other sources (not always counted): Donors, NGOs Counterpart contributions

7. Financing Issues Earmarked or not (e.g., fondo de pobreza) Federal, state, local Open entitlement vs. fixed budget allocation Reliability, pro-cyclical or counter- cyclical

8. How much is Spent? (1) Not an easy question to answer Spending spread across multiple programs, multiple agencies, and multiple levels of government Need an inventory of programs & spending: At federal level (across agencies) At state/municipal level: Probably can ’ t do complete inventory for all states/municipalities Select sample (some with high capacity/spending, some with low capacity/spending) to gauge overlaps, complementarities with federal spending

9. ...How much is spent? (2) With inventory of spending on each of the (main) programs, can estimate how much is spent on SP Calculate as % of GDP, total public spending, total social spending

10. Composition of Spending (1): Basic inventory of programs Social insurance: Pensions (general old age, survivor, disability, civil service) Unemployment insurance [Health insurance] Social assistance: Cash transfers Conditional (linked to health, education sectoral goals) Non-conditional (pure cash transfers) In-kind transfers (food – such as school feeding, other) Subsidies (food, energy, agricultural, housing) Workfare (transfers in exchange for public works) Active labor market programs

11. Composition of Spending (1): Basic inventory of programs: Mexico Example

12. Composition of Spending (2): Mapped Against Key Vulnerable Groups One quick and practical way to analyze the mix of programs is to map spending on the main programs against key vulnerable groups using a “ life-cycle ” approach Advantages of this “ life-cycle ” mapping: Easy, quick overview of where funding goes (main groups) Can identify possible overlaps, spending biases (e.g., in favor of elderly rather than youths), and possible gaps – for broad groups Some disadvantages: Not sure of actual gaps, duplications at household level Life-cycle approach mainly focuses on individual risks, ignores that these individuals are actually part of households

13. Composition of Spending (2, cont ’ d): Identifying Vulnerable Groups: Mexico Example (Highlighted areas signal prominent at-risk groups)

14. Mexico Vulnerable Groups …

15. The life-cycle faces its limits and so we move beyond it: Social risk among specific population groups in Mexico

16. Taking the next step: mapping programs onto risk groups Incidence of programs targeted to key social risks by decile and region

17. Composition of Spending (3, cont ’ d) Gaps, duplications, horizontal inequities across programs Another important aspect in analyzing the mix of social programs is the extent to which they incur duplications or gaps in coverage, which result in horizontal inequities Examine which households receive: No benefits (if poor, a gap) Benefits from one program Benefits from multiple programs

18. Composition of Spending (3, cont ’ d) Gaps, duplications, horizontal inequities across programs Data needed: Representative household survey data with comprehensive listing of main (nation-wide) programs Such data are not regularly collected in Brazil PNAD do not include comprehensive list of main social programs POF survey will yield some such data for first time since 1996... an important opportunity for policy feedback

19. Composition of Spending (3, cont ’ d) Gaps, duplications, horizontal inequities across programs, example

20. How Adequate?... A Judgment Call Judgments: Does the program mix have the appropriate blend of social assistance and social insurance? Appropriate mix of public – private provision/financing? Given the country ’ s profile of poverty and vulnerability, does the program mix provide an adequate balance of efforts to assist: The chronic poor (structural poor) The transient poor (due to shocks) Special vulnerable groups (young children, youths, disabled, etc.) Of formal/informal sectors? Are there big gaps in intervention? Significant overlaps, duplication, fragmentation? Is the overall level of effort sensible? Too high? too low?

21. Program Analysis: What do you get for the money?

22. Indicators to Evaluate (for each of main programs) Institutional Aspects: Objectives Institutional Arrangements & Delivery Mechanisms Sustainability Performance Indicators (What do you get for the spending?) Adequacy (Coverage, benefit levels) Equity Efficiency Impact (poverty, inequality, human capital, etc.)

23. Institutional Aspects Objectives of Program: Ideally should evaluate program against these Often, programs have multiple objectives Institutional Arrangements & Delivery Mechanisms: Agencies responsible for design, implementation Administrative structures Resources and systems (adequacy) Incentive structures Targeting mechanisms Delivery of benefits Sustainability: Is the burden on the budget sustainable? How would predicted demographic, poverty or fiscal changes affect this?

24. Adequacy of Programs Coverage Who benefits from the spending? disaggregated as relevant: urban/rural, poverty groups, region, formal/informal Adequacy of benefit level What is the average transfer? Benchmarks vary by program, e.g.,: Average pensions compared to average wages Unemployment insurance to average wages Social assistance to poverty line, etc. Data sources: Institutional data Household survey data (coverage)

25. Adequacy: Coverage (of poor)

26. Adequacy: Benefit Levels

27. Equity Examine: Who receives how much? (distributional incidence of benefits received across deciles/quintiles) Errors of exclusion, inclusion By welfare group (decile, quintile) Also by other pertinent groups (urban/rural, informal/formal, gender, race etc.) Data Needed: Nationally representative household survey data that includes questions on receipt of program benefits (and how much received) Such data are currently weak in Brazil (PNAD don ’ t include social programs)... POF opportunity

28. Equity: Distributional Incidence

29. Overall Effectiveness: Coverage, benefits, equity

30. Efficiency Specific indicators vary by program. Some examples: Social assistance: administrative costs Unit costs: how compare with international practice or local benchmarks? Pensions: Effective rate of return All programs: does intended budget reach beneficiaries or are there indications of resources being siphoned off for unintended uses? Impact on labor markets (discourage work?)

31. Impact Impact on relevant outcomes: Changes in poverty, inequality Changes in employment Human capital outcomes (e.g., do more kids attend school due to conditional transfer?) etc. Numerous methodologies for assessing these Simulations using household survey data Simple simulations (given transfer amount received) Simulations taking into account behavioral effects Ex post impact evaluations (with/without; before/after; control/treatment groups)

32. Conclusions Important to look both at individual programs Efficiency, effectiveness, impact But also at spending across programs in sector Gaps, duplications, fragmentation Appropriate mix or major biases Multiple providers: Levels of government (federal, state, local) Various agencies / ministries Uses of such analyses: Management and planning feedback Possibly suggestive of needed overhaul, integration, rebalancing of safety net