Pablo Ibarrarán & Juan M. Villa Inter-American Development Bank Strategy Development Division Office of Strategic Planning and Development Effectiveness.

Slides:



Advertisements
Similar presentations
ESA/STAT/AC.219/15 Survey Analysis for Gender Indicators Sulekha Patel Development Data Group World Bank Manila October 11, 2010 ESA/STAT/AC.219/15.
Advertisements

Child Care Subsidy Data and Measurement Challenges 1 Study of the Effects of Enhanced Subsidy Eligibility Policies In Illinois Data Collection and Measurement.
The World Bank Human Development Network Spanish Impact Evaluation Fund.
REGRESSION, IV, MATCHING Treatment effect Boualem RABTA Center for World Food Studies (SOW-VU) Vrije Universiteit - Amsterdam.
The Institute for Economic and Social Research University of Indonesia
Explanation of slide: Logos, to show while the audience arrive.
Conditional Cash Transfer penalties Vs. no penalties.
The World Bank Human Development Network Spanish Impact Evaluation Fund.
Benefits and limits of randomization 2.4. Tailoring the evaluation to the question Advantage: answer the specific question well – We design our evaluation.
Performance Based Incentives for Learning in the Mexican Classroom Brian Fuller, MPA, Foundation Escalera Victor Steenbergen, MPA Candidate, London School.
Impact Evaluation: The case of Bogotá’s concession schools Felipe Barrera-Osorio World Bank 1 October 2010.
Making Impact Evaluations Happen World Bank Operational Experience 6 th European Conference on Evaluation of Cohesion Policy 30 November 2009 Warsaw Joost.
Mexico’s Oportunidades: Self- Selection in Targeted Social Programs César Martinelli Professor of Economics, ITAM, Mexico City and Wilson Center/Comexi.
Conditional Cash Transfers for Improving Utilization of Health Services Health Systems Innovation Workshop Abuja, January 25 th -29 th, 2010.
Goal Paper  Improve our understanding on whether business training can improve business practices and firm outcomes (sales, profits, investment) of poor.
TRADUIRE LA RECHERCHE EN ACTION Employment RCTs in France Bruno Crépon.
Background, achievements and challenges in the external evaluation process of Oportunidades November 19 th, 2013.
Preparing for the Final Examination. Course Structure Probability Design Issues and Descriptive Statistics Confidence Intervals and Hypothesis Tests.
THE EFFECT OF INCOME SHOCKS ON CHILD LABOR AND CCTs AS AN INSURANCE MECHANISM FOR SCHOOLING Monica Ospina Universidad EAFIT, Medellin Colombia.
Ariel Fiszbein Chief Economist Human Development World Bank.
Office of Institutional Research, Planning and Assessment January 24, 2011 UNDERSTANDING THE DIAGNOSTIC GUIDE.
Rural Youth and Labor Outlook: Global and Regional Trends Jesica Seacor, JD, MBA Assistant Director ILO Washington Office June 4, 2007.
Do financial management tools improve credit access among disadvantaged sectors? Evidence from the use of an Integrated Platform for Company Management.
Opioid Misuse Prevention Program “OMPP” Strategic Planning Workshop
Health Programme Evaluation by Propensity Score Matching: Accounting for Treatment Intensity and Health Externalities with an Application to Brazil (HEDG.
Off-farm labour participation of farmers and spouses Alessandro Corsi University of Turin.
Measuring Equality of Opportunity in Latin America: a new agenda Washington DC January, 2009 Jaime Saavedra Poverty Reduction and Gender Group Latin America.
Moving to Opportunity in Boston: early results of a randomized mobility experiment Lawrence F. Katz; Jeffrey R. Kling & Jeffrey B. Liebman Presented by.
Comments on “Testing alternative approaches to CCT programs in education: evidence from Colombia” IEN, LACEA, October 2007.
Assessing the Distributional Impact of Social Programs The World Bank Public Expenditure Analysis and Manage Core Course Presented by: Dominique van de.
Discussion of Hujer, R. and S. Thomsen (2006). “How Do Employment Effects of Job Creation Schemes Differ with Respect to the Foregoing Unemployment Duration?”
Comments on: “The Effects of Income Shocks on Child Labor and Conditional Cash Transfer Programs as an Insurance Mechanism for Schooling” by Monica Ospina.
The World Bank Human Development Network Spanish Impact Evaluation Fund.
Permanent effects of economic crises on household welfare: Evidence and projections from Argentina’s downturns Guillermo Cruces Pablo Gluzmann CEDLAS –
HAOMING LIU JINLI ZENG KENAN ERTUNC GENETIC ABILITY AND INTERGENERATIONAL EARNINGS MOBILITY 1.
The OECD experience in using survey-based disability data: An illustration of current possibilities and limitations Washington City Group on Disability.
Beyond surveys: the research frontier moves to the use of administrative data to evaluate R&D grants Oliver Herrmann Ministry of Business, Innovation.
ECON 3039 Labor Economics By Elliott Fan Economics, NTU Elliott Fan: Labor 2015 Fall Lecture 21.
Ministry of Social Development SEDESOL Mexico Design and Evaluation of Social Programmes Gonzalo Hernandez, Gustavo Merino, Ana Santiago, Miguel Szekely.
Handbook on Residential Property Price Indices Chapter 5: Methods Jan de Haan UNECE/ILO Meeting, May 2010.
CHAPTER 12 Descriptive, Program Evaluation, and Advanced Methods.
The Health Consequences of Incarceration Michael Massoglia Penn State University.
MSRP Year 1 (Preliminary) Impact Research for Better Schools RMC Corporation.
Nigeria Impact Evaluation Community of Practice Abuja, Nigeria, April 2, 2014 Measuring Program Impacts Through Randomization David Evans (World Bank)
SAMPLE SELECTION in Earnings Equation Cheti Nicoletti ISER, University of Essex.
Applying impact evaluation tools A hypothetical fertilizer project.
New Measures of Data Utility Mi-Ja Woo National Institute of Statistical Sciences.
Presented by: Shubha Chakravarty (Economist, AFTPM) Impact Evaluation team: Mattias Lundberg (Sr. Economist, HDNCY) Markus Goldstein (Sr. Economist, AFTPM.
A generalized bivariate Bernoulli model with covariate dependence Fan Zhang.
TRADE LIBERALIZATION AND CHILDREN Understanding and coping with children vulnerabilities Javier Escobal Group for the Analysis of Development.
Answers to Review Questions  1.What are the ultimate targets of monetary policy?  The ultimate targets of monetary policy include stable prices, sustainable.
Using Propensity Score Matching in Observational Services Research Neal Wallace, Ph.D. Portland State University February
Implementing an impact evaluation under constraints Emanuela Galasso (DECRG) Prem Learning Week May 2 nd, 2006.
EDUCATION FOCUS GROUP MEETING July Welcome and Introduction CN Goals Metrics Resident Survey Findings Mapping Discussion on Data Collected Discussion.
1 Validating Ex Ante Impact Evaluation Models: An Example from Mexico Francisco H.G. Ferreira Phillippe G. Leite Emmanuel Skoufias The World Bank PREM.
What can a CIE tell us about the origins of negative treatment effects of a training programme Miroslav Štefánik miroslav.stefanik(at)savba.sk INCLUSIVE.
Performance Based Payments to Physicians in Turkish Public Hospitals: Issues in Impact Assessment Burcay Erus and Ozan Hatipoglu Bosphorus University,
Remittances and Human Capital Investment: Evidence from Albania Ermira Hoxha Kalaj December 2010.
Using administrative data to produce official social statistics New Zealand’s experience.
Effects of migration and remittances on poverty and inequality A comparison between Burkina Faso, Kenya, Nigeria, Senegal, South Africa, and Uganda Y.
1 The Training Benefits Program – A Methodological Exposition To: The Research Coordination Committee By: Jonathan Adam Lind Date: 04/01/16.
Assessing the Impact of Informality on Wages in Tanzania: Is There a Penalty for Women? Pablo Suárez Robles (University Paris-Est Créteil) 1.
[Presentation location] [Presentation date] (Confirm ABT logo) Building Bridges and Bonds (B3): An introduction.
IMPACT EVALUATION PBAF 526 Class 5, October 31, 2011.
S By Soazic Elise WANG SONNE & Sabine KOWSKI United Nations University-MERIT 2016 IYAS Conference, Pretoria, South Africa Impact of migration remittances.
How do land rental markets affect household income
Measuring Results and Impact Evaluation: From Promises into Evidence
Disability and Social Safety Nets in Developing Countries
Matching Methods & Propensity Scores
Presentation transcript:

Pablo Ibarrarán & Juan M. Villa Inter-American Development Bank Strategy Development Division Office of Strategic Planning and Development Effectiveness Washington, DC. Tulane University New Orleans, April 2010

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2.Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Written contract. 5.Brief discussion.

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2.Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Written contract. 5.Brief discussion.

Juan M. Villa 1. Introduction: CCTs and labor Markets Recognized effectiveness of CCTs on Human Capital accumulation (increased schooling and access to health services). Oportunidades: rigorously evaluated since random assignment design in Progressively Control localities became Treatment (from control to comparison groups). Main objective: to stop the intergenerational transition of poverty. How? Income matters, Labor Markets matter. CCTs and Labor Markets: widely studied but focused on adults or short run analysis. e.g. Attanasio et al (2004): Familias en Accion.

Juan M. Villa 1. Introduction: CCTs and labor Markets Human capital is not sufficient. A general equilibrium issue. In 2007, when latest data was collected and available, Oportunidades’ operation completed 10 and 7 years in rural and urban areas, respectively. How long should a recipient receive the CCT? How long it is optimum for achieving ‘good labor market’ outcomes? We explore the treatment of the program as a continuous “exposure to the treatment”

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2. Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Monthly worked hours. - Written contract and health insurance. 5.Brief discussion.

Juan M. Villa 2. Previous intensity assessments Rodriguez and Freije (2008) assessed the effect of OPORTUNIDADES on labor mobility and earnings of youngsters belonging to the program. After ten years of operations, they defined three types of localities according to their duration in the program: localities with 6 or more years; localities within 3 and 6 years and localities with less than three years. With these three indicator variables they modeled the differential exposure to the program. According to the programs guidelines, it is required to “(…) update and evaluate the socioeconomic characteristics of beneficiaries families in the registry that complete three years receiving the benefits from the program, and hence, deciding their membership…”

Juan M. Villa 2. Previous intensity assessments Their idea of considering the effect of the program in a short, middle and long term was accomplished by running a linear regression of each labor outcome specifying a dummy variable for the treatment and for each “exposition to the treatment”. THEY DID NOT FIND ANY DIRECT IMPACT OF THE PROGRAM ON LABOR INSERTION VARIABLES. The control group was too small due to the expansion of the program into the localities that were identified as controls in Besides, it was pointed out that people in the ENCEL 2007 were still living in such localities with high levels of poverty, limited labor opportunities, and where the program does not have any inference on the labor market’s demand (the general equilibrium problem).

Juan M. Villa 2. Previous intensity assessments Interestingly, as part of the qualitative studies of the long-term impact of Oportunidades, Gonzalez (2006) describes that in several communities those that had achieved higher schooling migrate either to large urban areas or to the United States. Another limitation of the paper acknowledged by the authors is that they imputed the duration of each youngster in the program as the maximum duration of the household based on the locality they live, regardless the date of enrollment in each particular case. Lack of administrative information.

Juan M. Villa 2. Previous intensity assessments Recipients were asked how long have they been in the Program but about 30% of the households that do not know / do not remember when they were enrolled. In absence of administrative data the estimates with this individual variable would yield biased results. It is more likely that early recipients do not remember when they were enrolled.

Juan M. Villa 2. Previous intensity assessments If we impute the duration of the locality to every recipient, we ignore its internal variation. For instance, the locality with the highest average duration of its beneficiaries in the program has 7.8 years and a standard deviation of 1.6 years. There are beneficiaries with 5 years and other with 9.8 years in the program. But the use of the declared duration from participants brings up three relevant sources of bias. - The first one is originated by the participation itself. - The recall-bias originated in the self-declaration of the beneficiaries. - The bias generated by those respondents that do not have idea about the date they were signed up into the program.

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2. Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Written contract. 5.Brief discussion.

Juan M. Villa 3. The Generalized Propensity Score-GPS. The biases generated by the use of the information on participants are eliminated by using the Generalized Propensity Score - GPS following Hirano and Imbens (2004). The authors deal with the results of a lottery with (obviously) random assignment and a continuous participation-treatment variable (the prize amount). Nonetheless, they had no information on about the 50% of the prize winners, and they lack a control group. Similarly in our case, despite the random coverage of the ENCEL 2007 for participants located in rural areas, there is no information about the duration of the 30% of them.

Juan M. Villa 3. The Generalized Propensity Score-GPS. Therefore, the use of the GPS gives us the opportunity to use the “exposition to the treatment” in terms of years of participation in the program as a continuous participation variable (not the usual binary choice). Hence, we are able to answer what is the optimum duration of recipients in OPORTUNIDADES according to youngsters’ labor insertion. The first step on the estimation of the GPS is to obtain a maximum likelihood estimation of the treatment given pretreatment covariates :

Juan M. Villa 3. The Generalized Propensity Score-GPS. Second, a mathematical transformation that yields the GPS, using the previous estimates: Then, it is possible to estimate the Dose-Response Function in order to obtain the effect of the “exposure to the treatment” through the next quadratic equation:

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2. Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Written contract. 5.Brief discussion.

Juan M. Villa 4. Data and results. The survey ENCEL 2007 was carried out in year in 43,946 households located in 219 localities of 13 Mexican states. Our key variable, exposition to the treatment of youngsters between 14 and 24 years is obtained by the declaration of beneficiaries who specified the month and year they were signed up.

Juan M. Villa 4. Data and results. Normality assumption. Transformation of treatment variable: The transformation of the exposure to the treatment relative to the age, besides giving a better goodness of fit for the normality assumption provides an opportunity to enhance our analysis.

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2. Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Written contract. 5.Brief discussion.

Juan M. Villa 4. Data and results. Years of education (0.28):

Juan M. Villa 4. Data and results. School attendance years in the program per year of age makes a 30% higher the probability of school attendance. A key issue in this finding is the years of age corresponding to this maximum level that are revealed in the following table:

Juan M. Villa 4. Data and results. Probability of being employed (0.21):

Juan M. Villa 4. Data and results. Probability of being employed:

Juan M. Villa 4. Data and results. Probability of having a written contract

Juan M. Villa CONTENT 1.Introduction: CCTs and Labor Markets. 2. Previous intensity assessments. 3.The Generalized Propensity Score-GPS. 4.Data and results. - Years of education. - Probability of being employed. - Written contract. 5.Brief discussion.

Juan M. Villa 5. Brief discussion. We have briefly addressed the potential effectiveness of OPORTUNIDADES on a continuous treatment scheme. The availability of labor information on those youths who have migrated from rural to urban areas is essential to understand the potential labor outcomes of the program. This has been the first attempt to determine the optimum duration of a Conditional Cash Transfer program according to labor outcomes. Labor supply – Labor demand. The core variables of the program’s objective (education, health or nutrition) are easily understandable in a continuous treatment framework. However, the participation on labor markets is essential for income generation and, therefore, poverty overcoming.

Juan M. Villa THANK YOU!