1. Part I: Impact Evaluation of Rural Roads Dominique van de Walle DECRG, October 2006

2. General belief that roads are good for development & living standards
With limited redistributive instruments, we look to sectors such as rural roads to achieve distributional objectives
General belief that roads are good for development & living standards
But, little is known about impacts

3. What Is Different About Roads?
Impacts are indirect: roads are an intermediate service; impacts depend on interactions w/other investments, household & community factors
Important to control for heterogeneity of other factors
But, impacts can be economy wide: potential controls may have been determined by road investment
Roads are not randomly placed

4. Therefore:
Traditional methods of conducting evaluation may be contaminated by the project, directly or indirectly
Potential for randomization is limited
Need:
baseline and panel
appropriate controls for exogenous time varying factors
i.e. double difference + controls for observed time varying factors

5. Getting at distributional impacts:
On average benefits may be positive
But, there may be both gainers & losers
Short & longer term distributional impacts may be different
So need:
ability to differentiate between welfare groups
sufficiently long post-project follow-up

6. Learning from the Ex-Ante Evaluation
Think about evaluation from day 1 of project
‘Appraisal’ stage can be thought of as ‘ex-ante evaluation’
How are intervention areas picked?
How are road links chosen?
Helps to understand biases in ex-post evaluation

7. Part II: Application to Rural transport project (RTPI) in Vietnam
Rural roads rehabilitation project in 18 provinces of Vietnam,
To link communities with markets and reduce poverty
Rehabilitation of rural roads; no new roads are to be built.
Selection criteria: costs less than $15,000 per km; population served is at least 300 per km.

8. Evaluation Questions
What are the impacts of rural roads on living standards (broadly defined) and their distribution?
What factors influence those outcomes?
How much do benefits depend on other investments (e.g. human capital)?
In what ways do first-round impacts differ from longer term impacts?

9. Survey of Rural Road Impacts in Vietnam (SIRRV)
Pre-project baseline in 1997
Post-project follow-up rounds every two years: 1999, and 2003.
100 “treatment,” 100 “comparison” communes in 6 provinces; randomly chosen
15 households per commune: stratified sampling
Panel of 200 communes & 3000 households. District and project data bases
No welfare indicator; possibility of linking up with VLSS98

10. Risks timing: project delays commune splits
weather: typhoons, flooding etc
waning interest
fungibility: project aid displacement

11. Evaluation methodology
Diff-in-diff with propensity score matching
Propensity-score matching
Using baseline characteristics likely to affect selection into the project, and outcomes
This deals with observable heterogeneity in initial conditions that can influence subsequent changes over time
Diff-in-diff
Difference in outcomes over time between matched project and non-project communes
Purges additive time-invariant unobservables

12. Evaluation methodology
Logit model of program participation on pooled project and non-project samples.
Create predicted values from the logit regression – the propensity scores
Check for common support
Nonparametric kernel matching is used for one estimator
An alternative estimator uses the propensity scores to weight the DD

13. Impacts on rehabilitated road kms?
Impacts on rehabilitated road km were less than intended
However, more roads were built in project areas.
Thus, we find fungibility within the sector, but evidence of a flypaper effect of aid on roads
Spending on rehab + building accords reasonably closely to total amount allocated by the project

14. Other findings so far:
Impacts on market development, availability of goods, household involvement with markets
But, very heterogeneous impacts across communes, depending on initial conditions.