1. Estimating Impact or rural electrification programs in Ethiopia
Maximo Torero International Food Policy Research Institute
With the collaboration of: Sebastian Galiani (UDESA), and Tanguy Bernard (IFPRI)
Impact evaluation in Ethiopia
Friday, July 20th, 2006

2. Background: Accelerated Electricity Access Expansion
Ethiopia is implementing an ambitious plan to expand the electric system to both support the high demand growth and to extend access to the rural population at a very accelerated pace.
The objective of the UEAP is to electrify rural towns, as well as villages, through extension of the grid. These rural towns/villages range in size from 70 to 4000 households.
Beneficiaries include residential, commercial and industrial customers (such as flour mills, irrigation centers, water pumping, and telecommunications).
The initial objective of the UEAP is to electrify approximately 50% of rural towns over five years, with a long term view to connecting virtually all towns and villages to the grid in a 10-year horizon.

3. Background: Accelerated Electricity Access Expansion
The Access Expansion component would:
Connect about 382 towns to the grid (through substation expansions and installation of sub-transmission lines (typically 33kV)), and installation of distribution systems in the towns (through transformers and LV lines);
Connect customers to the distribution systems in the towns electrified under component 1a (including installation of meters and drop down wires to homes);
Install efficient public street lighting in towns electrified; and
Distribute CFLs (compact fluorescent lamps) to rural customers to foster energy efficiency and therefore affordability.

4. How we can design the impact evaluation for the electricity access expansion program

5. Objective of impact evaluation
The objective of the impact evaluation is to measure the impact that the AEAE Program has on the welfare of the population, and
to investigate whether specific interventions are more effective than others in achieving positive outcomes.

6. Main questions to answer
Questions aimed at understanding the overall impact of AEAE on socioeconomic development, namely:
What is the impact of expanded access and use of electricity on household welfare and local economic development?
What is the impact of introducing energy efficient technology on uses of electricity?
Questions aimed at understanding the relative effectiveness of alternative program features in achieving the objectives of the project.
What is the impact of alternative financing mechanisms on household connection rates?
What is the optimal subsidy to CFLs to maximize CFL usage per dollar spent?
What is the impact of alternative modes of delivery of CFLs (public distribution, private, vouchers)?

7. Distribution of observations by control and treatment groups
Identification of 17 substations
Identification of the towns that fall in the 100k radius
Identification of towns that do not have access to electricity
feasibility study (?)

8. Distribution of observations by control and treatment groups
Identification of 17 substations
Identification of the towns that fall in the 100k radius
Identification of towns that do not have access to electricity
Non parametric matching to control for any additional selection bias as a result of the feasibility study

9. Statistical and Econometric Methodology: Two step matching – Base Scenario
Town A (with access)
Town B (no access)
Town C (no access)
Town D (no access)
Town E (no access)
Town F (no access)
1. TOWN MATCHNG:
Among towns selected with and without electricity, find comparable towns regarding the following criteria: (1) Comply same selection criteria (2) Access to public phone; (3) type of road; (4) secondary school in town; (5) quintile of population; (6) region.
2. HOUSEHOLD MATCHING (PROPENSITY SCORE MATCHING):
Within comparable towns, households with electricity are matched to those without electricity. Propensity score matching (kernel) is performed using the following variables: HH size, demographic composition, ethnicity of HH head (self-reported), age of HH head, education of HH head, sex of HH head, access to water public network, and other possible controls.
Town A (access)
Town B (no access)
Propensity score matching of households among comparable towns
Town F (no access)

10. Proposed Randomized experiments: 1
Proposed Randomized experiments: 1. Randomization on the last mile connection within identified treated and control towns
Objective: to randomize through incentive mechanisms the connection from the mainline of the town to each of the households and business
Instrument to be use: a discount voucher (could be a discount in the loan) to pay the last mile connection.
Procedure:
Between 10-50% of eligible survey respondents will be randomly assign vouchers for a discount charge to cover the connection of the hh or business
Varying amounts of the offer will be distributed in equal proportions among voucher recipients (we are proposing 2 to 3 sizes of vouchers)
Vouchers will have a period of duration (4 months)
In a subsample of towns randomly selected from the pool of the 382 intervened towns no vouchers will be distributed.
Methodologically:
Vouchers will serve as an IV for intensity of electricity access.
We need to assure treatment and control groups are demographically representative within each of the possible treatment amounts.
Difference in difference estimators.

11. Proposed Randomized experiments: 1
Proposed Randomized experiments: 1. Randomization on the last mile connection within identified treated and control towns
Potential problems:
How to define the size of the voucher – could be a WTP battery of questions in the baseline survey.
Local providers must comply with protocol established for the distribution of the vouchers.
How to avoid contagious effect – i.e. households that did not receive vouchers will be expecting them in the future and therefore will not buy the connection – possibility to announce the lottery publicly and clearly point out there is X number of vouchers.
Competitive underground market of vouchers – try to include identity in vouchers and train providers to match id of individuals to vouchers. We will also include random checks by undergraduate research assistants from the local universities.

12. Proposed Randomized experiments: 1
Proposed Randomized experiments: 1. Randomization on the last mile connection within identified treated and control towns
Potential problems:
Problems of non compliance – we will check this by examining whether the level of vouchers distributed within the village predict village-level electricity access but not household use.
Alternatively we can use random assignment as an IV for actual assignment (Duflo and Kremer 2003; Angrist and Krueger, 2001). Although initial assignment does not guarantee that someone actually receives treatment, in most circumstance assignment is nonetheless correlated with participation.
Also refer to local treatment effects paper of Imbens and Angrist (1994), i.e. it is possible to obtain the local average treatment effect estimate by comparing outcomes in the initially assigned group and scaling up the difference by dividing it by the difference in the probability of receiving the treatment in those two groups.
Political feasibility of limited number of vouchers – implement a sequencing in the distribution.

13. Impact Indicators: Impacts of access and better quality of electric service
Change in the proportion of time for activity “i”
Change in the total number of hours worked by the household
Interaction effect

14. Impact Indicators: Impacts of access and better quality of electric service
Total hours of work – household
Hours of work – household (inc. chores)
Hours of work – individual
Hours of work – individual (inc. chores)
% hours of non-ag work household
% hours of non-ag work household (inc. chores)
% hours of non-ag work individual
% hours of non-ag work individual (inc. chores)
All above by gender

15. Impact Indicators: Changes in the quality of the electricity service
Expenditure in electricity (proportion of total energy sources)
Expenditure in electricity (proportion of total expenditure)
Number of failures
Price
Sources of energy

16. Proposed Randomized experiments: 2
Proposed Randomized experiments: 2. Randomization of the provision and distribution of CFLS (Compact fluorescent lamps)
Objective: implement randomized trials to try to answer four sets of questions:
What is the willingness to pay of CFLs by households?
What is the optimal subsidy to CFLs to maximize CFL usage per dollar spent?
What is the impact of alternative modes of delivery of CFLs (public distribution, private, vouchers)?
How arbitration of replaced traditional bulbs by CFLs can be avoided?
Major advantage: this is a pilot so we can even randomized the towns in where to test.

17. Proposed Randomized experiments: 2
Proposed Randomized experiments: 2. Randomization of the provision and distribution of CFLS (Compact fluorescent lamps)
Instrument to be use: a discount voucher (could be a discount in the loan) to pay the last mile connection.
Procedure:
Between 10 to 50% of eligible survey respondents will be randomly assign vouchers for a discount charge to cover the cost of the CFL
Varying amounts of the offer will be distributed in equal proportions among voucher recipients. The quantity of vouchers will vary between towns.
Vouchers will have a period of duration (1 month)
In a subsample of towns we will also include in addition to the vouchers a promotion to buy the old bulbs (the amount will be established on the baseline through a willingness to receive battery of questions)
In a subsample of towns randomly selected for the pilot no vouchers will be distributed.

18. Thanks!