1. SEDA IMPACT EVALUATION WESTERN CAPE (SOUTH AFRICA) Varsha Harinath (the dti) Francisco Campos (World Bank) Finance and Private Sector Development IE Workshop Dakar, Senegal February 2010

2. Content Background Objective of the impact study Impact Evaluation methodology Process and timelines Challenges and lessons Next steps Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

3. South Africa background Unemployment is high: 19% for males 25% for females South Africa has identified the development of small, micro and medium enterprises (SMMEs) as means to mitigate the unemployment crisis in the country, as well as to ensure a more equitable distribution of income (Department of Trade and Industry, 1995). Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

4. SEDA Small Enterprise Development Services The Small Enterprise Development Agency (seda) offers non-financial business development services to small businesses in South Africa. The agency, overseen by the Department of Trade and Industry (the dti), was created in 2004 and operates in all 9 provinces of South Africa. Business assessments, training as well as assistance with business plans, promotional material, technology systems, registering process are among a range of non-financial services that seda offers to clients. Eighty percent of its support to enterprises is focused on businesses with less than 20 employees. Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

5. Objective of the impact evaluation The impact evaluation seeks to test if seda services in the Western Cape Province lead to improved business knowledge and practices by SMMEs and hence have a positive impact on growth, competitiveness, sustainability and formality. Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

6. Key outcomes Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 OutcomesIndicators 1. Growth 1a. Increase in turnover 1b. Increase in number of employees 2. Competitiveness2a. Increase in productivity 2b. Increase in net profits 2c. Improvement in operational margins 2d. Increase in number of customers 3. Sustainability 3a. Survival rate 4. Formality4a. Registered 4b. Certification of tax payment

7. Methodology To measure the impact of seda services, we would like to observe the performance of the same enterprise with seda’s intervention and without seda’s intervention over the same period in time………. Both states of the world do not exist at the same time! Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

8. Methodology We “construct” these states of the world by creating an “experiment”, whereby we randomly assign a representative group of small business to either: A treatment group - those that receive seda’s services. A control group - those that do not receive seda’s services. The treatment and control group is obtained from a representative sample of formal and informal small businesses in the Western Cape Province of South Africa. To date, there is no extensive list of formal and informal small businesses in the Western Cape. Hence, for this study a multi-stage sampling technique was used, including the sampling of households and the identification of small enterprises within those households. Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

9. Methodolody Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Populations of SMMEs SMME Sample (906 businesses) Control group (453 businesses) Treatment group (453 businesses) Average outcome for treatment group Average outcome for control group Impact of seda’s intervention __

10. Process 1. Sample identification - ~9,500 sampled households surveyed to identify SMMEs owned by household members 2. Baseline survey - ~900 SMME owners surveyed on business characteristics, financial accounts, support from government, registration status 3. Intervention - seda contacts the businesses in the treatment group to complete an office visit, assess their needs and provide appropriate services. 4. Follow-up survey - information collected on outcomes for treatment and control group 12 months after the intervention 5. Analysis - impacts measured with difference in differences Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

11. Timelines Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 2008-200920102011 Set ObjectivesRoll out of interventionFollow-Up survey Design MethodologyBaseline survey reportsImpact evaluation final report Budget and Implementation Plan Design Questionnaire Sampling Baseline survey Roll out of intervention

12. Baseline reports Baseline data can be useful for decision-making by assessing the status of entrepreneurship, as well as differences between sub-groups and their corresponding needs Growth Series: Profile of businesses Profile of entrepreneurs Awareness of SEDA Informality Access to finance Performance (financials) Labor Markets Challenges to growth Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

13. Baseline data Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 VariableTreatmentControlDifference Male business owners (%)53.0% 0pp Non-white business owners (%)80.6%79.7%0.9pp High-school business owners (%)50.0%52.5%-2.5pp Average age business owner40.940.20.7 Average number of employees5.25.10.1 Wholesale & retail (%)54.1%53.4%0.7pp Unregistered (%)72.9%72.1%0.8pp Number of observations453 0

14. Challenges and lessons Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 ChallengesLessons First rigorous IE study for the dti: buy-in for a large study is difficult Political head lobbying for IE fast tracks the process of buy-in IE can at first glance look expensive (especially if multi-stage sampling is done) Multi-stage sampling improves external validity and provides rich data at the household and enterprise level Pilot encouragement design, inviting treatment group to an event by seda at cinemas in the Western Cape yielded low turn-out rates and even lower take up rates of seda services Significant brainstorming with programme managers & marketing teams is crucial and flexibility when implementing an encouragement may be necessary. Encouragement can be a challenge, especially when invitees are dispersed geographically. Randomised outreach intervention techniques are also usually more successful

15. Next steps Office visit intervention for the Western Cape to be completed by March 2010. Following the interest of this study in the Western Cape, the dti was also requested by seda to do an impact evaluation for the agency on a national scale. The baseline data for national seda study is piggybacking of a national small business study that is presently being conducted for South Africa. Follow–up survey for Western Cape and for the National study in 2011. Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

16. SEDA Impact Evaluation Finance and Private Sector Development IE Workshop Dakar, Senegal February 2010