Impact Evaluation of the QIDS UP: Learning and Teaching Support Material Department of Education, South Africa AFRICA IMPACT EVALUATION INITIATIVE, AFTRL Africa Program for Education Impact Evaluation

QIDS UP QIDS UP aims to raise school achievement of South African pupils towards international standards by raising primary schools above a set standard of infrastructure and school inputs within the next 5 years in the poorest 40% of learners. QIDS UP will provide –Teaching and learning material –Qualified teachers –Infrastructure –Reduction of class size

Evaluation Questions What is the impact of the provision of the minimum resource package on teaching and learning outcomes in the quintile 1 and 2 schools? What is the additional impact of teacher and management training on utilization of learning material on teaching and learning outcomes?

Indicators Learning achievement: –literacy and numeracy for grade 3 Quality of teaching: –learner perception, –participation, –use of material (exercises from the textbooks, frequency of use of material) Availability of material: –learner textbook ratio per learning program (literacy, numeracy and life skills) Management: –textbook retrieval rate

Evaluation Design (Gauteng: Q1-2; Western Cape Q2, option Q1) Question 1: Regression Discontinuity design 250 participating schools representing 100% of eligible schools –Impact measured as the difference in outcome indicators for schools below (125) and above (125) the 21.9% cutoff. Question 2: Experimental design –Training will be randomly assigned to half of the schools in the program (125 out of the 250 participating schools) to all principals and grade 3 teachers in each school. –The additional impact will be measured as the difference in outcomes between trained schools and not trained schools. (w. double dif)

Sample and Data EXISTING March 2007 Auditing census of School Infrastructure (30,000 schools). Sept 2007 National Grade 3 Assessment. Random sample of schools and learners in the country (1 day in each school). Contextual questionnaires: principals, teachers and student and parents (1 additional day per school). Provincial administrative system/monitoring system of QIDS UP: financial and in kind resources delivered to each school. ADDITIONAL Gauteng. Expand sample to the 375 bottom schools. Test schools in Sept 2007 and Sept 2008 Western Cape. Expand sample to the 144 bottom schools. Test schools in Sept 2007 and Sept 2008 One additional province to be considered

Staffing Plan Team National project manager Provincial project managers Research team (Lead researcher WB, JPAL, local researcher(s), Quality Assurance (DoE)) Data collectors Steering Committee National project manager Provincial project managers Lead researcher Quality Assurance

Timeline Submit Concept Note to DDG and DG for approval. 6 or 13 Jul 2007 Feedback from management10 or 17 Jul 2007 Provide comments on contextual questionnairesJun-07 Consultative meeting with QIDSUP and Provinces End Jul 2007 Data collection (baseline)Sep-07 School trainingOct-Nov-07 QIDS UP roll outStarting Jan 2008 Data collection (follow up)Sep-08 Impact evaluation analysisDec-08

Budget