The impact of the introduction of Grade R on learning outcomes BRIDGE’s National Early Childhood Development Community of Practice 14 August 2014

Why evaluate? “Development programs and policies are typically designed to change outcomes, for example, to raise incomes, to improve learning, or to reduce illness. Whether or not these changes are actually achieved is a crucial public policy question but one that is not often examined. More commonly, program managers and policy makers focus on controlling and measuring the inputs and immediate outputs of a program—how much money is spent, how many textbooks are distributed—rather than on assessing whether programs have achieved their intended goals of improving well-being.” (World Bank) Accountability on the use of public funds Measure of progress and policy relevance Increasing demand for evidence-based policy and governance decisions

Research & evaluation system Monitoring & Evaluation units in government according to governments mandate of oversight –Overall in delivery of government programmes especially in concurrent functions like education –In review of current interventions or policies in key areas Accountability for donor-funded programmes National Evaluation Policy Framework through the Department of Planning, Monitoring and Evaluation –Government wide evaluation system focusing on priority areas –An attempt to provide political principles and managers in government with reliable information on service delivery “If we are to improve our performance we have to reflect on what we are doing” –Collaboration between Departments –Reports to cabinet The decision to conduct and release this evaluation therefore represents a substantial commitment on the part of government – including the President and the Department of Basic Education – to transparency and to the improvement of service delivery.

Grade R evaluation background In 2002 less than 40% of 5-year-olds in South Africa were attending an educational institution. By 2011, this figure had risen to over 80% and it is still rising. Between 2001 and 2012 the numbers enrolled in Grade R programmes at ordinary schools increased more than threefold, from to This dramatic improvement in access to educational opportunities for young children was largely thanks to the roll-out and rapid expansion of the Grade R programme, initiated by national government. The Theory of Change behind Grade R is that ECD lays a foundation for cognitive functioning, behavioural, social and self-regulatory capacities, and physical health. It is better and cheaper to intervene early The impact evaluation was commissioned in December 2012 by DPME in collaboration with the DBE (NEP Framework)

Evaluation methodology The impact evaluation of the Grade R programme was conducted by independent researchers at the University of Stellenbosch, and the full reports are now available on the DPME website. The evaluation included an extensive literature review describing the evidence on early educational interventions from around the world. This review highlights the critical importance of the early years of a child’s life for cognitive, behavioural and social development. The literature shows that early educational interventions prove more cost- effective than later remedial interventions, and can reduce the educational disadvantage faced by poor children. Constraint is distinguishing self- selection This confirms that government has been on the right track in expanding the Grade R programme and in doing so most extensively in poorer communities.

Evaluation methodology Key areas for good Grade R delivery in the literature were identified: –Quality is key: a quality curriculum, a quality teacher, and a quality response to developmental needs –ECD provision based on pedagogical practice and not a “watered-down Grade 1” –Require active, child-centred, participatory methods that are difficult to assimilate into the school system. –ECD should lay a foundation for literacy in Grade 1 –A previous South African study found that 65% of Grade R learners enter Grade 1 without the necessary skills or concepts to master reading.

Evaluation methodology It should be noted that the data analysis of the impact evaluation report reflects the Grade R programme during the years of 2006 to 2011 The data analysis was completed through a new dataset created by merging the EMIS masterlist of schools, the SNAP data on learners in each grade, and the Annual National Assessments (ANA) of 2011 and 2012 that provide test performance in mathematics and home language for Grades 1 to 6. There were a total of schools in the dataset The measure of ‘treatment’ was the percentage of learners in a school that had attended Grade R. Treatment for a cohort (exposure to school-based Grade R) is calculated as the number of children in Grade R as proportion of those in Grade 2 two years later. There may be underestimations based on learners that had left the school or attended private ECD

Evaluation methodology & findings Fixed effects at the school level were used to calculate impact (observing the relationship within rather than between schools) Impact is measured as the proportion of a standard deviation change in test scores as a result of full treatment, i.e. full exposure to Grade R. Exposure to Grade R increased mathematics scores in subsequent years by 2.5% of a standard deviation, and home language scores by 10.2% of a standard deviation (what learners should learn in 12 days or in 50 days for mathematics and home language respectively) Treatment has no statistically significant effect in lower quintiles, while a significant effect of approximately 10% and 20% of a standard deviation is estimated for Quintile 4 and Quintile 5 schools respectively in both maths and language Reviews of programmes in the United States had an average impact of 43%, while in Argentina this was 23%

Summary of findings The study confirms the strategic importance of the Grade R programme in our quest to improve education in South Africa, but shows that on average the learning gains attributable to the Grade R programme have been fairly small relative to what one might have hoped to see. In some schools Grade R has contributed towards better learning, but in other schools it has not. A low overall quality of instruction in some primary schools together with disadvantageous home background factors may also be working against the gains of Grade R provision. Grade R provision provides greater benefits for mathematics learning when implemented within a well-functioning education system, this is based on the analysis of provincial groupings (Gauteng, Northern Cape and Western Cape) The study also showed the utility of having systemic data such as the Annual National Assessments for understanding the performance of the education system

Recommendations, implications & responses Grade R remains an important part of schooling, particularly in the South African system where there are inequalities With such a large and rapid expansion in a new programme, one would expect there to be challenges with respect to the quality of implementation, especially in the early years. Such an expansion requires extensive development of infrastructure, the development and provision of learning support materials, new curriculum development and enough suitably qualified Grade R practitioners. These findings confirm that in Grade R, as is the case throughout the school system, there have been significant challenges in ensuring instructional quality. This is most true of the parts of the school system serving poor learners, as the report also showed. Now that access to Grade R is approaching universal coverage, the focus of the Department will be firmly on improving quality. An improvement plan based on recommendations from the report has already been approved by the Department of Basic Education. The improvement plan focuses on the human resource strategy for Grade R practitioners, curriculum implementation and monitoring initiatives.

Recommendations, implications & responses Dealing with quality Teacher training, quality and support received There has been an introduction of NQF standards for Grade R practitioners School management is an important contributor to learner performance, focus should include in-classroom practice and instructional support not just administration –focus on issuing basic competency standards for school management, and district directors –Focus on establishing norms and standards for Provincial Education Departments in the various areas to clarify responsibilities Practical curriculum guidelines and standards From 2010, the DBE has been distributing Grade R resource packs to schools, which include workbooks for each learner and teacher resources such as teaching guides, posters and large story books to be shown and read to the whole class There has been an independent formative evaluation of Grade R resource pack (pizza box) by ACER to evaluate the quality of the materials

Recommendations, implications & responses Monitoring and data on ECD provisioning It is essential to improve the data and knowledge base for the provision of Grade R. Agreements between the DBE and DSD to improve the knowledge base of ECD centres in order to improve support and access to materials and other good practices such as hygiene, food handling etc Standardised school and district monitoring have been developed by the DBE in support of using the same standard to measure provisioning. The tools focus on management, coverage and functionality The Annual National Assessments are beginning to equip parents to improve their monitoring and accountability tools as well as provide teachers with an instrument to assess themselves and learning gaps early in the system.

Recommendations, implications & responses The role of parents and society Parents have a significant impact on learning in the home environment. Campaigns on changing the culture of ECD at home and in small centres is important for all stakeholder. Provincial department are requested to lead this work and report to the Minister on this on a quarterly basis. It also forms part of the discussion in quarterly meetings between the Minister and District Directors An important milestone was the release in 2013 of the National Curriculum Framework for children from birth to age four.

Website: Call Centre: | | Facebook: DBE SA Thank you