What do we know about reading outcomes in SA? -Nic Spaull (OECD/UJ/SU) 18 May 2016 BRIDGE

Overview 1.SA context 2.Why focus on reading? 3.Background research on reading in SA 4.What data is there on reading outcomes? – How do sample-based assessments work? – Inequality as the over-arching feature of SA education system Language, Province, Former-department, Socioeconomic status (SES) – PIRLS (2011), prePIRLS (2011), SACMEQ 2007, NSES New research on Oral Reading Fluency – Draper & Spaull (2015) 6.Reading as THE binding constraint in the education system

The curse of inequality

Not all schools are born equal 4 SA public schools? ? Pretoria Boys High School

Complex language dynamics in SA ANA 2013 Language of Assessment

Background: qualitative research on reading in SA Hoadley (2016) summarizes the SA classroom-based research and finds the following descriptive features: – Lack of opportunities for reading and writing (oral discourse dominates) – Classroom interaction patterns that privilege the collective (chorusing) – Weak forms of assessment and lack of feedback on student responses – Students in poorer schools can usually mechanically decode text (i.e. pronounce sounds and words) but have little comprehension of the content of what they are reading Similarly Pretorius & co-authors have found that a number of instructional practices (prevalent in SA) contribute to poor reading development: – The tendency of teachers to rely on whole class oral chorusing of reading, – The lack of reading homework – Minimal reading of extended texts in the early grades – (Pretorius & Machet 2004; Pretorius & Mokhwesana 2009; Pretorius 2014).

What data is there on reading outcomes in SA? “Without data you are just another person with an opinion” - Andreas Schleicher

Sample-based Census- based Number of schools? Number of students? Comparable over time? Cross-national studies of educational achievement TIMSS 1995, 1999, 2003, 2011, Yes SACMEQ 2000, 2007, Yes PIRLS 2006, 2011, 2015 (Eng/Afr only) Sort of prePIRLS 2011, NA National assessments (diagnostic) Systemic Evaluations 2001 (Gr3) 2004 (Gr6), 2007 (Gr3) ,000Sort-of - ANA 2011/12/13/14 247milDefinitely not Verification-ANA 2011, 2013 (Gr 3 & 6) 2164 (125/ prov) No NSES* Gr3 (2007) Gr4 (2008) Gr5 (2009) (8383 panel) Yes (+ longitudinal) National assessments (certification) -Matric6591about 550,000 *Number of schools and students is for the most recent round of assessments In addition to these, the Western Cape is the only province to have a population-based assessment at Grades 3, 6 and 9, also (confusingly) called the Systemic Evaluations (sometimes WCED SE) How do sample-based assessments work?

Purple dots = 266 NSES sampled schools Brown dots = former black schools Blue dots = former white schools Purple dots = school included in NSES (courtesy of Marisa Coetzee)

What do they show?

Numerous correlated dimensions Given the apartheid-era policies, it is unsurprising that the inequalities we see in South Africa can be seen along a number of correlated dimensions, including – Language, – Geographical location, – Socioeconomic status, – Race – Former department Inequality as the over-arching feature of education in SA

Can children read in African languages?

Do children in English or Afrikaans LOLT schools learn to read?

Former department… (NSES) NSES (Taylor, 2011) Taylor, 2011

Socioeconomic status... SACMEQ III (2007) Distribution of student reading scores by quartiles of school socioeconomic status (Spaull, 2013) SACMEQ III (2007)

Province…SACMEQ III Reading (Gr6)

Bimodality – indisputable fact 17 PIRLS / TIMSS / SACMEQ / NSES / ANA / Matric… by Wealth / Language / Location / Dept…

Labour Market AttainmentQualityType 18 High SES background + ECD High quality primary school High quality secondary school Low SES background Low quality primary school Low quality secondary schoo l Unequal society High productivity jobs and incomes (17%) Mainly professional, managerial & skilled jobs Requires graduates, good quality matric or good vocational skills Historically mainly white Low productivity jobs & incomes Often manual or low skill jobs Limited or low quality education Minimum wage can exceed productivity University/ FET Type of institution (FET or University) Quality of institution Type of qualification (diploma, degree etc.) Field of study (Engineering, Arts etc.) Vocational training Affirmative action Majority (80%) Some motivated, lucky or talented students make the transition Minority (20%) -Big demand for good schools despite fees -Some scholarships/bursaries QLFS 2014

New research: Oral Reading Fluency

ORF literature Oral reading fluency, defined in terms of accuracy and speed in word recognition, has been found to be a reliable indicator of reading comprehension (Fuchs et al, 2001; Spear-Swerling 2006). Although prosody is part of fluency, difficult to measure objectively. There is a strong empirical basis attesting to a relationship between fluency and reading comprehension. Fluency typically measured as total words read correct per minute. Fuchs et al. (2001) report high correlations (0.8) between ORF and various kinds of reading comprehension measure such as high stakes state mandated comprehension tests, as well as a variety of other comprehension tests using different formats (e.g. multiple choice or open questions, cloze procedures or story recall protocols). The relationship obtains across schools serving children from different socioeconomic backgrounds or instructional programmes, and occurs with children without reading difficulties as well as with children with learning disabilities with reading (Deno et al. 2001; Wolf & Katzir-Cohen 2001).

How do ORF assessments work?

ORF Test 1

ORF Test 2

NEEDU testing procedure rural primary schools 4697 grade 5 students wrote a comprehension test (San-Hunter /20) All grade 5 students in one class were selected to participate 1786 students selected to write Oral Reading Fluency Test 1 (next day) The top 3, middle 4 and bottom 3 students did the ORF 1 test (+ comprehension Test /5) 878 students selected to write ORF Test 2 If students read 50 WCPM they did the more difficult ORF test 2 (+ comprehension Test /5) Tests were administered at different times in the year. Preliminary analysis shows little Relation between time of test and results.

What about ESL kids in rural areas? Draper & Spaull, 2015 (SAJCE) “We found that South African Grade 5 second language learners from rural areas in South Africa had essentially the same distribution as Grade 1 second language learners in Florida, or Grade 2 second language remedial learners who had been removed from normal classes because they ‘cannot communicate meaning orally in English and demonstrate very little understanding in English’.

2 Large Research Projects Two large research projects are completed: – Binding Constraints in Education in SA (PSPPD) – Laying Firm Foundations: Getting Reading Right (Zenex) Both make the argument that reading in the FP is the biggest solvable problem in our education system.

PSPPD & Zenex Report Launch 24 May 2016

Main conclusions 1.The majority of South African children are not learning to read in any language by the end of Grade 3. 2.This is the binding constraint to improved educational outcomes for the poor. 3.These students never get a firm hold on the first rung of the academic ladder and stumble forward into higher grades with low self esteem and compromised learning.

PSPPD: Binding Constraints in Education

Policy déjà vu? Plus que ça change? We’ve been on this rodeo before… 1.National Reading Strategy (DBE, 2008) – Pandor 2.Teaching Reading in the Early Grades: A Teacher’s Handbooks (DBE, 2008) 3.Western Cape Numeracy & Literacy Strategy (WCED, 2006) 4.Foundations for Learning Campaign (DBE, 2008) 5.Gauteng Primary Literacy Strategy (GDE, 2010) 6.NEED Reading Study 2013 (NEEDU, 2013) 7.Systematic Method for Reading Success (Hollingsworth & Gains, 2009) 8.Western Cape Living Labs Schools (WCED, 2015) (See PSPPD Report (2016) for full discussion of previous reading initiatives) “Reading is, without doubt, the most important linguistic skill that needs to be developed in young children. Reading serves as a building block upon which all other learning takes place…This National Reading Strategy takes as its focus that reading failure begins in early grades, and it is at that level that interventions must be made” (DoE, 2008: 18).

Conclusions? Unless we get reading right in the Foundation Phase we are “falling at the first hurdle” (Taylor, 1989) Need to understand why previous reading initiatives failed Need for both capacity & accountability - not one or the other #1 Goal of the DBE must become: “Every child must read for fluently and with comprehension by the end of Grade 3 (age 9)”

Thank you

Zenex: Extreme class sizes in the Foundation Phase Proportion of children in Grades 1-3 in class different class sizes (using ASS 2013 data)

My questions 1.What evidence do we have to say that FP teachers don’t know how to teach reading? 2.Is it possible to teach in-service FP teachers how to teach reading? 3.Is there consensus on the most effective way(s) to teach reading in the African languages? 4.Is the abrupt-switch model of language transition to English in Gr4 the only way or the best way? 5.Is there consensus on a ‘tight’/explicit definitions of “reading for meaning” or “basic reading” or “reading with fluency and comprehension” or “literate” or “reading grade appropriate texts”? 6.Measuring is only the first step, we need to know how to help teachers/kids who can’t read or teach reading. Programs? Evidence?

Table 13: Oral Reading Fluency scores for English Second Language (ESL/ELL) in Broward County Public Schools (Florida, US) (Broward County, 2012)

SACMEQ III Reading (Gr6)

the ‘minimum level’ – was the level of achievement they would expect from a student who would barely survive during the next year of schooling. the ‘desirable level’ – was the level associated with a student that was guaranteed to cope with the next year of schooling.

Language... PIRLS 2006 PIRLS Gr 5 (Shepherd, 2011) prePIRLS 2011 prePIRLS Gr 4 (Howie & Van Staden, 2012)

Hoadley (2016) Descriptive features of Foundation Phase classrooms