1. EGRA and EGMA as tools for assessing teaching / learning quality in early schooling The EGRA/EGMA findings Conducted in 2014 Week of Education, 7 th December 2015 Ministry of Education, Science and Technology

2. . Order of Presentation Background Introduction of EGRA/EGMA Methodology of the assessment Summary of findings Conclusions and recommendations

3. Background Significant progress of access and completion of primary education  Education Country Status Report (2013): quality of education needs to be improved Current assessments NPSE & BECE  Assessments needed that diagnose WHY and in WHAT areas pupils are having difficulties

4. . The purpose of EGRA/EGMA in Sierra Leone To assess reading and mathematics skills of grade 2 and grade 4 pupils In order to inform: national policy pre-service training & continued professional development of teachers curriculum development

5. Summary of assessments 130 schools were visited 2,367 students were assessed nation-wide 251 lesson observations conducted 1 head teacher interview per school

6. What is EGRA/EGMA? Early Grades Reading Assessment and Early Grade Mathematics Assessment (EGRA/EGMA) were developed to: identify the reading/maths levels of pupils in lower grades identify difficulties in children’s learning/school experience  EGRA/EGMA subtasks correspond to different learning stages

7. Methodology of the assessment Steering Committee Tool development Sampling Enumerator Training & Selection Data Collection Monitoring & Supervision Data analysis and reporting

8. Steering Committee Role: To give direction; to ensure participation in the assessment; to cultivate ownership; and to ensure capacity building Members: - MEST - SSL - UNICEF - DFID - IRC - World bank

9. EGRA/EGMA tool development review of existing tools & Sierra Leone learning standards Review subject matter specialists drafted the tools Draft the tools were piloted and reviewed Pilot Submission of final tools to Steering Committee, approval 14th Feb 2014 Submission/ approval 4 day workshop held on 7 th - 11 th February 2014

10. Tool development EGRA subtasksSkill demonstrated by students’ ability to: Letter identification – letter namesProvide the name of upper- and lower case letters presented in random order Letter recognition – letter soundsProvide the sound of upper- and lower case letters presented in random order Familiar word readingRead a list of words drawn from a corpus of frequent words presented in random order Non-word reading (invented words)Identify “nonwords” composed of legal syllables presented in random order Connected text passage (fluent reading)Read a narrative or informational text with accuracy, little effort and sufficient rate Reading comprehensionRespond to literal and inferential questions about the text EGRA subtasks for the Sierra Leone assessment

11. EGRA subtasksSkill demonstrated by students’ ability to:Example Number identificationProvide the numbers that are presented to them in random order20, 3, 14, 5 …. Quantity discriminationChoose the number that is bigger out of the two numbers that are presented to them 10 4 4 25 Number patterns (missing numbers) Identify the missing numbers of the series of numbers presented to them 3, 6,, 12 15,, 35, 45 Addition Level 1 & 2Solve the simple (Level 1) and more challenging (Level 2) addition problems 3 + 5 = 8 18 + 7 = 25 Subtraction Level 1 & 2Solve the simple (Level 1) and more challenging (Level 2) subtraction problems 9 – 7 = 2 28 – 13 = 15 Word problemsSolve a simple addition and/or subtraction story problem which is presented to them Tool development EGMA subtasks for the Sierra Leone assessment

12. Enumerator training and selection Trainings conducted: - 8 day enumerator training - 3 day senior enumerator training - 2 day refresher training Senior Enumerator Enumerator

13. Sampling School sampling: In total 130 schools were sampled: 90 national public schools and 40 “ISIS”-schools” (out of a total of 5,791 schools) 6 schools (“ISIS-districts”) and 7 schools (“non-ISIS districts”) of each district were selected

14. Sampling Pupil sampling: 10 pupils per grade were sampled (5 boys, 5 boys) from P2 and P4 = 20 per school Head-count-method of random sampling was applied to pupils attending during day of the assessment

15. Monitoring & Supervision Visits to each team at least twice by Montrose staff Additional monitoring was carried out by MEST, SSL, UNICEF and OPM staff members “Tracking-sheets” were utilised Desk-based monitoring of uploaded data to identify any unexpected data

16. Results of the 2014 assessment

17. Summary of EGRA results 1.The majority of pupils in Sierra Leone are not learning HOW to read (“decoding skill”) 97,4% of the grade 2 students and 86,5% of the grade 4 students do not know a single letter sound in English 92,6% of the grade 2 students and 68,2% of the grade 4 students cannot read a single invented or ‘non’ words 2. P2 and P4 cannot read fluently enough and thus cannot read with comprehension 0,0% of the grade 2 students and 0,1% of the grade 4 students was able to answer 80% of the questions of the reading passage correctly 3. 1.1% of all pupils speak English at home yet 83.7% are taught English in schools.

18. Summary of EGMA Results 1.Most pupils were able to identify single digit numbers correctly. 2.Most pupils were unable to identify double or triple digit numbers On average, grade 2 students can identify 32,9% numbers (out of 100, in 1 minute) correctly. Grade 4 students can identify 73,5% numbers correctly 3. Pupils have difficulties to carry out simple addition and subtraction exercises. On average, grade 2 students can answer 20,2% of addition level 1 correctly (out of 20, in 1 minute) Grade 4 students can answer 46,5% correctly Addition level 2 On average, grade 2 students can answer 5,2% of the addition level 2 correctly Grade 4 students can answer 26,5% correctly 4

19. Conclusions and Recommendations

20. Recommendation to improve reading learning outcomes Implement letter sounds in early grade reading Ensure that children master decoding Encourage individual silent reading Link reading and writing exercises and acquisition Focus on reading comprehension Maximize and increase the learning time for early grade reading (review/reduce number of subjects on the curriculum) Set literacy benchmarks for each grade Consider mother tongue education for early grades Focus on the early grades

21. Recommendation to improve mathematics learning outcomes Focus on automaticity of single digit addition and subtraction exercises Focus on numeric and place value of numbers Focus on counting/calculation strategies Ensure transfer of formal mathematic problems into real life situations Increase learning time for early grades mathematics (review/reduce number of subjects)

22. Thank you very much

23. BACK-UP slides

24. EGRA results Average fluency rates

25. Zero scores per subtask

26. EGMA results xxxxxxxxxxxxxxxxxxxxxx

27. EGMA Results Zero scores per subtask

28. Overview of sample schools DistrictTotal Schools National Sample Schools National Replacement Schools ISIS Total SchoolsISIS Sample Schools Bo57573 Bombali50064305 Bonthe20573 Kailahun34564300 Kambia33173 Kenema62764307 Koinadugu357646011 Kono36164305 Moyamba47573 Port Loko51273 Pujehun20964301 Tonkolili51064607 Western Rural25373244 Western Urban53173120 Total5791914930640

29. Overview of pupils sampled

30. EGRA Comparison of zero scores across Africa

31. Language spoken at school and at home