Bridging the Gap from Implementation to Attainment: Utilising Results from International Comparative Studies. Surette van Staden PIRLS 2011 Co National Research Coordinator University of Pretoria Edu Lead Conference 14 April 2015

Aim of the Presentation Evidence of curricular attainment that has taken place at Grade 4 level across 11 languages in South Africa. Interrogation of benchmark results to provide evidence for possible improvement of curricular implementation at classroom level. Reflections of how teacher practice can benefit from evidence provided by international comparative studies.

What is PIRLS? An international comparative study in reading literacy under the auspices of the IEA Measures trends across years Monitors policy implementation over time Provides a global context for achievement

prePIRLS 2011 Grade 4 AfrikaansEnglishisiNdebeleisiXhosaisiZuluSepediSesothoSetswanasiSwatiTshivendaXitsonga PIRLS 2011 Grade 5 Afrikaans English

PIRLS and prePIRLS 2011 Samples StudyGradeAttained sample No of learners prePIRLS PIRLS Total Implementation in 2011 Reading Literacy Assessment Background Questionnaires

prePIRLS 2011 Overall Results

International Benchmarks Represented by a four-point scale:  Low International Benchmark (400 points)  Intermediate International Benchmark (475 points)  High International Benchmark (550 points)  Advanced International Benchmark (625 points)

PrePIRLS 2011 Overall Benchmark Performance

prePIRLS 2011 Overall Benchmark Performance % Achieving Benchmarks Of 71%

prePIRLS 2011 Benchmark Performance by Test Language

HOW DO THE BENCHMARKS TRANSLATE INTO WHAT GRADE 4 LEARNERS ARE ABLE TO DO?

LOW INTERNATIONAL BENCHMARK 400 POINTS Literary Texts: - Locate and retrieve explicitly stated detail Informational Texts: -Locate and reproduce 2-3 pieces of information

This means learners are probably unskilled in: monitoring of their own memory and comprehension reading as a meaning-making process detection of contradictions in text successfully resolving inconsistencies in understanding exerting control of reading processes realising that they do not understand what is read

ADVANCED INTERNATIONAL BENCHMARK 625 POINTS Literary Texts: -Integrate ideas and evidence across text -Interpret story events to provide reasons, motivations, feelings, character traits with text-based support Informational Texts: -Distinguish and interpret complex information from different parts of text with text-based support -Integrate information across text to provide explanations, interpret significance and sequence activities

This means learners are skilled in: demonstrating awareness of what they are reading knowing reasons why they are reading having tentative plans or strategies in place for handling potential problems monitoring their comprehension of textual information

The Value of Benchmark Information Unequivocally points to the problems in the foundational levels of the education system in getting learners to even master literal comprehension regardless of the language of instruction. The international benchmark medians shows that it is possible for Grade 4 learners to attain higher order comprehension skills at this developmental stage. Benchmark profiles reveal to the South African system and teachers specifically that learners are not engaged with mastery of reading comprehension or the higher order reading comprehension development needed for further academic achievement in the schooling system.

HOW COULD TEACHER PRACTICE BENEFIT FROM EVIDENCE PROVIDED BY INTERNATIONAL COMPARATIVE STUDIES?

Implementation Challenges Teacher comparisons made only within their own school and neighbouring teachers. Uncertainty about activities that are undertaken, pace and the level which these are pitched are often only mediated by curricular expectations and the pace of work within the school as dictated by fellow teachers and school management.

The Value of prePIRLS 2011 Results to Teachers Making released reading materials and questions that are used for purposes of the study available to teachers. Familiarity with the processes of comprehension the assessment is aimed at:  Focus on and retrieve explicitly stated information  Making straightforward inference  Interpret and integrate ideas and information  Examine and evaluate language and text elements Making international benchmark descriptions available. These may serve as a guide to teachers’ own development of reading material in the classroom against an existing, international framework.

The Value of prePIRLS 2011 Results to Teachers (continued) The provision of released passages to teachers to serve as exemplars of levels of expectation at Grade 4 as benchmarked against international standards. Exemplary learner responses linked to these items in themselves may provide valuable insights into comprehension processes and developmental requirements to teachers. Could serve as a trigger to a more informed understanding of what learners at this developmental stage of learning are actually capable of achieving.

The Value of prePIRLS 2011 Results to Teachers (continued) The impact of the PIRLS cycles on macro level policy is irrefutable (e.g. Foundations for Learning, Drop All and Read). Yet teacher level feedback from international testing and assessment programmes is still in its infancy and not fully realised in South Africa. Feedback to teachers is of cardinal importance, since change at classroom level can only be implemented in the face of empirical evidence of what constitutes effective classroom practice.

Where international assessment results remain only as grim reminders of South African learners’ poor performance, assessment as integral to learning has failed. Assessment results could provide evidence to inform further support or intervention for teachers and learners in need. Assessment should provide indicators of those systemic factors that can be changed, adapted or used as leverage against which to ensure improved curricular implementation. Failing to do so, assessment as agent of such improved curricular implementation, may become powerless in providing feedback of significant factors that are at work in the home, in classrooms and in schools nationwide. Conclusions

Thank you for your attention.