1. PISA 2009 – New Approaches to Assessing Reading Literacy
Gerry Shiel
Rachel Perkins
Educational Research Centre

2. Overview Status of reading literacy in PISA 2009
Overview of the framework
Engagement and metacognition
Demonstration of electronic reading items
Scaling reading literacy in PISA 2009 and reporting on trends

3. Reading Literacy as a Major Assessment Domain in PISA 2009

4. Definition of Reading Literacy in 2009
No change since 2000
Reading literacy is understanding, using and reflecting on written texts, in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society.
Written texts is understood to mean a range of texts – handwritten, printed and electronic

5. Dimensions of Reading Literacy
Situation
Text (medium, format, rhetorical structure)
Aspect (strategy or cognitive approach)

6. Reading Situations
Refers to uses for which the author constructed a text

7. Distribution of Reading Tasks by Situation

8. Texts Medium – printed vs. electronic (hypertext)
Text format – continuous, non-continuous, mixed and multiple
Rhetorical structure – description, narration, exposition, argumentation and instruction

9. Text - Medium
Electronic texts – hypertext – a text or texts with access structures that make possible and encourage non-linear reading.
Electronic texts exist in a number of environments, including web environment, desk top and .
PISA 2009 excludes electronic texts outside computer environments (e.g., text messages)

10. Continuous and Non-continuous Texts – Print Medium

11. Continuous and Non-continuous Texts – Electronic Medium

12. Distribution of Continuous and Non-continuous Texts (Print)

13. Reading Processes (Aspects)

14. Three Broad Aspect Categories
retrieving information;
interpreting texts;
reflection and evaluation.

15. Relationship between task, text and aspects in the print medium

16. Relationship between task, text and aspects in the electronic medium

17. PISA 2009 – Distributions of Items in Printed Text, by Aspect

18. PISA 2009 Reading Response Formats
Print Texts
Multiple choice
Constructed Response
Electronic Texts
Multiple choice
Constructed response
Closed response (e.g., Drag and drop; copy and paste; select by clicking)
Open response (e.g., generating and entering a search term)

19. Reading Engagement
In PISA 2000, there was a stronger association between engagement in reading and reading performance than between SES and reading in all participating countries
It was argued that reading engagement could reduce the gap in achievement between subgroups
New definition of engagement, covering print and electronic texts, proposed for 2009.

20. Components of Reading Engagement
interest in reading – disposition to read literature and information text for enjoyment and the satisfaction of curiosity;
perceived autonomy – perceived control and self-direction of one’s reading activities, choices, and behaviours;
efficacy – belief in one’s capacity to read well in the present and future;
social interaction – social goals for reading and interactive competence;
mastery goal – aim of reading to comprehend, understand, and explain text;
reading practices – behavioural engagement referring to the amount and types of reading activities.

21. Measuring Engagement in Reading
Concept of unified engagement
Equal emphasis on reading motivation (interest, autonomy, efficacy, social interaction and mastery goal) and reading practices
Contextualisation of engagement across a range of types of reading material (fiction, information and social communication).

22. Educational Context of Engagement
Relevance: My teacher tells us how the reading relates to our experience.
Autonomy support: This teacher lets me choose what to read.
Self-efficacy support: My teacher shows me how to read the material for this course.
Social interaction in literacy: In class, I talk with other students about what the text means.
Mastery goal structure: The teacher explains how the topics fit together.
Opportunity to read: In this class the teacher helps us with a lot of challenging reading.

23. Metacognition and Reading
Metacognition in reading refers to the awareness of and ability to use a variety of appropriate strategies when processing texts in a goal oriented manner.
It also involves an understanding of the differential processing demands associated with various kinds of tasks and how to apply this understanding.
Metacognition has both a significant correlation with reading proficiency, and is amenable to teaching and learning

24. Measuring Metacognition

25. Assessing Metacognition

26. Pilot Assessment of Electronic Reading in the PISA 2009 Field Trial (2008)
22 countries participated in international option for reading of electronic texts
Australia, Austria, Belgium, Canada, Chile, Denmark, France, Germany, Hong-Kong, Hungary, Iceland, Ireland, Japan, Korea, Macao-China, New Zealand, Norway, Poland, Russian Federation, Spain, Sweden and United Kingdom.
Decision on participation in electronic reading component of main study to be made in October 2008.

27. Relationship between navigation and text processing for ERA tasks

28. Reading Literacy Scales in PISA 2009
Reading trend scale (based on 28 ‘common’ items)
PISA 2009 reading literacy scale (print only)
PISA 2009 electronic reading literacy scale
PISA 2009 reading literacy scale (print and electronic combined)

29. Reading Literacy Sub-scales in PISA 2009 (Print)
Access and retrieve
Integrate and interpret
Reflect and evaluate
Continuous
Non-continuous
(Items based on mixed and multiple texts not to be scaled – too few items)

30. Reading Literacy Sub-scales in PISA 2009 (Electronic)
Access and retrieve
Integrate and interpret
Reflect and evaluate
Complex (processes)