PISA OECD Programme for International Student Assessment Students On Line Students On Line Digital Technologies and Performance Programme for International Student Assessment Volume VI of the first international report of results from PISA 2009 Juliette Mendelovits Australian Council for Educational Research

PISA OECD Programme for International Student Assessment Students On Line Two sources for the report Digital reading assessment Optional international assessment A 40-minute assessment administered after the main PISA 2009 cognitive assessment (usually on the same day) Participation by 16 OECD countries and three partner countries and economies ICT familiarity survey Survey offered as an international option A short questionnaire administered to PISA students about their computer use at home and at school Participation by 29 OECD countries and 16 partner countries and economies Intersection: 15 OECD countries and two partner countries and economies

PISA OECD Programme for International Student Assessment Students On Line Why PISA 2009 included a digital reading assessment Digital reading  reading on line  is an increasingly important form of reading in the 21 st century Digital reading has different features to print reading and makes new demands on readers

PISA OECD Programme for International Student Assessment Students On Line Digital reading is an increasingly important form of reading in the 21 st century By June 2010… Almost 2 billion people were using the Internet worldwide Japan had the 3rd highest number of Internet users worldwide Almost 100 million Japanese people: 78% of the population This is, for Japan, a 100% growth in use from 2000 to 2010

PISA OECD Programme for International Student Assessment Students On Line Digital reading is an increasingly important form of reading in the 21 st century Digital reading is used for multiple and increasing purposes taxpayers fill in online forms; students search the web for information; jobseekers look up ads on employment websites; consumers order goods in online stores; people build and maintain social communities on line.

PISA OECD Programme for International Student Assessment Students On Line Digital reading has different features to print reading and makes new demands on readers  Different kinds of texts  mixed texts, including short pieces, diagrams, graphics, fragments  interactive texts, where readers can change, add to, or create their own texts  The reader has more responsibility, in selecting and constructing what they read  often no guide to the order in which a text should be read – The reader can take many pathways  an almost infinite number of texts available  the extent of a text is abstract and unknown – the reader cannot see the whole text  The reader has more responsibility, in evaluating the text  Traditional filters, like publishers and booksellers, are absent  The reader has to ask, “Is this text useful for me?” “Do I trust this text?”

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (1)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (1)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (2)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (2)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (3)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (3)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (4)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (4)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (5)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (5)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (6)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (6)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (7)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (7)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (8)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (8)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (9)

PISA OECD Programme for International Student Assessment Students On Line Example of a PISA digital reading task (9)

PISA OECD Programme for International Student Assessment Students On Line Identifying key elements of digital reading text processing locating, interpreting and evaluating words on the screen navigating from one screen to another, to find relevant information: predicting, integrating, critically evaluating Number of relevant page visits Number of visits to relevant pages Number of pages visited

Developing a scale to measure performance Level 5 Tasks at this level typically require the reader to locate, analyse and critically evaluate information, related to an unfamiliar context, in the presence of ambiguity. They require the generation of criteria to evaluate the text. Tasks may require navigation across multiple sites without explicit direction, and detailed interrogation of texts in a variety of formats Level 4 Tasks at this level may require the reader to evaluate information from several sources, navigating across several sites comprising texts in a variety of formats, and generating criteria for evaluation in relation to a familiar, personal or practical context. Other tasks at this level demand that the reader construe complex information according to well-defined criteria in a scientific or technical context. Level 3 Tasks at this level require that the reader integrate information, either by navigating across several sites to find well-defined target information, or by generating simple categories when the task is not explicitly stated. Where evaluation is called for, only the information that is most directly accessible or only part of the available information is required. Level 2 Tasks at this level typically require the reader to locate and interpret information that is well-defined, usually relating to familiar contexts. They may require navigation across a limited number of sites and the application of web-based tools such as dropdown menus, where explicit directions are provided or only low-level inference is called for. Tasks may require integrating information presented in different formats, recognising examples that fit clearly defined categories. Skilled digital readers Baseline digital readers

PISA OECD Programme for International Student Assessment Students On Line How proficient are students in digital reading? Tasks at Level 5 or above typically require the reader to locate, analyse and critically evaluate information, related to an unfamiliar context, in the presence of ambiguity. They require criteria to evaluate the text. Tasks may require navigation across multiple sites without explicit direction, and detailed interrogation of texts in a variety of formats. Tasks at Level 3 require that the reader integrate information, either by navigating across several sites to find well-defined target information, or by generating simple categories when the task is not explicitly stated. Where evaluation is called for, only the information that is most directly accessible or only part of the available information is required. Tasks at Level 4 may require the reader to evaluate information from several sources, navigating across several sites comprising texts in a variety of formats, and generating criteria for evaluation in relation to a familiar, personal or practical context. Other tasks at this level demand that the reader interpret complex information according to well-defined criteria in a scientific or technical context. Percentage of students Tasks at Level2 typically require the reader to locate and interpret information that is well-defined, usually relating to familiar contexts. They may require navigation across a limited number of sites and the application of web-based navigation tools such as dropdown menus, where explicit directions are provided or only low-level inference is called for. Tasks may require integrating information presented in different formats, recognising examples that fit clearly defined categories. Students performing below level 2 can scroll and navigate across web pages, as long as explicit directions are provided, and can locate simple pieces of information in a short block of hypertext. Nevertheless, although the digital reading skills of these students are not necessarily negligible, they are performing at levels that are not likely to allow them full access to educational, employment and social opportunities in the 21st century.

PISA OECD Programme for International Student Assessment Students On Line Low digital reading performance High digital reading performance Poland Hungry Spain Denmark Macao-China France Norway Belgium Ireland Sweden Iceland Hong Kong-China Japan AustraliaNew Zealand Korea Chile Colombia OECD Average - 16 Austria

PISA OECD Programme for International Student Assessment Students On Line Comparison of results on digital reading and print reading 16 OECD countries digital reading (%) Japan digital reading (%) Japan print reading (%) 16 OECD countries print reading (%) Proficiency Level 86139Level 5 and above Level Level Level Below Level 2 Skilled readers Baseline readers

PISA OECD Programme for International Student Assessment Students On Line Differences between boys and girls compared with print reading Gender difference in digital reading performance (girls-boys) Gender difference in print reading performance (girls- boys) Colombia Denmark Hong Kong -China Macao - China Korea Spain New ZealandNorway Poland Ireland Iceland Australia Sweden OECD Average Belgium Japan Austria Hungary France Chile Girls have a larger advantage in print reading Girls have a larger advantage in digital reading

PISA OECD Programme for International Student Assessment Students On Line Navigation Navigation is considered to be part of the cognitive process of digital reading Tracking and analysing the sequences of pages students visit provide insights into effective navigation behaviours in digital reading The index number of relevant pages visited describes how many of the pages judged to be relevant to a task were accessed while the student worked on that task.

PISA OECD Programme for International Student Assessment Students On Line Number of relevant pages visited and digital reading performance The index number of relevant pages visited describes how many of the pages judged to be relevant to a task were accessed while the student worked on that task

PISA OECD Programme for International Student Assessment Students On Line ICT questionnaire Students’ access to and use of computers and Internet Their general attitudes towards and self- confidence in using computers Not designed to assess the quality of ICT use at school and the integration of ICT in pedagogy Focus on students’ use of ICT to access, manage and present information

PISA OECD Programme for International Student Assessment Students On Line Changes in percentage of students who reported having a computer at home between PISA 2000 and 2009

PISA OECD Programme for International Student Assessment Students On Line ICT use at home for leisure and digital reading performance

PISA OECD Programme for International Student Assessment Students On Line ICT use at home for schoolwork and digital reading performance

PISA OECD Programme for International Student Assessment Students On Line ICT use at school and digital reading performance

PISA OECD Programme for International Student Assessment Students On Line Some conclusions Identifying effective strategies to teach digital reading skills is an important policy objective Ability to critically evaluate the quality and credibility of texts, integrate information from multiple texts and – crucially – navigate effectively ICT use at home for leisure is – up to a point – positively related to performance, navigation skills and self-confidence in completing high-level ICT tasks Parents and teachers need to encourage students to use computers so that they can improve their navigation skills but also provide guidance on balancing time spent using computers with time for other activities Digital reading can be a lever to reduce the gender gap The gender gap in digital reading is much smaller than in print reading, and relates to differences in navigation skills between boys and girls Reading more and reading with enjoyment promotes better reading, and better reading fosters stronger engagement.

PISA OECD Programme for International Student Assessment Students On Line Some conclusions Access to computers has improved significantly, both at home and at school, but important gaps remain between countries and social groups Strategies that promote wider access to ICT at school can help minimise impact of social background on digital competency gaps, with possible consequences for future employment opportunities Schools can do more to integrate ICT into learning Opportunities for students to solve problems using ICT Reading methods that improve students’ ability to distinguish between relevant and irrelevant material, and to structure, prioritise, distil and summarise texts

PISA OECD Programme for International Student Assessment Students On Line Some conclusions for Japan Less variation in digital reading proficiency than in print reading among Japanese students Japanese students with high socio-economic status are likely to be more proficient in digital reading, but this pattern is less marked than the average for OECD countries Girls performed better than boys on digital reading, but the gap is much narrower than for print.

PISA OECD Programme for International Student Assessment Students On Line Some conclusions for Japan When given a task, Japanese students are among the most thorough of digital readers BUT Japanese students are less likely to be engaged online readers than students in other countries If they were more involved in reading (both digital and print reading) then they would likely be more proficient digital readers.