OECD Programme for International Student Assessment (PISA) Launch of PISA 2006 London, 4 December 2007 Barbara Ischinger Director Directorate for Education, OECD

A three-yearly global assessment that… PISA A three-yearly global assessment that… … examines the performance of 15-year-olds in key subject areas as well as a wider range of educational outcomes Including students attitudes to learning, their beliefs about themselves, and their learning strategies … collects contextual data from students schools, parents and systems to identify policy levers Coverage Representative samples of between 3,500 and 50,000 15-year-old students drawn in each country Most federal countries also draw regional samples PISA covers roughly 90% of the world economy .

PISA countries in 2006 1998 2003 2009 2000 2001 Coverage of world economy 86% 85% 87% 81% 77% 83%

How PISA works A strong international network of expertise among the participating countries… From establishing the assessment frameworks… The PISA assessments include tasks from more than 40 countries … developing the instruments… Cross-national and cross-cultural validity … to analysing and interpreting the results National, regional and international analyses and reports In-depths country peer reviews … supported by a consortium of the leading research institutions… ACER, CITO, ETS, NIER, WESTAT … co-ordinated through the OECD in collaboration with other international organisations .

Science in PISA 2006 PISA defines scientific literacy in terms of an individual’s: Scientific knowledge and use of that knowledge to… … identify scientific issues, … explain scientific phenomena, and … draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues .

Quality in educational outcomes High science performance Average performance of 15-year-olds in science – extrapolate and apply Quality in educational outcomes … 18 countries perform below this line Low science performance

Mean science scores – OECD countries OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1c

Comparison of performance on the different scales in science (UK) Scientific competencies Scientific knowledge OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13

Gender differences in science performance (UK) OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Tables 2.1c, 2.2c, 2.3c, 2.4c, 2.7, 2.8, 2.9, 2.10

PISA Proficiency Levels in Science OECD UK OECD Science Level 6 Student can consistently identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations 1% 3% Level 6 Level 5 8% 11% 20% 22% Level 4 27% 26% Science Level 1 Student have such a limited scientific knowledge that it can only be applied to a few, familiar situations Level 3 24% 22% Level 2 14% 12% Level 1 Below Level 1 Unable to use scientific skills in ways required by easiest PISA tasks. Below Level 1 5% 5% OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1a

Top and bottom performers Large proportion of top performers Large prop. of poor perf. 20 OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 2.1a

Investments and outcomes Since 2000, expenditure per primary and secondary student increased across OECD countries by 39% (in real terms) … … while PISA outcomes generally remained flat… … but there are notable exceptions…

Between PISA 2000 and 2003 Poland delayed the separation of students into different school types beyond the age of 15 years Poland raised its reading performance by 28 PISA points, equivalent to ¾ of a school year - What happened? In 2003, performance variation among schools had fallen from 51% to 16% of the variation of student performance But did this lead to genuine improvements of school performance? Between 2000 and 2003 showed the second-largest increase in reading (17 points) and a further 11 point increase since 2003 Most of that increase resulted from smaller proportions at the bottom level (23% in 2000, and three-quarters in vocational tracks, 17%in 2003) Did this harm the better performers? 20 OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 6.1a

Mean reading scores – OECD countries OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 6.1c

Mean mathematics scores – OECD countries OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 6.2c

Attitudes towards science High science performance Average performance of 15-year-olds in science – extrapolate and apply Attitudes towards science … 18 countries perform below this line Low science performance

General value of science OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 3.2

Personal value of science OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 3.4

Student engagement with science Students show strong support for scientific enquiry 93% said that science was important for understanding the natural world 92% said that advances in science and technology usually improved people’s living conditions 75% said that science helped them to understand things around them 57% said that science was very relevant to them personally Students expressed confidence in be able to do scientific tasks, but more so for some tasks than others 20 76% said they could explain why earthquakes occurred more frequently in some areas than in others 64% said they could predict how changes to an environment would affect the survival of certain species 51% said they could discuss how new evidence could lead to a change in understanding about the possibility of life on Mars OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 4.1a

Equity in educational opportunities High science performance Average performance of 15-year-olds in science – extrapolate and apply High average performance Large socio-economic disparities High average performance High social equity Equity in educational opportunities Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

Durchschnittliche Schülerleistungen im Bereich Mathematik High science performance Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

School performance and socio-economic background Finland Student performance and students’ socio-economic background within schools School performance and schools’ socio-economic background Student performance and students’ socio-economic background Student performance Schools proportional to size Advantage PISA Index of socio-economic background Disadvantage

School performance and socio-economic background United Kingdom Student performance and students’ socio-economic background within schools School performance and schools’ socio-economic background Student performance and students’ socio-economic background Student performance Schools proportional to size Advantage PISA Index of socio-economic background Disadvantage

Student performance and migration Native students Second-generation students OECD average = 500 First-generation students PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.2a.

Is it all innate ability? Variation in student performance Performance variation across schools 20 OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 4.1a

Is it all innate ability? Variation in student performance PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.1a.

Is it all innate ability? Variation in student performance Variation of performance within schools Variation of performance between schools PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.1a.

Some school and system factors Money matters - but other things do too Some school and system factors

School autonomy and standards-based examination on science performance School autonomy in selecting teachers for hire PISA score in science

Impact of selected student and school factors on school performance (after accounting for all other factors in the model) School principal’s positive evaluation of quality of educational materials (gross only) Schools with more competing schools (gross only) Schools with greater autonomy (resources) (gross and net) School activities to promote science learning (gross and net) One additional hour of self-study or homework (gross and net) One additional hour of science learning at school (gross and net) School results posted publicly (gross and net) Academically selective schools (gross and net) but no system-wide effect Schools practicing ability grouping (gross and net) One additional hour of out-of-school lessons (gross and net) 20 Each additional 10% of public funding (gross only) School principal’s perception that lack of qualified teachers hinders instruction (gross only) Effect after accounting for the socio-economic background of students, schools and countries Measured effect OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a

Thank you ! www.oecd.org; www.pisa.oecd.org All national and international publications The complete micro-level database (data from students, parents, schools and systems) email: pisa@oecd.org Thank you !