PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Strong Performers and Successful Reformers in Education Francesca Borgonovi Early Childhood and Schools Division, EDU OECD Programme for International Student Assessment

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Foundation skills for the future  Rapid pace of change in most work environments requires strong foundation skills even in relatively routine manual occupations  Strong foundation skills are needed to make the most of computerisation and digitisation Literacy to understand descriptions of new procedures for rapidly changing tasks Numeracy to be able to understand the basic mathematics that permeates models used by computers

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Advanced skills for the future  Expert thinking and ability to solve problems with no rules-based solutions  Complex communication and the ability of interacting with humans to acquire information, explain it and lead others to action.

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 How the demand for skills has changed Economy-wide measures of routine and non-routine task input (US) (Levy and Murnane) Mean task input as percentiles of the 1960 task distribution The dilemma of schools: The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Implications for education systems  Technology can change the nature of work faster than people can change their skills  Education systems need to proactively and continuously re-think about how to develop the advanced skills that will be needed in the unpredictable labour markets and societies of the future

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September PISA countries in %81% 83% 85%86% Coverage of world economy 87% PISA in brief  Methods… Target population 15-year-old students Random samples –PISA 2009: over half a million students… ….representing 28 million 15-year-olds in 74 countries/economies 2 hour paper and pencil test in reading, mathematics and science + digital reading assessment Item types- constructed response and multiple choice; real life tasks Student background questionnaire –their personal background, their schools and their engagement with learning and school School and system background questionnaires –school policies, practices, resources and institutional factors that help explain performance differences.

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 PISA... a (relatively) long story  PISA in 2000, 2003, 2006 and 2009  PISA 2012 – In the field  PISA 2015 – Work in Progress  Focus on students’ capacity to extrapolate from what they know and creatively apply their knowledge in novel situations  Less emphasis on whether they can reproduce what they were taught

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Strong Performers and Successful Reformers in Education  Quality of Learning outcomes  Equity of learning opportunities  Factors that make a difference

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Average performance of 15-year-olds in reading High reading performance Low reading performance … 17 countries perform below this line

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Performance in reading (2009) 539

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Change in reading performance between 2000 and 2009 Score point change Reading performance improved Reading performance declined

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Percentage of students below proficiency Level 2 in reading between 2000 and

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Percentage of top performers in reading between 2000 and

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Equity of learning opportunities  Are students held back because of their socio- economic condition?  The role of schools: between and within school differences in performance  Gender gap

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Strength of the relationship between performance and socio-economic background above the OECD average impact below the OECD average impact Percentage of variance in performance explained by ESCS (r-squared x 100) Mean performance in reading Above average reading performance Above average impact of socio-economic background Below average reading performance Above average impact of socio-economic background Below average reading performance Below average impact of socio- economic background not statistically significantly different from the OECD average impact Above average reading performance Below average impact of socio- economic background

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 School performance and socio-economic background Finland Student performance Advantage PISA Index of socio-economic background Disadvantage Private school Public school in rural area Public school in urban area 700

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 School performance and socio-economic background Germany Student performance Advantage PISA Index of socio-economic background Disadvantage Private school Public school in rural area Public school in urban area 700

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Variability in student performance Variance

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Variability in student performance between and within schools Variance Performance variation of students in schools Performance differences between schools

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Gender Gap in Reading (PISA 2009, girls - boys) Girls perform better

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Factors that make a difference

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Differences between boys and girls compared with print reading Gender difference in digital reading performance (girls-boys) Gender difference in print reading performance (girls- boys) Colombia Denmark Hong Kong -China Macao - China Korea Spain New Zealand Norway Poland Ireland Iceland Australia Sweden OECD Average Belgium Japan Austria Hungary France Chile Girls have a larger advantage in print reading Girls have a larger advantage in digital reading

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 The role of digital resources  Digital reading can be a lever to reduce the gender gap The gender gap in digital reading is much smaller than in print reading, and relates to differences in navigation skills between boys and girls Reading more and reading with enjoyment promotes better reading, and better reading fosters stronger engagement.

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Digital Reading Performance and computer use at home

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Digital Reading Performance and computer use at school

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Computer use at home for leisure and digital reading performance - Japan

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Computer use at home for leisure and digital reading performance - Chile

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 ICT use at school and digital reading performance

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 The role of digital resources  ICT use at home for leisure is – up to a point – positively related to performance, navigation skills and self-confidence in completing high-level ICT tasks Parents and teachers need to encourage students to use computers so that they can improve their navigation skills but also provide guidance on balancing time spent using computers with time for other activities

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Changes in percentage of students who reported having a computer at home between PISA 2000 and 2009

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Changes in computers-per-student ratio between PISA 2000 and 2009

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Digital divide by country However, some countries lag behind on digital use Percentage of students who use a computer at home Percentage of students who use a computer at school

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Disadvantaged students have less access to a computer at home Digital divide by socio-economic background within countries Disadvantaged students have less access to a computer at school Disadvantaged students have more access to a computer at school Within countries, there is a digital divide by socio-economic background of students

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 The role of digital resources at school  Access to computers has improved significantly, both at home and at school, but important gaps remain between countries and social groups Strategies that promote wider access to ICT at school can help minimise impact of social background on digital competency gaps, with possible consequences for future employment opportunities

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 ICT can…  Enable students to obtain more regular feedback on their learning processes  Make students more active participants in learning processes in classrooms and tailor these processes to individual students’ needs  Provide students with up-to-date access to the world’s current research and thinking  However, identifying effective strategies to teach digital reading skills is an important policy objective Ability to critically evaluate the quality and credibility of texts, integrate information from multiple texts and – crucially – navigate effectively

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Who chooses which textbooks are used in school? Percentage of students in schools whose principals reported that only teachers or school principals choose which textbooks are used School level autonomy in the choice of textbooks

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Who chooses which textbooks are used in school? Percentage of students in schools whose principals reported that only regional and/or national educational authority choose which textbooks are used School level autonomy in the choice of textbooks

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Who determines course content in school? School level autonomy in the choice of course content (curricular autonomy) Percentage of students in schools whose principals reported that only teachers or school principals determine course content

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Who determines course content in school? School level autonomy in the choice of course content (curricular autonomy) Percentage of students in schools whose principals reported that only regional and/or national educational authority determine course content

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Does autonomy in curriculum and assessment make a difference?  Between country association with reading performance: Positive (countries with more autonomy in curriculum and assessment have higher mean performance)  Within country association with reading performance: Without accounting for the socio- economic and demographic background of students and schools After accounting for the socio- economic and demographic background of students and schools Negative relationship Positive relationship Negative relationship Positive relationship OECDAustria, Germany Luxembourg, Portugal, Switzerland Italy, Luxembourg Belgium, Netherlands, Switzerland OECD average change in score: 1.6 OECD average change in score: -1.0 Partner countries and economies Argentina, Bulgaria, Kazakhstan, Panama, Peru, Serbia, Shanghai- China Dubai (UAE) Argentina, Bulgaria, Peru, Shanghai-China, Chinese Taipei Dubai (UAE), Lithuania

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 PISA score in reading School autonomy, accountability and student performance Impact of s chool autonomy on performance in systems with and without accountability arrangements

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada) after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1) Odds ratio higher education entry School marks at age 15 PISA performance at age 15

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Some students learn at high levels All students need to learn at high levels Student inclusion Routine cognitive skills, rote learning Learning to learn, complex ways of thinking, ways of working Curriculum, instruction and assessment Few years more than secondary High-level professional knowledge workers Teacher quality ‘Tayloristic’, hierarchical Flat, collegial Work organisation Primarily to authorities Primarily to peers and stakeholders Accountability Education reform trajectories The old bureaucratic systemThe modern enabling system

PISA OECD Programme for International Student Assessment 8 th Educational Publishers Forum - Paris Francesca Borgonovi 20 September 2012 Find out more about PISA at…