ECS Forum on National Education Policy Austin, 2 July 2008 Benchmarking Internationally The Need Confronts Reality ECS Forum on National Education Policy July 2, 2008 Andreas Schleicher Head, Indicators and Analysis Division OECD Directorate for Education

ECS Forum on National Education Policy Austin, 2 July 2008 In the dark… …all institutions and education systems look the same… But with a little light….

But with a little light…. …important differences become apparent…. In the dark… …all institutions and education systems look the same…

ECS Forum on National Education Policy Austin, 2 July 2008 Today

PISA OECD Programme for International Student Assessment Briefing of Council 14 November 2007 There is nowhere to hide How the global talent pool has changed Outlook on demand and supply of skills

ECS Forum on National Education Policy Austin, 2 July 2008 A world of change in baseline qualifications Approximated by percentage of persons with high school or equivalent qualfications in the age groups 55-64, 45-55, und years % 1. Excluding ISCED 3C short programmes 2. Year of reference Including some ISCED 3C short programmes 3. Year of reference

ECS Forum on National Education Policy Austin, 2 July 2008 High school graduation rates Percentage of graduates to the population at the typical age of graduation %

ECS Forum on National Education Policy Austin, 2 July 2008 College-level graduation rates Percentage of tertiary type A graduates to the population at the typical age of graduation % 15 2 Decline of the relative position of the US from 1995 to 2005 r Note also: rising higher education qualifications seem generally not to have led to an “inflation” of the labour-market value of qualifications. In all but three of the 20 countries with available data, the earnings benefit increased between 1997 and 2003, in Germany, Italy and Hungary by between 20% and 40%

ECS Forum on National Education Policy Austin, 2 July 2008 The effects of the higher education expansion: A high calibre workforce or the overqualified crowding out the lesser qualified? Lower secondary unemployment rate as a ratio of upper secondary unemployment rate “Top group” The nine countries that expanded tertiary education fastest in the 1990s (5.9% on average) “Middle group” The eight countries with modest increases in tertiary education (2.4% on average) “Bottom group” The nine countries with no or very modest increases in tertiary education (0.1% on average)

ECS Forum on National Education Policy Austin, 2 July 2008 Moving targets Future supply of high school graduates

ECS Forum on National Education Policy Austin, 2 July 2008 Future supply of high school graduates Future supply of college graduates

ECS Forum on National Education Policy Austin, 2 July 2008 How the demand for skills has changed Economy-wide measures of routine and non-routine task input (US) (Levy and Murnane) Mean task input as percentiles of the 1960 task distribution

ECS Forum on National Education Policy Austin, 2 July 2008 Mathematics in PISA The real world The mathematical World A real situation A model of reality A mathematical model Mathematical results Real results Understanding, structuring and simplifying the situation Making the problem amenable to mathematical treatment Interpreting the mathematical results Using relevant mathematical tools to solve the problem Validating the results

ECS Forum on National Education Policy Austin, 2 July 2008 PISA A three-yearly global assessment that… … examines the performance of 15-year-olds in key subject areas as well as a wider range of educational outcomes Including students attitudes to learning and their learning behaviour …collects contextual data from… …students, parents, schools and systems… …in order to identify policy levers Coverage Representative samples of between 3,500 and 50, year-old students drawn in each country Most federal countries also draw state-level samples PISA covers roughly 90% of the world economy.

ECS Forum on National Education Policy Austin, 2 July 2008 Coverage of world economy 77%81% 83% 85%86%87% OECD’s PISA assessment of the knowledge and skills of 15-year-olds

PISA OECD Programme for International Student Assessment Briefing of Council 14 November 2007 OECD-PISA framework National educ, social and economic context Structures, resource alloc and policies Social & economic outcomes of education Community and school characteristics Student learning, teacher working conditions Socio-economic background of learners Antecedents contextualise or constrain ed policy The learning environment at school Teaching, learning practices and classroom climate Individ attitudes, engagement and behaviour Output and performance of institutions Quality of instructional delivery Quality and distribution of knowledge & skills Policy Levers shape educational outcomes Outputs and Outcomes impact of learning Individual learner Level A Instructional settings Level B Schools, other institutions Level C Country or system Level D Domain 3Domain 2 Domain 1

ECS Forum on National Education Policy Austin, 2 July 2008 Average performance of 15-year-olds in scientific literacy– extrapolate and apply (2006) High science performance Low science performance … 18 countries perform below this line

ECS Forum on National Education Policy Austin, 2 July 2008 Average performance of 15-year-olds in science – extrapolate and apply High science performance Low science performance … 18 countries perform below this line

ECS Forum on National Education Policy Austin, 2 July 2008 Average performance of 15-year-olds in science – extrapolate and apply Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High science performance Low science performance

ECS Forum on National Education Policy Austin, 2 July 2008 Durchschnittliche Schülerleistungen im Bereich Mathematik Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High science performance Low science performance

ECS Forum on National Education Policy Austin, 2 July 2008 OECD average = 500 Immigrants and science performance Native students Second-generation students First-generation students PISA 2006: Science Competencies for Tomorrow’s World, Figure 4.2a.

ECS Forum on National Education Policy Austin, 2 July 2008 Immigrants and science performance r Among 15-year-olds, the proportion of students with an immigrant background… …is 36% in Luxemburg and between 21 and 23% in Switzerland, Australia, New Zealand and Canada …is 15% in the United States …still exceeds 10% in Germany, Belgium, Austria, France, the Netherlands and Sweden r Immigrant students tend perform less well… …but in the OECD countries other than Luxembourg that have a greater immigrant share, immigrant students perform better r US second-generation immigrant students do not perform better than first-generation students r Immigrant students tend to face the double disadvantage of being in schools with a more disadvantaged socio-economic intake r Immigrant students tend to report stronger attitudes towards science.

ECS Forum on National Education Policy Austin, 2 July 2008 Large proportion of top performers Top and bottom performers in science 20 Large prop. of poor perf. These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position… These students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations…

ECS Forum on National Education Policy Austin, 2 July 2008 Increased likelihood of postsec. particip. at age 19 associated with reading proficiency at age 15 (Canada) after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group Level 1)

PISA OECD Programme for International Student Assessment Briefing of Council 14 November 2007 How to get there Some policy levers that emerge from international comparisons

ECS Forum on National Education Policy Austin, 2 July 2008 Some myths r US coverage of the sampled population is more comprehensive than in other countries US covered 96% of 15-year-olds enrolled (OECD 97%) US covered 86% of all 15-year-olds (OECD 89%) No impact on mean performance r No relationship between size of countries and average performance r No relationship between proportion of immigrants and average performance r Few difference in students’ reported test motivation r Limited impact of national item preferences.

ECS Forum on National Education Policy Austin, 2 July 2008 Money matters - but other things do too

ECS Forum on National Education Policy Austin, 2 July 2008 Investments and outcomes r Since 2000, expenditure per primary and secondary student increased across OECD countries by 39% (in real terms) … …while PISA outcomes generally remained flat… …but there are notable exceptions…

ECS Forum on National Education Policy Austin, 2 July 2008 Poland raised its reading performance by 28 PISA points, equivalent to ¾ of a school year OECD (2007), Learning for tomorrow’s world: First results from PISA 2006, Table 6.1a 20 In 2003, performance variation among schools had fallen from 51% to 16% of the variation of student performance But did this lead to genuine improvements of school performance? Between 2000 and 2003 showed the second- largest increase in reading (17 points) and a further 11 point increase since 2003 Most of that increase resulted from smaller proportions at the bottom level (23% in 2000, and three-quarters in vocational tracks, 17%in 2003) Did this harm the better performers?

ECS Forum on National Education Policy Austin, 2 July 2008 High ambitions and universal standards Access to best practice and quality professional development

ECS Forum on National Education Policy Austin, 2 July 2008 High ambitions Access to best practice and quality professional development Accountability and intervention in inverse proportion to success Devolved responsibility, the school as the centre of action

ECS Forum on National Education Policy Austin, 2 July 2008 PISA score in science School autonomy, standards-based examinations and science performance School autonomy in selecting teachers for hire

ECS Forum on National Education Policy Austin, 2 July 2008 Public and private schools Private schools perform better Public schools perform better % Score point difference

ECS Forum on National Education Policy Austin, 2 July 2008 Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a 20 Schools practicing ability grouping (gross and net) Academically selective schools (gross and net) but no system-wide effect School results posted publicly (gross and net) One additional hour of science learning at school (gross and net) One additional hour of out- of-school lessons (gross and net) One additional hour of self-study or homework (gross and net) School activities to promote science learning (gross and net) Schools with greater autonomy (resources) (gross and net) Each additional 10% of public funding (gross only) Schools with more competing schools (gross only) School principal’s perception that lack of qualified teachers hinders instruction (gross only) School principal’s positive evaluation of quality of educational materials (gross only) Measured effect Effect after accounting for the socio-economic background of students, schools and countries 64% of US students in schools that compete with more than 2 schools in same area, 11% with one school, 26% with no school 91% of US students in schools posting achievement data publicly (OECD 38%) 26% of US students in schools with no vacant science teaching positions (OECD 38%), 71% where all vacant positions had been filled (OECD 59%), BUT 20% where principals report that instruction is hindered by a lack of qualified science teachers

ECS Forum on National Education Policy Austin, 2 July 2008 Strong ambitions Access to best practice and quality professional development Accountability Devolved responsibility, the school as the centre of action Integrated educational opportunities From prescribed forms of teaching and assessment towards personalised learning

ECS Forum on National Education Policy Austin, 2 July 2008 Durchschnittliche Schülerleistungen im Bereich Mathematik Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High science performance Low science performance Early selection and institutional differentiation High degree of stratification Low degree of stratification

ECS Forum on National Education Policy Austin, 2 July 2008 A second chance? Expected hours in non-formal job-related training (2003) This chart shows the expected number of hours in non-formal job-related education and training, over a forty year period, for 25-to-64 year olds. % C5.1a

ECS Forum on National Education Policy Austin, 2 July 2008 Some paradigm shifts Prescription Informed profession UniformityEmbracing diversity DemarcationCollaboration Provision Outcomes Bureaucratic – look upDevolved – look outwards Talk equityDeliver equity Hit & miss Universal high standards Received wisdomData and best practice The old bureaucratic education systemThe modern enabling education system

ECS Forum on National Education Policy Austin, 2 July 2008 Why care? r Progress Concerns about skill barriers to economic growth, productivity growth and rates of technological innovation –One additional year of education equals to between 3 and 6% of GDP –Rising college-level qualifications seem generally not to have led to an “inflation” of the labour-market value of qualifications (in all but three of the 20 countries with available data, the earnings benefit increased between 1997 and 2003, in Germany, Italy and Hungary by between 20% and 40%) r Fairness Concerns about the role of skills in creating social inequity in economic outcomes –Both average and distribution of skill matter to long-term growth r Value for money Concerns about the demand for, and efficiency and effectiveness of, investments in public goods

ECS Forum on National Education Policy Austin, 2 July –All national and international publications –The complete micro-level database …and remember: Without data, you are just another person with an opinion