1. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Is the sky the limit to educational improvement? Global Education Competitiveness Summit Washington, 29-30 June 2009 Prof. Andreas Schleicher Head, Indicators and Analysis Division OECD Directorate for Education

2. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher There is nowhere to hide The yardstick for success is no longer improvement by national standards but the best performing education systems

3. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher A world of change in baseline qualifications Approximated by percentage of persons with high school or equivalent qualfications in the age groups 55-64, 45-55, 45-44 und 25-34 years % 1. Excluding ISCED 3C short programmes 2. Year of reference 2004 3. Including some ISCED 3C short programmes 3. Year of reference 2003. 13 1 1 27

4. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education Graduate supply Cost per student

5. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States Finland Graduate supply Cost per student

6. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education Australia United States (2000) Finland

7. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education

8. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education

9. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education

10. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education

11. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States Australia Finland

12. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States

13. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States

14. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States

15. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States

16. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education United States

17. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education

18. Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate A world of change – college education Finland United States r Rising higher education qualifications seem generally not to have led to an “inflation” of the labour-market value of qualifications. In all but three of the 20 countries with available data, the earnings benefit increased between 1997 and 2003, in Germany, Italy and Hungary by between 20% and 40%

19. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Moving targets Future supply of high school graduates

20. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Future supply of high school graduates Future supply of college graduates

21. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher How the demand for skills has changed Economy-wide measures of routine and non-routine task input (US) (Levy and Murnane) Mean task input as percentiles of the 1960 task distribution The dilemma of schools: The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource

22. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher A three-yearly global assessments to… … examine the performance of 15-year-olds in key subject areas as well as a wider range of educational outcomes Including students attitudes to learning and their learning behaviour …collect contextual data from… …students, parents, schools and systems… …in order to identify policy levers shaping learning outcomes Coverage Representative samples of between 3,500 and 50,000 15-year-old students drawn in each country Federal countries typically also draw state-level samples OECD’s PISA assessment of the knowledge and skills of 15-year-olds

23. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Coverage of world economy 77%81% 83% 85%86%87% OECD’s PISA assessment of the knowledge and skills of 15-year-olds

24. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Mathematics in PISA The real world The mathematical World A real situation A model of reality A mathematical model Mathematical results Real results Understanding, structuring and simplifying the situation Making the problem amenable to mathematical treatment Interpreting the mathematical results Using relevant mathematical tools to solve the problem Validating the results

25. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher PISA defines science performance in terms of a student’s: Scientific knowledge and use/extrapolation of that knowledge to… …identify scientific issues, …explain scientific phenomena, and …draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues For example When reading about a health issue, can students separate scientific from non- scientific aspects of the text, apply knowledge and justify personal decisions ?

26. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher PISA defines science performance in terms of a student’s: Scientific knowledge and use/extrapolation of that knowledge to… …identify scientific issues, …explain scientific phenomena, and …draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues For example Can students distinguish between evidence-based explanations and personal opinions ?

27. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher PISA defines science performance in terms of a student’s: Scientific knowledge and use/extrapolation of that knowledge to… …identify scientific issues, …explain scientific phenomena, and …draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues For example Can individuals recognise and explain the role of technologies as they influence a nation’s economy ? Or are they aware of environmental changes and the effects of those changes on economic/social stability ?

28. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher PISA defines science performance in terms of a student’s: Scientific knowledge and use/extrapolation of that knowledge to… …identify scientific issues, …explain scientific phenomena, and …draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues Interest in science, support for scientific enquiry, responsibility for the environment This addresses the value students place on science, both in terms of topics and in terms of the scientific approach to understanding the world and solving problems

29. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Context - Personal - Social/public - Global Science competencies in PISA Science competencies in PISA -Identify scientific issues -Explain phenomena scientifically -Use scientific evidence Knowledge -Knowledge of science -Knowledge about science Attitudes -Interest in science -Support for scientific enquiry -Responsibility Identifying Recognising issues that can be investigated scientifically Identifying keywords in a scientific investigation Recognising the key features of a scientific investigation Explaining Applying knowledge of science in a situation Describing or interpreting phenomena scientifically or predicting change Using evidence Interpreting scientific evidence and drawing conclusions Identifying the assumptions, evidence and reasoning behind conclusions Knowledge of science Physical systems (structure of matter, properties of matter, chemical changes of matter, motions and forces, energy and its transformations, energy and matter) Living systems (cells, humans, populations, ecosystems, biosphere) Earth and space (structures of the earth system, energy in the earth system, change in the earth system, earth’s history, space) Technology systems (Concepts and principles, science and technology) Knowledge about science Scientific enquiry (purpose, experiments, data, measurement, characteristics of results) Scientific explanations (types, rules, outcomes) Interest science Indicate curiosity in science and science-related issues and endeavours Demonstrate willingness to acquire additional scientific knowledge and skills, using variety of resources and methods Demonstrate willingness to seek information and have an interest in science, including consideration of science-related careers Support for science Acknowledge the importance of considering different scientific perspectives and arguments Support the use of factual information and rational explanation Logical and careful processes in drawing conclusions

30. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Average performance of 15-year-olds in science – extrapolate and apply High science performance Low science performance … 18 countries perform below this line

31. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Strengths and weaknesses of countries in science relative to their overall performance France OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13 Science competencies Science knowledge

32. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Strengths and weaknesses of countries in science relative to their overall performance Czech Republic OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13 Scientific competencies Scientific knowledge 20

33. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Strengths and weaknesses of countries in science relative to their overall performance United States OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13 Science competencies Science knowledge

34. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Context - Personal - Social/public - Global Competencies Competencies -Identify scientific issues -Explain phenomena scientifically -Use scientific evidence Knowledge -Knowledge of science -Knowledge about science Attitudes -Interest in science -Support for scientific enquiry -Responsibility Identifying Recognising issues that can be investigated scientifically Identifying keywords in a scientific investigation Recognising the key features of a scientific investigation Explaining Applying knowledge of science in a situation Describing or interpreting phenomena scientifically or predicting change Using evidence Interpreting scientific evidence and drawing conclusions Identifying the assumptions, evidence and reasoning behind conclusions OECD Level 6 OECD Level 2 Students can demonstrate ability to understand and articulate the complex modelling inherent in the design of an investigation. Students can determine if scientific measurement can be applied to a given variable in an investigation. Students can appreciate the relationship between a simple model and the phenomenon it is modelling. Students can draw on a range of abstract scientific knowledge and concepts and the relationships between these in developing explanations of processes Students can recall an appropriate, tangible, scientific fact applicable in a simple and straightforward context and can use it to explain or predict an outcome. Students demonstrate ability to compare and differentiate among competing explanations by examining supporting evidence. They can formulate arguments by synthesising evidence from multiple sources. Students can point to an obvious feature in a simple table in support of a given statement. They are able to recognise if a set of given characteristics apply to the function of everyday artifacts.

35. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Large proportion of top performers Top and bottom performers in science 20 Large prop. of poor perf. These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position… These students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations…

36. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Increased likelihood of postsec. particip. at age 19 associated with reading proficiency at age 15 (Canada) after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group Level 1)

37. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Scientific excellence among 15-year-olds and countries’ research intensity

38. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Average performance of 15-year-olds in science – extrapolate and apply Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High science performance Low science performance

39. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Durchschnittliche Schülerleistungen im Bereich Mathematik Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High science performance Low science performance 15

40. PISA OECD Programme for International Student Assessment Briefing of Council 14 November 2007 How to get there Some policy levers that emerge from international comparisons

41. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Money matters - but other things do too

42. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Spending choices on secondary schools Contribution of various factors to upper secondary teacher compensation costs per student as a percentage of GDP per capita (2004) Percentage points

43. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher High ambitions and universal standards Rigor, focus and coherence Great systems attract great teachers and provide access to best practice and quality professional development

44. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Challenge and support Weak support Strong support Low challenge High challenge Strong performance Systemic improvement Poor performance Improvements idiosyncratic Conflict Demoralisation Poor performance Stagnation

45. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher High ambitions Access to best practice and quality professional development Accountability and intervention in inverse proportion to success Devolved responsibility, the school as the centre of action

46. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher PISA score in science School autonomy, standards-based examinations and science performance School autonomy in selecting teachers for hire

47. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a 20 Schools practicing ability grouping (gross and net) Academically selective schools (gross and net) but no system-wide effect School results posted publicly (gross and net) One additional hour of science learning at school (gross and net) One additional hour of out- of-school lessons (gross and net) One additional hour of self-study or homework (gross and net) School activities to promote science learning (gross and net) Schools with greater autonomy (resources) (gross and net) Each additional 10% of public funding (gross only) Schools with more competing schools (gross only) School principal’s perception that lack of qualified teachers hinders instruction (gross only) School principal’s positive evaluation of quality of educational materials (gross only) Measured effect Effect after accounting for the socio-economic background of students, schools and countries

48. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Strong ambitions Access to best practice and quality professional development Accountability Devolved responsibility, the school as the centre of action Integrated educational opportunities From prescribed forms of teaching and assessment towards personalised learning

49. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Durchschnittliche Schülerleistungen im Bereich Mathematik Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High science performance Low science performance Early selection and institutional differentiation High degree of stratification Low degree of stratification 6

50. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Paradigm shifts Prescription Informed profession UniformityEmbracing diversity DemarcationCollaboration Provision Outcomes Bureaucratic – look upDevolved – look outwards Talk equityDeliver equity Hit & miss Universal high standards Received wisdomData and best practice The old bureaucratic education systemThe Finnish education system

51. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Why care? r Progress Concerns about skill barriers to economic growth, productivity growth and rates of technological innovation –One additional year of education equals to between 3 and 6% of GDP –Rising tertiary level qualifications seem generally not to have led to an “inflation” of the labour-market value of qualifications (in all but three of the 20 countries with available data, the earnings benefit increased between 1997 and 2003, in Germany, Italy and Hungary by between 20% and 40%) r Fairness Concerns about the role of skills in creating social inequity in economic outcomes –Both average and distribution of skill matter to long-term growth (high percentages of low skill impede growth) r Value for money Concerns about the demand for, and efficiency and effectiveness of, investments in public goods

52. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Next generation of PISA assessments Overcoming limits of current instruments Moving from an autopsy to real-time diagnosis

53. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher Low policy value High policy value Low feasibilityHigh feasibility Money pits Must haves Low-hanging fruits Quick wins Examine individual, institutional and systemic factors associated with high performance Establish the relative standing of countries in terms of quality and equity in basic school subjects Extending the range of competencies through which quality is assessed (including ICT) Measuring growth in learning A real-time assessment environment that helps build capacity in the profession. Monitor educational progress Assuming that every new skill domain is orthogonal to all others r Specific goals for PISA 2006: an experimental computer-based assessment of science competencies 2009: assessing digital literacy / electronic texts 2012: Assessing (collaborative) problem-solving, assessing dynamic competencies, moving towards electronic delivery of all assessment domains. Computer-delivered dynamic assessment task Adaptive assessment Make student thinking visible and allow for divergent thinking Explaining individual, institutional and systemic factors associated with high performance

54. Global Education Competitiveness Summit Washington, 29-30 June 2009 Is the sky the limit to educational improvement? Andreas Schleicher www.oecd.org; www.pisa.oecd.org –All national and international publications –The complete micro-level database email: pisa@oecd.org Andreas.Schleicher@OECD.org …and remember: Without data, you are just another person with an opinion