1. 10 things policymakers should know about learning goals and assessment
I want to start with a brief overview of the objectives and origins of PISA, then show you where the US stands and what the most effective school systems show can be achieved, and conclude with what all of this means for education policy.
I need to remind you that the material remains under embargo until next Tuesday.
Andreas Schleicher
St. Petersburg, May 2014

2. Benchmark globally 2
Lesson 1
In a global economy, the benchmark for educational success is no longer solely improvement by national standards, but the best performing education systems internationally

3. A world of change – higher education
Cost per student
Graduate supply

4. A world of change – higher education
United States
Cost per student
Graduate supply

5. A world of change – higher education
United Kingdom

6. A world of change – higher education

7. A world of change – higher education

8. A world of change – higher education

9. A world of change – higher education

10. A world of change – higher education

11. A world of change – higher education

12. A world of change – higher education

13. A world of change – higher education

14. A world of change – higher education

15. A world of change – higher educationUK
Iceland
Poland

16. A world of change – higher educationUS

17. Don’t sacrifice validity gains for efficiency gains17
Lesson 2
The kinds of things that are easy to teach and test are also easy to digitise, automate and outsource

18. Changes in the demand for skills Trends in different tasks in occupations (United States)
Another way of looking at the evolution of demand for skills is provided by Autor, Levy and Murnane (2003), who
classify jobs into routine and non-routine tasks. They argue that the share of non-routine analytic and interactive job
tasks (tasks that involve expert thinking and complex communication skills) performed by American workers has
increased steadily since The share of routine cognitive and manual tasks began to decline in the
early 1970s and 1980s, respectively – coinciding with the introduction of computers and computerised production
processes. These are tasks that are more readily automated and put into formal algorithms. The share of non-routine
manual tasks also declined, but stabilised in the 1990s, possibly due to the fact that they cannot be easily computerised
or outsourced.
Source: Autor, David H. and Brendan M. Price "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane (2003)." MIT Mimeograph, June.

19. The false choice between equity and excellence
Monitor excellence and equity jointly 19
Lesson 3
The false choice between equity and excellence

20. High mathematics performance
… Shanghai-China performs above this line (613)
Average performance of 15-year-olds in
Mathematics
Fig I.2.13
This chart illustrates the reading literacy scale, from below the OECD average, marked in red, to around the OECD average, marked in yellow, to high performance, marked in green. .
… 12 countries perform below this line
Low mathematics performance

21. High mathematics performance
Average performance of 15-year-olds in
mathematics
Strong socio-economic
impact on student
performance
Socially equitable
distribution of learning
opportunities
But I do want to introduce a second dimension into this picture, that PISA pays great attention to. When you look at the distribution of student performance within each country, there are some countries in which social background has a strong impact on student performance, in other words, where educational opportunities are very unequally distributed, where there is a large gap between winners and losers and where a lot of the potential that children bring with them is wasted. There are other countries, where it matters much less into which social context students are born, where outcomes are socially equitably distributed.
If you look at this, it is clear where we all want to be, namely where performance and equity are both strong. And nobody, and no country, can accept to be where performance is low and opportunities are very unequally distributed. Whether it is better to have high performance at the price of large disparities, or better to invest in small disparities at the price of mediocracy, that is subject to debate.
Low mathematics performance

22. 2012 Strong socio-economic Socially equitable impact on student
Shanghai-China
Strong socio-economic
impact on student
performance
Socially equitable
distribution of learning
opportunities
Strength of ESCS on performance (% variance explained) and maths perf 2012

23. 2012 Strong socio-economic Socially equitable impact on student
performance
Socially equitable
distribution of learning
opportunities
Strength of ESCS on performance (% variance explained) and maths perf 2012

24. Strength of ESCS on performance (% variance explained) and maths performance 2012
Size of bubbles shows spending levels

25. Not more money but better spending choices make a difference
Spend wisely to make a difference 25
Lesson 4
Not more money but better spending choices make a difference

26. Contribution of various factors to upper secondary teacher compensation costs, per student as a percentage of GDP per capita (2004)
Percentage points
The red dot indicates classroom spending per student, relative to the spending capacity of countries, the higher the dot, the more of its GDP a country invests.
High salaries are an obvious cost driver. You see Korea paying their teachers very well, the green bar goes up a lot. Korea also has long school days, another cost driver, marked here by the white bar going up. Last but not least, Korea provides their teachers with lots of time for other things than teaching such as teacher collaboration and professional development, which costs money as well. So how does Korea finances all of this? They do this with large classes, the blue bar pulls costs down.
If you go to the next country on the list, Luxembourg, you see that the red dot is about where it is for Korea, so Luxembourg spends roughly the same per student as Korea. But parents and teachers in Luxembourg mainly care about small classes, so policy makers have invested mainly into reducing class size, you see the blue bar as the main cost driver. But even Luxembourg can only spend its money once, and the result is that school days are short, teacher salaries are average at best and teachers have little time for anything else than teaching. Finland and the US are a similar contrast.
Countries make quite different spending choices. But when you look at this these data long enough, you see that many of the high performing education systems tend to prioritise the quality of teachers over the size of classes.

27. 2003 - 2012 Shanghai Singapore Singapore
Germany, Turkey and Mexico improved both their mathematics performance and equity levels
Singapore
Liechtenstein, Norway, the United States and Switzerland improved their equity levels (no change in performance)
Singapore
Brazil, Italy, Macao-China, Poland, Portugal, Russian Federation, Thailand and Tunisia improved their mathematics performance (no change in equity)
Strength of ESCS on performance (% variance explained) and maths performance 2012
Size of bubbles shows spending levels

28. Keep track of ‘added value’28
Lesson 5
The country where students go to class matters more than what social class students come from

29. School performance and socio-economic background: Brazil
Student performance
Advantage
PISA Index of socio-economic background
Disadvantage

30. Desempeño del colegio y nivel socioeconómico
Vietnam
Desempeño de los estudiantes
Ventaja
Índice de PISA de nivel socioeconómico
Desventaja

31. School performance and socio-economic background: Brazil
Student performance
Advantage
PISA Index of socio-economic background
Disadvantage

32. Resilience in education
PISA performance by decile of social background 32
Source: PISA 2012

33. Provide a data-rich school environment to combat inequities33
National and school data
School-based strategies for analysis
Data for supporting learning

34. Catching up with the top-performers
High impact on outcomes
Must haves
Quick wins
Catching up with the top-performers
Low feasibility
High feasibility
Money pits
Low hanging fruits
Low impact on outcomes

35. Commitment to universal achievement
High impact on outcomes
Must haves
Quick wins
Commitment to universal achievement
Capacity
at point of delivery
Resources
where they yield most
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
Low impact on outcomes

36. Lesson 6: Believe that all children can achieve
High impact on outcomes
Must haves
Lesson 6: Believe that all children can achieve
Universal educational standards and personalization as the approach to heterogeneity in the student body…
… as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity
Clear articulation who is responsible for ensuring student success and to whom
Quick wins
Commitment to universal achievement
Capacity
at point of delivery
Resources
where they yield most
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
Low impact on outcomes

37. High expectations for all students37
Countries where students have stronger beliefs in their abilities perform better in mathematics
Source: PISA 2012

38. Perceived self-responsibility for failure in mathematics
Fig III.3.6
Percentage of students who reported "agree" or "strongly agree" with the following statements:

39. A continuum of support 39
Make learning central, encourage engagement and responsibility
Be acutely sensitive to individual differences
Provide continual assessment with formative feedback
Be demanding for every student
Ensure that students feel valued and included and learning is collaborative

40. High impact on outcomes
Must haves
Quick wins
Commitment to universal achievement
Lesson 7: Have clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems
Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)
High level of metacognitive content of instruction …
Capacity
at point of delivery
Resources
where they yield most
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
Low impact on outcomes

41. Lesson 8: Build capacity at the point of delivery
High impact on outcomes
Lesson 8: Build capacity at the point of delivery
Attracting, developing and retaining high quality teachers and school leaders and a work organisation in which they can use their potential
Instructional leadership and human resource management in schools
Keeping teaching an attractive profession
System-wide career development …
Must haves
Quick wins
Commitment to universal achievement
Capacity
at point of delivery
Resources
where they yield most
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
Low impact on outcomes

42. Teacher shortage
Fig IV.3.5

43. Prepare for work in disadvantaged schools
Reinforce initial teacher training including curriculum content for disadvantage
Strengthening diagnostic capacity
Include practical field experience 43
Preparation
Prepare teachers for work in disadvantage
Provide mentoring in disadvantage
Improve working conditions
Career and financial incentives
Both new and experienced teachers benefit
Pedagogical and relational strategies

44. Lesson 9: Align autonomy with accountability
High impact on outcomes
Must haves
Quick wins
Lesson 9: Align autonomy with accountability
Aligned incentive structures
For students
How gateways affect the strength, direction, clarity and nature of the incentives operating on students at each stage of their education
Degree to which students have incentives to take tough courses and study hard
Opportunity costs for staying in school and performing well
For teachers
Make innovations in pedagogy and/or organisation
Improve their own performance and the performance of their colleagues
Pursue professional development opportunities that lead to stronger pedagogical practices
A balance between vertical and lateral accountability
Effective instruments to manage and share knowledge and spread innovation – communication within the system and with stakeholders around it
A capable centre with authority and legitimacy to act
Commitment to universal achievement
Capacity
at point of delivery
Resources
where they yield most
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
Low impact on outcomes

45. Align autonomy with accountability45
The question is not how many charter schools you have but how you enable every teacher to assume charter-like autonomy

46. Countries that grant schools autonomy over curricula and assessments tend to perform better in mathematics   46
Source: PISA 2012

47. Across all participating countries and economies
Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration
Fig IV.1.17
School autonomy for resource allocation x System's level of teachers participating in school management
Across all participating countries and economies
Score points

48. School autonomy for curriculum and assessment
Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements
Fig IV.1.16
School autonomy for curriculum and assessment
x system's level of posting achievement data publicly
Score points

49. School autonomy for curriculum and assessment
Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies
Fig IV.1.16
School autonomy for curriculum and assessment
x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials)
Score points

50. Quality assurance and school improvement
Fig IV.4.14

51. Square school choice with equity51
Controlled choice
Financial incentives
Inform parents
Foster collaboration among teachers and schools
Use student and school assessments
Financial incentives for schools
Assistance for disadvantaged parents

52. Lesson 10: Invest resources where they can make most of a difference
High impact on outcomes
Must haves
Quick wins
Commitment to universal achievement
Lesson 10: Invest resources where they can make most of a difference
Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms)
Effective spending choices that prioritise high quality teachers over smaller classes
Capacity
at point of delivery
Resources
where they yield most
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
Low impact on outcomes

53. Align the resources with the challenges53
Countries with better performance in mathematics tend to allocate educational resources more equitably 
Adjusted by per capita GDP
Greater equity
Less equity
Source: PISA 2012

54. Adequate resources to address disadvantage54
A shortage of qualified teachers is more of concern in disadvantaged schools
Disadvantaged schools reported more teacher shortage
Advantaged schools reported more teacher shortage

55. Reduce tracking and grade repetition55
Both vertical and horizontal stratification hurt equity

56. A final thought Commitment to universal achievement Capacity
High impact on outcomes
Must haves
Quick wins
Commitment to universal achievement
Capacity
at point of delivery
Resources
where they yield most
A final thought
Alignment of policies across all aspects of the system
Coherence of policies over sustained periods of time
Consistency of implementation
Fidelity of implementation (without excessive control)
Gateways, instructional systems
Coherence
A learning system
Low feasibility
High feasibility
Incentive structures and accountability
Money pits
Low hanging fruits
CAN
Low impact on outcomes

57. Find out more about our work at www.oecd.org
Thank you 57
Find out more about our work at
All publications
The complete micro-level database
Twitter: SchleicherEDU
and remember:
Without data, you are just another person with an opinion