1. Improving both quality and equity
OECD Programme for International Student Assessment (PISA)
Improving both quality and equity
Insights from PISA 2000
EI Round Table, Florence, 17 November 2003
Andreas Schleicher
Head, Indicators and Analysis Division OECD

2. Improving both quality and equity
1. Why we need to worry
The significance of educational outcomes for individuals and societies
2. Where we are today
What PISA shows about student performance and the social distribution of learning opportunities in OECD countries
3. Where we can be
What the best performing countries show can be achieved
4. How we can get there
Levers for policy that emerge from PISA

3. The significance of educational outcomes.
Changing contexts for education systems.

4. Changing contexts for education
Living with an ageing population
In some OECD countries, the share of over-65s in the population is growing twice has fast as the share of under 35s is shrinking
Unwanted but needed
School systems in most OECD countries will need to integrate unprecedented numbers of immigrants
...but only few are prepared

5. Changing contexts for education
Knowledge workers the only fast growing share of the workforce
By about the time when school reforms put in place today will begin to show effects in labour markets...
...Manufacturing output in OECD area is likely to double…
…Manufacturing employment is likely to shrink to 10% of the workforce
…And this is affects all jobs…

6. Total number of pages of repair manuals for Opel cars
Traditional jobs are changing too… Literacy skills in the manufacturing sector
Pages
Total number of pages of repair manuals for Opel cars
from 1933 to 1998
Source : Georg Spöttl, 2002
Year
1933
1951
1966
1983
1998

7. Changing contexts for education
Knowledge workers - the new “capitalists“
Knowledge is becoming the key economic and social resource
… and the only scarce one
Collectively, knowledge workers own the modern means of production
... and through their stakes in pension and mutual funds, they have become majority shareholders and owners of key businesses in the knowledge society
The knowledge society is a society of juniors and seniors
... rather than of bosses and subordinates

8. Unique opportunities Unlimited upward mobility…
Knowledge changes rapidly
Everybody starts from ignorance
Knowledge differs from traditional means of production in that it cannot be inherited or bequeathed
Knowledge is “public”
Knowledge has to be put in a form in which it can be taught and is therefore universally accessible
Every impediment to mobility…
…is perceived as a form of discrimination
… If our education systems deliver on their promises

9. Where we are today.
What PISA shows about student performance and the social distribution of learning opportunities in OECD countries.

10. PISA - The OECD Programme for International Student Assessment
A regular assessment of the yield of education (2000, 2003, 2006, 2009,…)
including and beyond the curriculum
Comparable skill measures
that can guide policy decisions
Insights into the mix of factors which contribute to the development of knowledge and skills
and how these factors operate similarly or differently across countries
A strong substantive and cross-cultural core for defining performance targets

11. PISA - The OECD Programme for International Student Assessment
The most comprehensive international assessment to date
Geographic and economic coverage
340,000 students randomly sampled
All 30 OECD countries plus a growing number of non-OECD countries
Subject matter coverage
Reading, Mathematics, Science
Cross-curricular competencies
Variety of task formats
Depths
A total of 7 hours of assessment material

12. Three broad categories of key competencies
Using “tools” interactively to engage with the world
To analyse, compare, contrast, and evaluate
e.g. Using language, symbols and texts
Interacting with information
Capitalising on the potential of technologies
To think imaginatively
PISA 2000: A new concept of literacy
Interacting in diverse groups
Acting autonomously
Accessing, managing, integrating and evaluating written information in order to develop ones knowledge and potential, and to participate in, and contribute to, society
e.g. Relating well to others
Co-operating, working in teams
Managing and resolving conflicts
e.g. Acting within the bigger picture
Forming and conducting life plans
Taking responsibility and understanding rights and limits
To apply knowledge in real-life situations
To communicate thoughts and ideas effectively

13. Using “tools” interactively to engage with the world
To analyse, compare, contrast, and evaluate
e.g. Using language, symbols and texts
Interacting with information
Capitalising on the potential of technologies
To think imaginatively
Reading literacy
Interacting in diverse groups
Acting autonomously
Using, interpreting and reflecting on written material
e.g. Relating well to others
Co-operating, working in teams
Managing and resolving conflicts
e.g. Acting within the bigger picture
Forming and conducting life plans
Taking responsibility and understanding rights and limits
To apply knowledge in real-life situations
To communicate thoughts and ideas effectively

14. Mathematical literacy
Using “tools” interactively to engage with the world
To analyse, compare, contrast, and evaluate
e.g. Using language, symbols and texts
Interacting with information
Capitalising on the potential of technologies
To think imaginatively
Mathematical literacy
Interacting in diverse groups
Acting autonomously
Emphasis is on mathematical knowledge put into functional use in a multitude of different situations in varied, reflective and insight-based ways
e.g. Relating well to others
Co-operating, working in teams
Managing and resolving conflicts
e.g. Acting within the bigger picture
Forming and conducting life plans
Taking responsibility and understanding rights and limits
To apply knowledge in real-life situations
To communicate thoughts and ideas effectively

15. Using “tools” interactively to engage with the world
To analyse, compare, contrast, and evaluate
e.g. Using language, symbols and texts
Interacting with information
Capitalising on the potential of technologies
To think imaginatively
Scientific literacy
Interacting in diverse groups
Acting autonomously
Using scientific knowledge, identifying scientific questions, and drawing evidence-based conclusions to understand and make decisions about the natural world
e.g. Relating well to others
Co-operating, working in teams
Managing and resolving conflicts
e.g. Acting within the bigger picture
Forming and conducting life plans
Taking responsibility and understanding rights and limits
To apply knowledge in real-life situations
To communicate thoughts and ideas effectively

16. Under development: PISA assessment of PISA self-reports on:
Using “tools” interactively to engage with the world
Under development:
PISA assessment of
Problem-solving skills
PISA self-reports on:
Dispositions to learning
Learning strategies
Engagement with school
Interacting in diverse groups
Acting autonomously

17. Not yet developed Using “tools” interactively to engage with the world
Interacting in diverse groups
Acting autonomously
Not yet developed

18. What the best performing countries show can be achieved.
Where we can be.
What the best performing countries show can be achieved.

19. Strong impact of social background on performance
High Performance
High performance
Low social equity
High performance
High social equity
Strong impact of social background on performance
Moderate impact of social background on performance
Low performance
Low social equity
Low performance
High social equity
Low Performance

20. Quality and equity can be achieved together
High Performance
Quality and equity can be achieved together
‘dumbing down’ is not an inevitable consequence of the pursuit of equity
‘levelling up’ is achievable (e.g. Finland, Korea, Canada)
High performance
Low social equity
High performance
High social equity
Strong impact of social background on performance
Moderate impact of social background on performance
Low performance
Low social equity
Low performance
High social equity
Low Performance .

21. Is it all innate ability? Variation in student performance
8 3-10
7 5-9
6 4-9
5 3-9 1

22. Is it all innate ability? Variation in student performance
8 3-10
7 5-9
6 4-9
5 3-9 1

23. Is it all innate ability? Variation in student performance
Variation of performance within schools
Variation of performance between schools
8 3-10
7 5-9
6 4-9
5 3-9 1

24. Policy levers.
Overall findings

25. Policy Levers Performance in reading
Students from advantaged backgrounds…
… have a greater chance of coming to school more engaged in reading and entering into a virtuous circle of increasing reading interest and improved reading performance
… but not all engaged students come from privileged homes…
… and those from more modest backgrounds who read regularly and feel positive about it are better readers than students with home advantages and weaker reading engagement
Schools can make a significant difference to bring students into the virtuous circle
Seeking mutual reinforcement of cognitive skills and motivation, particularly for boys

26. Policy Levers Student approaches to learning
The ability to manage one’s learning is both an important outcome of education and a contributor to student literacy skills at school
Learning strategies, motivation, self-related beliefs, preferred learning styles
Different aspects of students’ learning approaches are closely related
Well-motivated and self-confident students tend to invest in effective learning strategies and this contributes to their literacy skills
Immigrant students tend to be weaker performers
… but they do not have weaker characteristics as learners
Boys and girls each have distinctive strengths and weaknesses as learners
Girls stronger in relation to motivation and self-confidence in reading
Boys believing more than girls in their own efficacy as learners and in their mathematical abilities

27. Policy Levers Student engagement at school
An important outcome in itself
Disaffection at age 15 can potentially be a precursor to the onset of more serious problems among vulnerable young people
Engagement at age 15 is likely to influence students’ choices and educational pathways
The prevalence of disaffected students varies significantly across schools in each country
Only weak link to student’s social background – there is thus scope for school policy/practice to engage students
But strong link to school’s social background
Students in schools with strong average engagement tend to perform well
Engagement and performance seem to work complementary
The school climate seems to make more of a difference than resources
For individual students, strong performance does not necessarily ensure strong engagement at school
Relationship complex

28. Policy levers.
Some characteristics shared by some strongly performing countries Combining the empirical results obtained through PISA with qualitative information on the socio-cultural conditions and education policy strategies.

29. . High Performance High performance Low social equity High performance
High social equity
plus Netherlands
Strong impact of social background on performance
Moderate impact of social background on performance
Low performance
Low social equity
Low performance
High social equity
Low Performance .

30. Sympathy doesn’t raise standards – aspiration does
In the countries studied
National research teams report a strong “culture of performance”
Which drives students, parents, teachers and the educational administration to high performance standards
PISA shows that students perceived a high degree of teacher support
Which should not be simply equated with “achievement press”

31. Governance of the school system
In the countries studied…
Decentralised decision-making is combined with devices to ensure a fair distribution of substantive educational opportunities
The provision of standards and curricula at national/subnational levels is combined with advanced evaluation systems
That are implemented by professional agencies
Process-oriented assessments and/or centralised final examinations are complimented with individual reports and feed-back mechanisms on student learning progress

32. . High degree of school-level autonomy
High Performance
E.g. Learning environment and course offering
High degree of school-level autonomy
Low degree of school-level autonomy
% Variance between schools
11%
20% 9%
76% 7%
Strong impact of social background on performance
Moderate impact of social background on performance
75%
71%
r=.51
Low Performance .

33. Organisation of instruction
In the countries studied…
Schools and teachers have explicit strategies and approaches for teaching heterogeneous groups of learners
A high degree of individualised learning processes
Disparities related to socio-economic factors and migration are recognised as major challenges
Students are offered a variety of extra-curricular activities
Schools offer differentiated support structures for students
E.g. school psychologists or career counsellors
Institutional differentiation is introduced, if at all, at later stages
Integrated approaches also contributed to reducing the impact of students socio-economic background on outcomes

34. . High degree of integration Early selection and stratification
High Performance
Early selection and institutional stratification
High degree of integration
Early selection and stratification
Strong impact of social background on performance
Moderate impact of social background on performance
Low Performance .

35. Support systems and professional teacher development
In the countries studied…
Effective support systems are located at individual school level or in specialised support institutions
Teacher training schemes are selective
The training of pre-school personnel is closely integrated with the professional development of teachers
Continuing professional development is a constitutive part of the system
Special attention is paid to the professional development of school management personnel

36. . Students perceived teacher support High degree of support
High Performance
Students perceived teacher support
High degree of support
Low degree of support
Strong impact of social background on performance
Moderate impact of social background on performance
Low Performance .

37. Summary of common characteristics
Uniformity
Diversity
“hit and miss”
Universal high standards
“Inputs”
Outcomes
Bureaucratic
Devolved responsibility
Look up
Look outwards
Received wisdom
Data and best practice
Motivating feedback and incentivising success and innovation
Evaluation to control
Prescription
Informed profession

38. One challenge – different approaches
The future of education systems needs to be “knowledge rich”
Informed professional judgement, the teacher as a “knowledge worker”
Informed prescription
National prescription
Professional judgement
Uninformed prescription, teachers implement curricula
Uninformed professional judgement
The tradition of education systems has been “knowledge poor”

39. Further information www.oecd.org www.pisa.oecd.org
… and remember:
Without data, you are just another person with an opinion