Seeing schools through the prism of PISA An international comparative perspective on quality and equity in education systems Organisation for Economic.

Slides:



Advertisements
Similar presentations
OECD Programme for International Student Assessment (PISA)
Advertisements

Individual, Home, and Community Factors PISA/PIRLS Task Force International Reading Association January 2005.
Potential impact of PISA
Equity - Research Reveals the What, the Where and the How November 21, 2011.
LITERACY IN THE MIDDLE YEARS OF SCHOOLING INITIATIVE
Wynne Harlen. What do you mean by assessment? Is there assessment when: 1. A teacher asks pupils questions to find out what ideas they have about a topic.
A “Best Fit” Approach to Improving Teacher Resources Jennifer King Rice University of Maryland.
Why this Research? 1.High School graduates are facing increased need for high degree of literacy, including the capacity to comprehend texts, but comprehension.
24 July 2014 PISA 2012 Financial Literacy results – New Zealand in an international context.
What is PIAAC?. About PIAAC PIAAC is an international large-scale assessment administered in in 23 countries It assessed 16 - to 65-year-olds,
What light does PISA shed on student learning? Selected results from PISA 2003 Organisation for Economic Cooperation and Development (OECD) Paris 2 February.
Organisation for Economic Cooperation and Development (OECD)
The teacher profile for the future Organisation for Economic Cooperation and Development (OECD) ETUCE Conference Europe Needs Teachers Brussels 12 June.
OECD Programme for International Student Assessment (PISA)
PISA OECD Programme for International Student Assessment Science Competencies for Tomorrow’s World Science competencies for tomorrow’s world Seeing school.
American Diploma Project 11 September 2009 Andreas Schleicher International Benchmarking International Benchmarking What it means – what it takes Washington,
May 2015 Andreas Schleicher Skills Outlook 2015 Youth, Skills and Employability.
Home, school & community partnerships Leadership & co-ordination Strategies & targets Monitoring & assessment Classroom teaching strategies Professional.
Margaret J. Cox King’s College London
Programme for International Student Assessment - PISA Organisation for Economic Cooperation and Development (OECD)
Raising Academic Standards for all School Development Planning Initiative.
Interstate New Teacher Assessment and Support Consortium (INTASC)
PIAAC results tell a story about the systemic nature of the skills deficit among U.S. adults. Overview of U.S. Results: Focus on Numeracy.
Raising the Quality of Educational Outcomes and Improving E-learning: Cross-national Evidence on Challenges and Opportunities Judit Kádár-Fülöp Indicators.
ECS Forum on National Education Policy Austin, 2 July 2008 Benchmarking Internationally The Need Confronts Reality ECS Forum on National Education Policy.
International Outcomes of Learning in Mathematics and Problem Solving: PISA 2003 Results from the U.S. Perspective Commissioner Robert Lerner National.
Civic and Citizenship Education in Times of Change: Curriculum and its Implementation Some Results of the IEA Studies Civic Education in Iraq: Study Tour.
Agnieszka Chłoń-Domińczak Michał Federowicz How did educational reforms in Poland lead to significant progress in PISA ?
Knowledge Economy Forum Organisation for Economic Cooperation and Development (OECD) Results from PISA 2003 Istanbul March 22, 2005 Dr John Cresswell OECD/
Improving both quality and equity Hong Kong, 21 November 2003 Andreas Schleicher Head, Indicators and Analysis Division OECD OECD Programme for International.
Overview of U.S. Results: Focus on Literacy PIAAC results tell a story about the systemic nature of the skills deficit among U.S. adults.
PISA OECD Programme for International Student Assessment PISA for Development Andreas Schleicher Paris, 27 June 2013 Welcome PISA for Development Andreas.
OECD World Forum “Statistics, Knowledge and Policy”, Palermo, November
Key Findings of the OECD Policy Review of Migrant Education and Implications for Language Education Policy for Immigrant Children Miho Taguma Project Manager.
Leonardo Menchini, UNICEF Innocenti Research Centre Poverty and inequality among children in economically advanced.
A focus on student outcomes Key influences on enhancing student outcomes System wide lasting and deep change Knowledge and understanding Capacity and.
2007 by The Education Trust, Inc. Program for International Student Assessment (PISA) 2006 Prepared by the Education Trust December 2007.
Quality and equity in educational outcomes Seeing school systems through the prism of PISA Organisation for Economic Cooperation and Development (OECD)
Education and educational policy in the enlarged EU – the case of the Czech Republic NATIONAL CONTEXT New school act equity as a priority self evaluation.
Education at a Glance 2004 – Andreas Schleicher Organisation for Economic Co-Operation and Development I risultati dell’Italia nell’indagine OCSE “Education.
Educational standards and economic and social development Organisation for Economic Cooperation and Development (OECD) Lisbon Council Brussels, 14 September.
Student learning outcomes from a gender perspective What do international assessments tell us? Organisation for Economic Cooperation and Development (OECD)
VISTA 19 August October 2005 Park Hyatt, Melbourne Assessment & Reporting Dr Andreas Schleicher – OECD Key Note Address 1.
PISA International Conference. Reading Performance of Hong Kong’s 15-Year-Old Students in PISA.
In the dark all schools and education systems look the same… But with a little light….
LEVEL 3 I can identify differences and similarities or changes in different scientific ideas. I can suggest solutions to problems and build models to.
CERI/OECD “Improving Learning through Formative Assessment” 3 February, 2005.
The background of the improvement of PISA results in Hungary Trends in Performance Since 2000 International Launch of PISA 2009 Report February 10 th,
Incentives and stimuli: OECD TALIS (Teaching and Learning International Survey) Improving Quality in Education: OECD - MEXICO Joint Conference Mexico City,
Israel Accession Seminar PIAAC: Programme for International assessment of Adult Competencies Skills strategy in OECD Programme for the International Assessment.
Responsive Education Systems and Skills for the Knowledge Economy Using education as a lever to compete by working smarter, rather than working harder.
Using the results of International Reading Tests to Inform Good Practice and Policy Thomas Schmit and Eva Birzniece, Pro Futuro, Latvia 31 July 2008 THE.
Andreas Schleicher, OECD Directorate for Education, 2003 Organisation for Economic Co-Operation and Development Literacy Skills for the World of Tomorrow.
1 United States Education at a Glance 2015 Andreas Schleicher Director for Education and Skills Release date: 24 November 2015.
Andreas Schleicher, Quality of Education – Teachers’ Professional Training and Development, Athens, 2003 Organisation for Economic Co-Operation and Development.
International Large-Scale Assessments – Best practice and what are they good for? Dirk Hastedt, IEA Moscow, October 2015.
Results from the 2009 Programme for International Student Assessment (PISA): How does the United States compare to other nations? December 2010.
Image: dan / FreeDigitalPhotos.net Deborah Nusche Early Childhood and Schools Division Directorate for Education, OECD Education for Innovative Societies,
11 PIRLS The Trinidad and Tobago Experience Regional Policy Dialogue on Education 2-3 December 2008 Harrilal Seecharan Ministry of Education Trinidad.
1 Perspectives on the Achievements of Irish 15-Year-Olds in the OECD PISA Assessment
1 Main achievement outcomes continued.... Performance on mathematics and reading (minor domains) in PISA 2006, including performance by gender Performance.
Commission on School Reform 13th April OECD - some positives  Above average levels in PISA science and reading  Around average in PISA maths 
What is PIAAC?.
Programme for International Student Assessment
PISA 2009 – New Approaches to Assessing Reading Literacy
Integrating Transversal Competencies in Policy and Practice
OECD Strategic Education Governance A perspective for Scotland
FINLAND AT A GLANCE independent since 1917, member of the European Union since 1995 total area 338,000 km2, population 5.2 million (17 inhabitants / km2)
Organisation for Economic Co-Operation and Development Indicators on the Quality of Educational Performance Quality of Education Teachers’ Professional.
Improving both quality and equity
Presentation transcript:

Seeing schools through the prism of PISA An international comparative perspective on quality and equity in education systems Organisation for Economic Cooperation and Development (OECD) Tokyo 24 June 2005 Andreas Schleicher Head, Indicators and Analysis Division OECD Directorate for Education

In the dark, all education systems look the same… But with a little light…. 暗がりのなかでは、どの学校も教育システムも同じように見える … だが、少し光を当てると ….

But with a little light…. …important differences become apparent… … 重要な違いが明らかになってくる …. だが、少し光を当てると ….

Average performance of 15-year-olds in mathematics High mathematics performance Low mathematics performance

Overview

The PISA approach Measuring the quality of learning outcomes

Three broad categories of key competencies Using “tools” interactively to engage with the world Acting autonomously Interacting in diverse groups e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Learning strategies Taking responsibility and understanding rights and limits To analyse, compare, contrast, and evaluate To think imaginatively To apply knowledge in real-life situations To communicate thoughts and ideas effectively PISA concept of literacy Accessing, managing, integrating and evaluating written information in order to develop ones knowledge and potential, and to participate in, and contribute to, society

Using “tools” interactively to engage with the world Acting autonomously Interacting in diverse groups e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To analyse, compare, contrast, and evaluate To think imaginatively To apply knowledge in real-life situations To communicate thoughts and ideas effectively Reading literacy Using, interpreting and reflecting on written material

Using “tools” interactively to engage with the world Acting autonomously Interacting in diverse groups e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To analyse, compare, contrast, and evaluate To think imaginatively To apply knowledge in real-life situations To communicate thoughts and ideas effectively Scientific literacy Using scientific knowledge, identifying scientific questions, and drawing evidence-based conclusions to understand and make decisions about the natural world

Using “tools” interactively to engage with the world Acting autonomously Interacting in diverse groups e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To analyse, compare, contrast, and evaluate To think imaginatively To apply knowledge in real-life situations To communicate thoughts and ideas effectively Mathematical literacy Emphasis is on mathematical knowledge put into functional use in a multitude of different situations in varied, reflective and insight-based ways

Mathematical literacy in PISA The real world The mathematical World A real situation A model of reality A mathematical model Mathematical results Real results Understanding, structuring and simplifying the situation Making the problem amenable to mathematical treatment Interpreting the mathematical results Using relevant mathematical tools to solve the problem Validating the results

Deciding what to assess... looking back at what students were expected to have learned …or… looking ahead to what they can do with what they have learned. For PISA, the OECD countries chose the latter.

Development of assessments r Frameworks by international experts r Assessment materials submitted by countries developed by research consortium screened for cultural bias –by countries –by expert, international panel –items with prima facie cultural bias removed at this stage internationally validated translations trialled to check items working consistently in all countries r Final tests items shown in trial to be culturally biased removed best items chosen for final tests –balanced to reflect framework –range of difficulties –range of item types (constructed response, multiple choice)

Key features of PISA 2003 r Information collected volume of questions –3½ hours of mathematics assessment –1 hour for each of reading, science and problem solving each student –2 hours on paper-and-pencil tasks (subset of all questions) –½ hour for questionnaire on background, learning habits, learning environment, engagement and motivation school principals –questionnaire (school demography, learning environment quality ) r Coverage PISA covers roughly nine tens of the world economy

Where we are - and where we can be What PISA shows students can do Examples of the best performing countries

PISA provides five key benchmarks for the quality of education systems

Average performance of 15-year-olds in mathematics Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High mathematics performance Low mathematics performance Top-performers Finland remained first in reading and since 2000 moved further in math and science… …and is now on a par with the East Asian countries that were previously unmatched in math and science Also the Netherlands is among the top-performers in math …though not in reading and science. As is the Flemish Community of Belgium Progress Other countries with improvements in at least two assessment areas were Belgium, the Czech Republic and Germany …In Belgium and Germany it was the top performers who drove improvements. Progress Poland raised it’s overall performance in all four assessment areas …thanks to big improvements among lower-performing students in the wake of a major reform in A widening gap More improvement at the top of the scale has widened the gap between the top and bottom performers in the OECD.

Durchschnittliche Schülerleistungen im Bereich Mathematik Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High mathematics performance Low mathematics performance Differences in socio-economic background pose major challenges for education systems Students whose parents have better-paid jobs, are better educated or have more “cultural” possessions in their homes tend to perform better… …But the performance advantage varies –Australia, Canada, Finland, Iceland and Japan provide examples showing that it is possible to combine quality and equity –In contrast, results for Belgium, Germany, Hungary and the Slovak Republic reveal large socio-economic inequalities in the distribution of learning opportunities.

Is it all innate ability? Variation in student performance OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p

Variation of performance between schools Variation of performance within schools Is it all innate ability? Variation in student performance in mathematics OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383. In some countries, parents can rely on high and consistent standards across schools In Canada, Denmark, Finland, Iceland and Sweden average student performance is high… …and largely unrelated to the individual schools in which students are enrolled

Student performance School performance and schools’ socio- economic background - Germany Advantage PISA Index of social background Disadvantage Figure 4.13 School proportional to size Student performance and student SES Student performance and student SES within schools School performance and school SES

生徒の成績 学校の成績と学校の社会経済的背景(日本) 有利 社会的背景に関する PISA 指数 不利 生徒の成績と生徒の社会経済的地位( SES ) 生徒の成績と学校内における生徒の SES 学校の成績と学校の SES 学校の規模

Student performance School performance and schools’ socio- economic background - Norway Advantage PISA Index of social background Disadvantage Figure 4.13 School proportional to size Student performance and student SES Student performance and student SES within schools School performance and school SES OECD

Student performance School performance and schools’ socio- economic background - Finland Advantage PISA Index of social background Disadvantage Figure 4.13 Student performance and student SES Student performance and student SES within schools School performance and school SES School proportional to size

Number of school types orprogrammesProportion of 15-year-oldsin programmes leading tovocational education/workFirst age of selection in theeducation systemRepeaters in primaryeducationProportion of repeaters inlower secondary educationProportion of repeaters inupper secondary educationPerformance on themathematics scale – meanscorePerformance in mathematics– standard deviationBetween school varianceRelationship between sesand performance Number of school types or programmes1 Proportion of 15-year-olds in programmes leading to vocational education/work0.501 First age of selection in the education system Repeaters in primary education Proportion of repeaters in lower secondary education Proportion of repeaters in upper secondary education Performance on the mathematics scale – mean score Performance in mathematics – standard deviation Between school variance Relationship between ses and performance Cross-country correlations with structural features

Student performance School performance and schools’ socio- economic background Advantage PISA Index of social background Disadvantage Figure 4.13 School proportional to size Universal policies Increasing educational performance of all children through reforms applied equally across the school system, e.g. –Altering content or pace of curriculum –Improving instructional techniques –Changing the learning environment in schools and classrooms –Standards and accountability –Teacher professional development

Student performance School performance and schools’ socio- economic background Advantage PISA Index of social background Disadvantage Figure 4.13 School proportional to size Socio-economically targeted policies Providing a specialised curriculum or additional educational resources to students from disadvantaged backgrounds –Students are often also identified through other risk factors, e.g. immigration, ethnicity, low- income community

Student performance School performance and schools’ socio- economic background Advantage PISA Index of social background Disadvantage Figure 4.13 School proportional to size Compensatory policies Providing additional economic resources to students from disadvantaged backgrounds –Different to socio-economically targeted policies, efforts are directed to ameliorating economic circumstances, rather than providing specialised curriculum or additional educational resources

Student performance School performance and schools’ socio- economic background Advantage PISA Index of social background Disadvantage Figure 4.13 School proportional to size Performance targeted policies Providing additional economic resources to students based on their academic performance –Early intervention programmes –Remedial and recovery programmes –Performance-based tracking or streaming Countries with flat gradients In combination with SES-targeted policies for countries with steeper gradients

Gender differences r In reading, girls are far ahead l In all countries, girls significantly outperform boys in reading r In mathematics, boys tend to be somewhat ahead l In most countries, boys outperform girls …but mostly by modest amounts… …and mainly because boys are overrepresented among top- performers while boys and girls tend to be equally represented in the “at risk” group –Within classrooms and schools, the gender gap is often larger l Strong problem-solving performance for girls suggests… …that it is not the cognitive processes underlying mathematics that give boys an advantage… …but the context in which mathematics appears in school l Gender differences in interest and attitudes towards mathematics are significantly greater than the observed performance gap –Girls report much lower intrinsic (though not instrumental) motivation in mathematics, more negative attitudes and much greater anxiety with mathematics… …and this may well contribute to the significant gender difference in educational and occupational pathways in mathematics-related subjects

Interest in and enjoyment of mathematics OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 3.4, p.367 and Figure 3.4, p.126.

Instrumental motivation in mathematics OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 3.2a, p.360 and Figure 3.3a, p.122.

Anxiety in mathematics OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 3.8, p.374 and Figure 3.8, p.139.

Attitudes towards school OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 3.4, p.367 and Figure 3.4, p.126.

Low Performance High Mathematics performance Low performance Low social equity High performance Low social equity Low performance High social equity High performance High social equity Strong impact of social background on performance Moderate impact of social background on performance                 Student anxiety in mathematics High degree of anxiety Low degree of anxiety  

How can we get there? Levers for policy that emerge from international comparisons

OECD framework National educ, social and economic context Structures, resource alloc and policies Social & economic outcomes of education Community and school characteristics Student learning, teacher working conditions Socio-economic background of learners Antecedents contextualise or constrain ed policy The learning environment at school Teaching, learning practices and classroom climate Individ attitudes, engagement and behaviour Output and performance of institutions Quality of instructional delivery Quality and distribution of knowledge & skills Policy Levers shape educational outcomes Outputs and Outcomes impact of learning Individual learner Level A Instructional settings Level B Schools, other institutions Level C Country or system Level D Domain 3Domain 2 Domain 1

Money matters but other things do too Mexico Greece Portugal Italy Spain Germany Austria Ireland United States Norway Korea Czech republic Slovak republic Poland Hungary Finland Netherlands Canada Switzerland Iceland Denmark France Sweden Belgium Australia Japan R 2 = 0.28 Cumulative expenditure (US$) Performance in mathematics r Spending per student is positively associated with average student performance… …but not a guarantee for high outcomes Australia, Belgium, Canada, the Czech Republic, Finland, Japan, Korea and the Netherlands do well in terms of “value for money”… …while some of the big spenders perform below-average

Sympathy doesn’t raise standards – aspiration does r In the focus countries National research teams report a strong “culture of performance” –Which drives students, parents, teachers and the educational administration to high performance standards r PISA suggests… …that students and schools perform better in a climate characterised by high expectations and the readiness to invest effort, the enjoyment of learning, a strong disciplinary climate, and good teacher-student relations –Among these aspects, students’ perception of teacher-student relations and classroom disciplinary climate display the strongest relationships

Governance of the school system r In the focus countries Decentralised decision-making is combined with devices to ensure a fair distribution of substantive educational opportunities The provision of standards and curricula at national/subnational levels is combined with advanced evaluation systems –That are implemented by professional agencies Process-oriented assessments and/or centralised final examinations are complimented with individual reports and feed-back mechanisms on student learning progress r Standard setting and equity-related goals Key objectives: –Raise educational aspirations, establish transparency over educational objectives, reference framework for teachers Approaches range from definition of broad educational goals up to formulation of concise performance expectations Some countries go beyond establishing educational standards as mere yardsticks and use performance benchmarks that students at particular age or grade levels should reach Instruments –Minimum standards, targets defining excellence, normative performance benchmarks r Monitoring and equity-related goals Diverging views how evaluation and assessment can and should be used –Some see them primarily as tools to reveal best practices and identify shared problems in order to encourage teachers and schools to improve and develop more supportive and productive learning environments –Others extend their purpose to support contestability of public services or market-mechanisms in the allocation of resources –e.g. by making comparative results of schools publicly available to facilitate parental choice or by having funds following students Differences in type of performance benchmarks being used and reported for the various stakeholders involved, including parents, teachers and schools

Durchschnittliche Schülerleistungen im Bereich Mathematik Low average performance Large socio-economic disparities High average performance Large socio-economic disparities Low average performance High social equity High average performance High social equity Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High mathematics performance Low mathematics performance

Durchschnittliche Schülerleistungen im Bereich Mathematik Strong socio- economic impact on student performance Socially equitable distribution of learning opportunities High mathematics performance Low mathematics performance School with responsibility for deciding which courses are offered High degree of autonomy Low degree of autonomy

Public and private schools Private schools perform better Public schools perform better

Organisation of instruction r In the focus countries Schools and teachers have explicit strategies and approaches for teaching heterogeneous groups of learners –A high degree of individualised learning processes –Disparities related to socio-economic factors and migration are recognised as major challenges Students are offered a variety of extra-curricular activities Schools offer differentiated support structures for students –E.g. school psychologists or career counsellors Institutional differentiation is introduced, if at all, at later stages –Integrated approaches also contributed to reducing the impact of students socio-economic background on outcomes

Low Performance High Mathematics performance Low performance Low social equity High performance Low social equity Low performance High social equity High performance High social equity Strong impact of social background on performance Moderate impact of social background on performance Early selection and stratification High degree of stratification Low degree of stratification

Support systems and professional teacher development r In the focus countries Effective support systems are located at individual school level or in specialised support institutions Teacher training schemes are selective The training of pre-school personnel is closely integrated with the professional development of teachers Continuing professional development is a constitutive part of the system Special attention is paid to the professional development of school management personnel

Teacher support in mathematics Students’ views OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 5.1a, p.403 and Figure 5.1, p.213.

Student-related factors affecting school climate Principals’ views OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 5.2a, p.406 and Figure 5.2, p.216.

Teacher-related factors affecting school climate Principals’ views OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 5.4a, p.410 and Figure 5.4, p.220.

Pre-school attendance and performance Percentage of students who attended pre-school Difference in performance between those who attended pre-school for more than one year and those with no pre-school 38 score points is the average performance difference associated with one school year

Creating a knowledge-rich profession in which schools and teachers have the authority to act, the necessary knowledge to do so wisely, and access to effective support systems The tradition of education systems has been “knowledge poor” The future of education systems needs to be “knowledge rich” National prescription Professional judgement Informed professional judgement, the teacher as a “knowledge worker” Informed prescription Uninformed professional judgement Uninformed prescription, teachers implement curricula

OECD countries participating from PISA 2000 OECD countries participating from PISA from 2003 OECD partner countries participating from PISA 2000 OECD partner countries participating from PISA 2003 OECD partner countries participating from PISA 2006 PISA – Participating Countries

Further information –All national and international publications –The complete micro-level database …and remember: Without data, you are just another person with an opinion

Space & shape item Answers: Yes, No, Yes, Yes Process skill: connections Context: educational quasi-realistic problem typical in maths classes not genuine occupational problem Form: complex multiple-choice Source: OECD (2004) Learning for tomorrow’s world: First results from PISA 2003, Figure 2.4a, p.52.

Change and relationships item Scores: 1 for n = 140x0.8 = 112 but no further work shown 2 for correct steps/min but not m/min; correct m/min but not km/hr; correct method but error of calculation; correct km/hr but not giving m/sec 3 for correct m/min (89.6) and m/hr (5.4), rounding acceptable. Process skill: score 1=connections score 2=connections score 3=reflection Context: personal Form: open-constructed

Quantity item Form: open constructed response Answer: Yes, with adequate explanation Process skill: reflection Context: public Form: short constructed response Answer: ZAR (unit not required) Process skill: reproduction Context: public Form: short constructed response Answer: 975 SGD (unit not required) Process skill: reproduction Context: public

Uncertainty item Scores: 1 for “No, not reasonable” but explanation lacking detail (e.g. focusing on exact increase in number of robberies without comparison with total) 2 for “No, not reasonable” with argument focusing on only small part of graph shown, ratio or percentage increase, or need for trend data. Process skill: connections Context: personal Form: open- constructed