Jenny Bradshaw NCETM National CPD Conference 23rd March 2011 A view of mathematics in England: selected findings from PISA and TIMSS Jenny Bradshaw NCETM National CPD Conference 23rd March 2011

What is PISA and who organises it? Programme for International Student Assessment The world's biggest international education survey, involving schools and students in 65 countries. An assessment of the knowledge and skills of 15- year-olds as they approach the end of compulsory education. OECD Organisation for Economic Co-operation and Development Aims to promote policies that will improve the economic and social well-being of people around the world Provides a forum for governments to work together to share experiences and seek solutions to common problems 34 member countries and 70+ partners

The PISA Cycle

A comprehensive assessment of the yield of education systems What does PISA produce? A comprehensive assessment of the yield of education systems including and beyond the curriculum Comparable outcome measures that can guide policy decisions and resource allocations Insights into the mix of factors which contribute to the development of knowledge and skills and how these factors operate similarly or differently across countries

What is involved in PISA? 65 countries in 2009 (up from 57 in 2006) 15-year-olds (Year 11) Tests of reading, science and mathematics Pupil and School Questionnaires PISA 2009: 165 schools out of a possible 190 took part in England (including independent and academies) Approx 4,000 pupils tested in England

Total number of countries 65 (57) Number of countries higher 27 (23) PISA 2009 : Mathematics OECD average 496 (498) England’s score 493 (495) Total number of countries 65 (57) Number of countries higher 27 (23) Significantly higher 20 (18) 2006 figures in brackets

Mathematics: countries significantly higher than England Shanghai-China 600 (~~~) Netherlands 526 (531) Singapore 562 (~~~) Macao-China 525 (525) Hong Kong-China 555 (547) New Zealand 519 (522) Korea 546 (547) Belgium 515 (520) Chinese Taipei 543 (549) Australia 514 (520) Finland 541 (548) Germany 513 (504) Liechtenstein 536 (525) Estonia 512 (515) Switzerland 534 (530) Iceland 507 (506) Japan 529 (523) Denmark 503 (513) Canada 527 (527) Slovenia 501 (504) Italics = non-OECD country 2006 figures in brackets NEW COUNTRIES HIGHER: SHANGHAI; SINGAPORE EXISTING COUNTRIES NOW OVERTAKING UK IN RANKINGS: NORWAY (SIGN HIGHER) ICELAND UNITED STATES GERMANY FRANCE DENMARK SLIPPING: KOREA SWEDEN REPUBLIC OF IRELAND

Mathematics – countries similar to England Norway 498 (490) France 497 (496) Slovak Republic 497 (492) Austria 496 (505) Poland 495 (495) Sweden 494 (502) England 493 (495) Czech Republic 493 (510) Hungary 490 (491) Luxembourg 489 (490) United States 487 (474) Ireland 487 (501) Portugal 487 (466) Italics = non-OECD country 2006 figures in brackets

Mathematics: OECD countries lower than England Spain 483 (480) Italy 483 (462) Greece 466 (459) Turkey 445 (424) Chile 421 (411) Mexico 410 (406) 25 non-OECD countries also lower Lowest scoring country: Kyrgyzstan, 331 Italics = non-OECD country 2006 figures in brackets

Comparisons within the UK – mean scores for Reading, Maths & Science England, Scotland and Northern Ireland not significantly different from one another across all three subjects Wales significantly lower than the other UK countries across all three subjects Eng Scot NI Wales R 495 500 499 476 M 493 492 472 S 515 514 511 496

Boys scored higher than girls boys 503 girls 483 Maths : other results Boys scored higher than girls boys 503 girls 483 Spread of attainment less than the average fewer at lowest levels, but also fewer at highest 80% achieved PISA level 2 or above OECD average 78%

Level 2 ... a baseline level of mathematics proficiency .. at which students begin to demonstrate the kind of … skills that enable them to actively use mathematics, which are considered fundamental for future development and use of mathematics OECD 2007

TIMSS 2007

TIMSS: Trends in International Mathematics and Science Survey 4-yearly international survey, maths and science Organised by the International Association for the Evaluation of Educational Achievement (IEA) UK participation (2007): England and Scotland 9-10 and13-14 year olds (G4/G8: Y5/Y9) Aims: inform educational policy; improve teaching and learning in maths and science UK participation (2011): England and Northern Ireland

Grade 4 Maths (Y5) 37 Countries TIMSS 2007 RESULTS Grade 4 Maths (Y5) 37 Countries 4 Countries Outscored England England outscored 28 other countries including: Score Hong Kong 607 United States 529 Singapore 599 Germany 525 Chinese Taipei 576 Denmark 523 Japan 568 Australia 516 Italy 507 England Part of a Group of 5 Countries Sweden 503 Kazakhstan 549 International average 500 Russian Federation 544 Scotland 494 England 541 New Zealand 492 Latvia 537 Netherlands 535

Grade 8 Maths (Y9) 50 Countries TIMSS 2007 RESULTS Grade 8 Maths (Y9) 50 Countries 5 Countries Outscored England England outscored 39 other countries including: Score Chinese Taipei 598 International average 500 Korea 597 Australia 496 Singapore 593 Sweden 491 Hong Kong 572 Scotland 487 Japan 570 Italy 480 England Part of a Group of 6 Countries Norway 469 Hungary 517 England 513 Russian Federation 512 United States 508 Lithuania 506 Czech Republic 504

EXAMPLES OF QUESTIONS GRADE 4 TIMSS 2007 EXAMPLES OF QUESTIONS GRADE 4

TIMSS 2007 M04_01

TIMSS 2007 M02-04

TIMSS 2007 M01-02

TIMSS 2007 M04-09A

EXAMPLES OF QUESTIONS GRADE 8 TIMSS 2007 EXAMPLES OF QUESTIONS GRADE 8

TIMSS 2007 M02-01

TIMSS 2007 M07-02

TIMSS 2007 M02-14

PISA 2003 EXAMPLES OF QUESTIONS

Percentage correct OECD average 54 United Kingdom 52 Hong Kong 47%

Percentage correct OECD average 29 United Kingdom 34 Hong Kong 27%

Percentage correct OECD average 47 United Kingdom 43 Hong Kong 75%

Percentage correct OECD average 50 United Kingdom 61 Hong Kong 72%

Percentage correct OECD average 52 United Kingdom 65 Hong Kong 68%

TIMSS and PISA Assessment Frameworks

ASSESSMENT FRAMEWORK TIMSS 2007 Grade 4 Grade 8 Content domains Number 50% 30% Geometric shapes and measures 35% Algebra Data display 15% Geometry 20% Data and chance Cognitive domains Knowing 40% Applying Reasoning 25%

ASSESSMENT FRAMEWORK – MATHS G4 TIMSS 2007 ASSESSMENT FRAMEWORK – MATHS G4 Number Whole numbers Fractions and decimals Number sentences Patterns and relationships Data display Reading and interpreting Organising and representing Geometric shapes and measures Lines and angles 2- and 3- dimensional shapes Location and movement

ASSESSMENT FRAMEWORK – MATHS G8 TIMSS 2007 ASSESSMENT FRAMEWORK – MATHS G8 Number Whole numbers Fractions and decimals Integers Ratio, proportion and percent Algebra Patterns Algebraic expressions Equations/formulas and functions Data and chance Data organisation and representation Data interpretation Chance Geometry Geometric shapes Geometric measurement Location and movement

ASSESSMENT FRAMEWORK – Maths cognitive domains, both grades TIMSS 2007 ASSESSMENT FRAMEWORK – Maths cognitive domains, both grades Knowing Recall Recognise Compute Retrieve Measure Classify/order Applying Select Represent Model Implement Solve routine problems Reasoning Analyse Generalise Synthesise/integrate Justify Solve non-routine problems

PISA 2003 Assessment framework: definition Mathematical Literacy is an individual's capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to engage in mathematics, in ways that meet the needs of that individual's life as a constructive, concerned, and reflective citizen.

PISA 2003 Framework : 4 ‘Overarching Ideas’ Quantity Space and shape Change and relationships Uncertainty Even distribution across the four ideas

PISA 2003 Framework: Competency clusters Distribution 1:2:1

Main focus - Mathematics Questions have been devised PISA 2012 Main focus - Mathematics Questions have been devised Field Trials March 2011 Main Study Nov 2012 International report Dec 2013 (OECD) National reports Dec 2013 Scottish Government: Scotland NFER: England, Wales, Northern Ireland

Head of Centre for International Comparisons Contacts Jenny Bradshaw Head of Centre for International Comparisons National Foundation for Educational Research j.bradshaw@nfer.ac.uk PISA websites http://www.nfer.ac.uk/pisa http://www.pisa.oecd.org TIMSS websites http://timss.bc.edu/