1 PISA 2012

What is PISA?  International large-scale assessment organized by the Organization for Economic Cooperation and Development (OECD)  NCES is U.S. coordinating and funding organization  PISA Governing Board (PGB) determines policy  Assessment of 15-year-old students  Administered every 3 years since

 In 2012, 68 participating education systems  Three U.S. states: Connecticut, Florida, and Massachusetts 3 What is PISA?

 Assessment subjects:  Mathematics, science, reading literacy  Mathematics was main subject assessed in 2012  Computer-based mathematics and reading literacy  Optional assessment taken by 32 education systems  Content knowledge, not limited to school-based curricula  PISA assesses applied knowledge/literacy:  “How well can students nearing the end of compulsory schooling apply their knowledge to real- life situations?” 4 What is PISA?

Average Scores: Scale of for all domains Proficiency Levels: Percentages of students scoring at 6 levels Trends: Change between average scores in 2012 and the scores in previous assessment years Subgroup scores: International (e.g., gender, language spoken in home) and U.S. specific variables (e.g., race/ethnicity) 5 What PISA Reports

6 U.S PISA Findings

 Ranked better in reading literacy than in mathematics and science literacy  Below OECD average score in mathematics only  Higher percentage at low proficiency levels than OECD average in mathematics only  Lower percentage of top performers than OECD average in mathematics only  No measurable change in average scores in mathematics, science, or reading literacy 7 General patterns of U.S. results

8 Mathematics Literacy

Quantity (25%) Are 15-year-olds able to comprehend multiple representations of numbers, engage in mental calculation, employ estimation, and assess the reasonableness of results? Change and Relationships (25%) Can students model change and relationships with the appropriate functions and equations? Space and Shape (25%) Uncertainty and Data (25%) Can students understand perspective, create and read maps, and manipulate 3D objects? Can students use probability and statistics and other techniques of data representation and description to mathematically describe, model, and interpret uncertainty? Note: Percentages shown are the approximate percentage of scale score points. 9 PISA 2012 mathematics literacy content categories

Formulate (25%) Can 15-year-olds recognize and identify opportunities to use mathematics and then provide mathematical structure to a problem presented in some contextualized form in order to formulate situations mathematically? Interpret (25%) Employ (50%) Can students interpret, apply and evaluate mathematical outcomes in order to determine whether results are reasonable and make sense in the context of the problem? Are students able to employ mathematical concepts, facts, procedures, and reasoning to solve mathematically-formulated problems and obtain mathematical conclusions? 10 Note: Percentages shown are the approximate percentage of scale score points. PISA 2012 mathematics literacy process categories

11 Identify information and carry out routine procedures according to direct instructions in explicit situations. Employ basic algorithms, formulae, procedures, or conventions. Capable of direct reasoning and making literal interpretations of the results. Execute clearly described procedures, including those that require sequential decisions. Select and apply simple problem- solving strategies. Work effectively with explicit models that may involve constraints or making assumptions. Capable of reasoning with some insight in straightforward contexts. Work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representa- tions, symbolic and formal characteriza- tions and insight pertaining to these situations. Apply insight along with a mastery of symbolic and formal mathematical operations and relationships to develop new approaches and strategies for attacking novel situations. Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 PISA mathematics literacy proficiency levels

12 Question: CHARTS Proficiency level 1 Mathematics literacy – Example item

13 Question: CHARTS - Continued Percentage of students answering correctly in 2012: OECD average: 80% U.S.: 77% Mathematics literacy – Example item

14 Question: DRIP RATE Proficiency level 5 Mathematics literacy – Example item

15 Question: DRIP RATE - Continued Percentage of students answering correctly in PISA 2012: OECD average: 26% U.S.: 30% Mathematics literacy – Example item

Source: Of 34 OECD Countries, U.S.A. Ranks 27 th in Mathematics Literacy U.S. OECD Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average National Center for Education Statistics, 2013,

Average higher than U.S. average Average not measurably different from U.S. average Average lower than U.S. average 17 U.S. 15-year-olds OECD average in mathematics literacy MA 514 (*,**) CT 506 (*) FL 467 (*,**) * = State avg. different from U.S. ** = State avg. different from OECD

Massachusetts ConnecticutFlorida Number of education systems higher than state Number of education systems not measurably different from state Number of education systems lower than state Note: Comparisons include 65 education systems. State results in mathematics literacy

SubscaleU.S. average scoreOECD average score Quantity Uncertainty and data Change and relationships Space and shape Not measurably different from U.S. average Significantly higher than U.S. average U.S. not measurably different from OECD average in two mathematics content subscales

20 In mathematics literacy, 9 percent of U.S. 15-year-old students scored at proficiency level 5 or above Percentage higher than U.S. Percentage not measurably different from U.S. Percentage lower than U.S. * = State percentage different from U.S. ** = State percentage different from OECD MA 19% (*,**) CT 16% (*,**) FL 6% (*,**)

21 No measurable change since 2009 in average mathematics literacy scores in more than half of PISA education systems, including U.S. Of the 62 education systems in PISA 2009 and 2012: o In 33 (including the U.S.) no measurable change in average scores o In 18 average scores increased  Russian Federation was below U.S. and OECD averages in 2003; not measurably different in 2012  Poland was lower than OECD and not measurably different from U.S. averages in 2003; higher than both in 2012 o In 11 average scores declined  Finland declined between and again between ; scored 548 in 2006 and 519 in 2012  Norway was above U.S. average in 2003; not measurably different from U.S. average and below OECD average in 2012

22 Science Literacy

23 Present scientific explanations that are obvious and follow explicitly from given evidence. Use direct reasoning and make literal interpretations of the results of scientific inquiry or technological problem solving. Identify clearly described scientific issues in a range of contexts. Select facts and knowledge to explain phenomena and apply simple models or inquiry strategies. Select and integrate explanations from different disciplines of science or technology and link those explanations directly to aspects of life situations. Use well- developed inquiry abilities, link knowledge appropriately, and bring critical insights to situations. Construct explanations based on evidence and arguments based on critical analysis. Link different information sources and explanations and use evidence from those sources to justify decisions. Demonstrate advanced scientific thinking and reasoning, and use scientific understanding in support of solutions to unfamiliar scientific situations. Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 PISA science literacy proficiency levels

24 Question: MARY MONTAGU Proficiency level 2 Science literacy – Example item

25 Question: MARY MONTAGU - Continued Percentage of students answering correctly in 2006: OECD average: 75% U.S.: 73% Science literacy – Example item

26 Question: GREENHOUSE Proficiency level 6 Science literacy – Example item

27 Question: GREENHOUSE - Continued Percentage of students answering correctly in 2006: OECD average: 19% U.S.: 18% Science literacy – Example item

Source: National Center for Education Statistics, 2013, 28http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_4a.asp Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average Of 34 OECD Countries, U.S.A. Ranks 20 th in Science Literacy U.S.

Average higher than U.S. average Average not measurably different from U.S. average Average lower than U.S. average 29 U.S. 15-year-old students not measurably different from OECD average in science literacy MA 527 (*,**) CT 521 (*,**) FL 485 (**) * = State avg. different from U.S. ** = State avg. different from OECD

Massachusetts ConnecticutFlorida Number of education systems higher than state 6726 Number of education systems not measurably different from state 1415 Number of education systems lower than state Note: Comparisons include 65 education systems. State results in science literacy

31 Percentage higher than U.S. Percentage not measurably different from U.S. Percentage lower than U.S. In science literacy, 7 percent of U.S. 15-year-old students scored at proficiency level 5 or above * = State percentage different from U.S. ** = State percentage different from OECD MA 14% (*,**) CT 13% (*,**) FL 5% (**)

Of the 62 education systems that participated in PISA 2009 and 2012: o In 43 (including the U.S.) there was no measurable change in average scores o In 13 average scores increased  Poland was not measurably different from U.S. and OECD averages in 2006; above U.S. and OECD averages in 2012 o In 6 average scores declined  Sweden was higher than U.S. and OECD averages in 2006; below U.S. and OECD averages in No measurable change since 2009 in average science literacy scores in more than half of PISA education systems, including the U.S.

33 Reading Literacy

34 Locate a single piece of explicitly stated information in a prominent position in a short, simple text. Recognize the main theme or author’s purpose in a text about a familiar topic. Locate one or more pieces of information, which may need to be inferred and may need to meet several conditions. Integrate several parts of a text in order to identify a main idea, understand a relationship, or construe the meaning of a word or phrase. Interpret the meaning of nuances of language in a section of text by taking into account the text as a whole. Locate and organize several pieces of deeply embedded information, inferring which information in the text is relevant. Make multiple inferences, compari- sons, and contrasts that are both detailed and precise. Deal with unfamiliar ideas and generate abstract categories for interpreta- tions. Level 1b Level 1a Level 2 Level 3 Level 5 Level 6 Level 4 PISA reading literacy proficiency levels

Reading literacy – Example item 35 Question: TELECOMMUTING Proficiency level 3

36 Question: TELECOMMUTING-Continued Percentage of students answering correctly in 2009: OECD average: 52% U.S.: 55% Reading literacy – Example item

37 Question: THE PLAY’S THE THING Proficiency level 6 Reading literacy – Example item

38 Question: THE PLAY’S THE THING- Continued What were the characters in the play doing just before the curtain went up? Reading literacy – Example item Percentage of students answering correctly in PISA 2009: OECD average: 13% U.S.: 13%

Source: National Center for Education Statistics, 2013, Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average Of 34 OECD Countries, U.S.A. Ranks 17 th in Reading Literacy U.S.

Average is higher than U.S. average Average not measurably different from U.S. average Average is lower than U.S. average 40 U.S. 15-year-olds not measurably different from OECD average in reading literacy * = State avg. different than U.S. ** = State avg. different than OECD FL 492 MA 527 (*,**) CT 521 (*,**)

Massachusetts ConnecticutFlorida Number of education systems higher than state 3421 Number of education systems not measurably different than state Number of education systems lower than state Note: Comparisons include 65 education systems. State results in reading literacy

42 Percentage higher than U.S. Percentage not measurably different from U.S. Percentage lower than U.S. In reading literacy, 8 percent of U.S. 15-year-old students scored at proficiency level 5 or above, not measurably different from OECD average * = State percentage different from U.S. ** = State percentage different from OECD CT 15% (*,**) MA 16% (*,**) FL 6% (*,**)

Of the 62 education systems that participated in PISA 2009 and 2012: o In 34 (including the U.S.) no measurable change in average scores o In 21 average scores increased  Germany was lower than U.S. and OECD averages in 2000; higher than U.S. and OECD averages in 2012 o In 7 average scores declined  Sweden was higher than OECD average and not different from U.S. in 2000; lower than both U.S. and OECD averages in No measurable change since 2009 in average reading literacy scores in more than half of PISA education systems, including the U.S.

Source: Among OECD Countries, U.S. has the 4 th Largest Achievement Gap Between High-SES and Low-SES Students for 2006 PISA – Science Literacy PISA 2006 Results, OECD, table 4.8b 44 U.S.

Source: Among OECD Countries, U.S. has the 5 th Largest Achievement Gap Between High-SES and Low-SES Students for 2009 PISA – Reading Literacy PISA 2009 Results, OECD, Table II U.S. OECD

Source: PISA 2012 Results, OECD, Annex B1, Chapter 2, Table II.2.7a The U.S. ranks 26 th among 34 OECD Countries on the Percentage of Low-SES Students who are High-Performing PISA 2012 Mathematics Literacy U.S. OECD Note: High-performing, low-SES students are those who are in the bottom quarter of the ESCS in their country but perform in the top quarter across students from all countries after accounting for socioeconomic background. 46

Source: The U.S. Achievement Gap Between High-SES and Low-SES Students is Equivalent to Over Two Years of Schooling 2012 PISA – Mathematics Literacy PISA 2012 Results, OECD, Annex B1, Chapter 2, Table II.2.4a 47 U.S. OECD

For more information Contact: Dana Kelly NCES PISA at NCES: