1. 1 PISA 2012

2. What is PISA?  International large-scale assessment organized by the Organization for Economic Cooperation and Development (OECD)  NCES is U.S. coordinating and funding organization  PISA Governing Board (PGB) determines policy  Assessment of 15-year-old students  Administered every 3 years since 2000 2

3.  In 2012, 68 participating education systems  Three U.S. states: Connecticut, Florida, and Massachusetts 3 What is PISA?

4.  Assessment subjects:  Mathematics, science, reading literacy  Mathematics was main subject assessed in 2012  Computer-based mathematics and reading literacy  Optional assessment taken by 32 education systems  Content knowledge, not limited to school-based curricula  PISA assesses applied knowledge/literacy:  “How well can students nearing the end of compulsory schooling apply their knowledge to real- life situations?” 4 What is PISA?

5. Average Scores: Scale of 0-1000 for all domains Proficiency Levels: Percentages of students scoring at 6 levels Trends: Change between average scores in 2012 and the scores in previous assessment years Subgroup scores: International (e.g., gender, language spoken in home) and U.S. specific variables (e.g., race/ethnicity) 5 What PISA Reports

6. 6 U.S PISA Findings

7.  Ranked better in reading literacy than in mathematics and science literacy  Below OECD average score in mathematics only  Higher percentage at low proficiency levels than OECD average in mathematics only  Lower percentage of top performers than OECD average in mathematics only  No measurable change in average scores in mathematics, science, or reading literacy 7 General patterns of U.S. results

8. 8 Mathematics Literacy

9. Quantity (25%) Are 15-year-olds able to comprehend multiple representations of numbers, engage in mental calculation, employ estimation, and assess the reasonableness of results? Change and Relationships (25%) Can students model change and relationships with the appropriate functions and equations? Space and Shape (25%) Uncertainty and Data (25%) Can students understand perspective, create and read maps, and manipulate 3D objects? Can students use probability and statistics and other techniques of data representation and description to mathematically describe, model, and interpret uncertainty? Note: Percentages shown are the approximate percentage of scale score points. 9 PISA 2012 mathematics literacy content categories

10. Formulate (25%) Can 15-year-olds recognize and identify opportunities to use mathematics and then provide mathematical structure to a problem presented in some contextualized form in order to formulate situations mathematically? Interpret (25%) Employ (50%) Can students interpret, apply and evaluate mathematical outcomes in order to determine whether results are reasonable and make sense in the context of the problem? Are students able to employ mathematical concepts, facts, procedures, and reasoning to solve mathematically-formulated problems and obtain mathematical conclusions? 10 Note: Percentages shown are the approximate percentage of scale score points. PISA 2012 mathematics literacy process categories

11. 11 Identify information and carry out routine procedures according to direct instructions in explicit situations. Employ basic algorithms, formulae, procedures, or conventions. Capable of direct reasoning and making literal interpretations of the results. Execute clearly described procedures, including those that require sequential decisions. Select and apply simple problem- solving strategies. Work effectively with explicit models that may involve constraints or making assumptions. Capable of reasoning with some insight in straightforward contexts. Work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representa- tions, symbolic and formal characteriza- tions and insight pertaining to these situations. Apply insight along with a mastery of symbolic and formal mathematical operations and relationships to develop new approaches and strategies for attacking novel situations. Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 PISA mathematics literacy proficiency levels

12. 12 Question: CHARTS Proficiency level 1 Mathematics literacy – Example item

13. 13 Question: CHARTS - Continued Percentage of students answering correctly in 2012: OECD average: 80% U.S.: 77% Mathematics literacy – Example item

14. 14 Question: DRIP RATE Proficiency level 5 Mathematics literacy – Example item

15. 15 Question: DRIP RATE - Continued Percentage of students answering correctly in PISA 2012: OECD average: 26% U.S.: 30% Mathematics literacy – Example item

16. Source: Of 34 OECD Countries, U.S.A. Ranks 27 th in Mathematics Literacy U.S. OECD Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average National Center for Education Statistics, 2013, http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_3a.asp.16http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_3a.asp

17. Average higher than U.S. average Average not measurably different from U.S. average Average lower than U.S. average 17 U.S. 15-year-olds OECD average in mathematics literacy MA 514 (*,**) CT 506 (*) FL 467 (*,**) * = State avg. different from U.S. ** = State avg. different from OECD

18. Massachusetts ConnecticutFlorida Number of education systems higher than state 91236 Number of education systems not measurably different from state 12145 Number of education systems lower than state 443924 18 Note: Comparisons include 65 education systems. State results in mathematics literacy

19. SubscaleU.S. average scoreOECD average score Quantity478 495 Uncertainty and data488 493 Change and relationships488 493 Space and shape463 490 19 Not measurably different from U.S. average Significantly higher than U.S. average U.S. not measurably different from OECD average in two mathematics content subscales

20. 20 In mathematics literacy, 9 percent of U.S. 15-year-old students scored at proficiency level 5 or above Percentage higher than U.S. Percentage not measurably different from U.S. Percentage lower than U.S. * = State percentage different from U.S. ** = State percentage different from OECD MA 19% (*,**) CT 16% (*,**) FL 6% (*,**)

21. 21 No measurable change since 2009 in average mathematics literacy scores in more than half of PISA education systems, including U.S. Of the 62 education systems in PISA 2009 and 2012: o In 33 (including the U.S.) no measurable change in average scores o In 18 average scores increased  Russian Federation was below U.S. and OECD averages in 2003; not measurably different in 2012  Poland was lower than OECD and not measurably different from U.S. averages in 2003; higher than both in 2012 o In 11 average scores declined  Finland declined between 2006-2009 and again between 2009- 2012; scored 548 in 2006 and 519 in 2012  Norway was above U.S. average in 2003; not measurably different from U.S. average and below OECD average in 2012

22. 22 Science Literacy

23. 23 Present scientific explanations that are obvious and follow explicitly from given evidence. Use direct reasoning and make literal interpretations of the results of scientific inquiry or technological problem solving. Identify clearly described scientific issues in a range of contexts. Select facts and knowledge to explain phenomena and apply simple models or inquiry strategies. Select and integrate explanations from different disciplines of science or technology and link those explanations directly to aspects of life situations. Use well- developed inquiry abilities, link knowledge appropriately, and bring critical insights to situations. Construct explanations based on evidence and arguments based on critical analysis. Link different information sources and explanations and use evidence from those sources to justify decisions. Demonstrate advanced scientific thinking and reasoning, and use scientific understanding in support of solutions to unfamiliar scientific situations. Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 PISA science literacy proficiency levels

24. 24 Question: MARY MONTAGU Proficiency level 2 Science literacy – Example item

25. 25 Question: MARY MONTAGU - Continued Percentage of students answering correctly in 2006: OECD average: 75% U.S.: 73% Science literacy – Example item

26. 26 Question: GREENHOUSE Proficiency level 6 Science literacy – Example item

27. 27 Question: GREENHOUSE - Continued Percentage of students answering correctly in 2006: OECD average: 19% U.S.: 18% Science literacy – Example item

28. Source: National Center for Education Statistics, 2013, http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_4a.asp. 28http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_4a.asp Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average Of 34 OECD Countries, U.S.A. Ranks 20 th in Science Literacy U.S.

29. Average higher than U.S. average Average not measurably different from U.S. average Average lower than U.S. average 29 U.S. 15-year-old students not measurably different from OECD average in science literacy MA 527 (*,**) CT 521 (*,**) FL 485 (**) * = State avg. different from U.S. ** = State avg. different from OECD

30. Massachusetts ConnecticutFlorida Number of education systems higher than state 6726 Number of education systems not measurably different from state 1415 Number of education systems lower than state 454324 30 Note: Comparisons include 65 education systems. State results in science literacy

31. 31 Percentage higher than U.S. Percentage not measurably different from U.S. Percentage lower than U.S. In science literacy, 7 percent of U.S. 15-year-old students scored at proficiency level 5 or above * = State percentage different from U.S. ** = State percentage different from OECD MA 14% (*,**) CT 13% (*,**) FL 5% (**)

32. Of the 62 education systems that participated in PISA 2009 and 2012: o In 43 (including the U.S.) there was no measurable change in average scores o In 13 average scores increased  Poland was not measurably different from U.S. and OECD averages in 2006; above U.S. and OECD averages in 2012 o In 6 average scores declined  Sweden was higher than U.S. and OECD averages in 2006; below U.S. and OECD averages in 2012 32 No measurable change since 2009 in average science literacy scores in more than half of PISA education systems, including the U.S.

33. 33 Reading Literacy

34. 34 Locate a single piece of explicitly stated information in a prominent position in a short, simple text. Recognize the main theme or author’s purpose in a text about a familiar topic. Locate one or more pieces of information, which may need to be inferred and may need to meet several conditions. Integrate several parts of a text in order to identify a main idea, understand a relationship, or construe the meaning of a word or phrase. Interpret the meaning of nuances of language in a section of text by taking into account the text as a whole. Locate and organize several pieces of deeply embedded information, inferring which information in the text is relevant. Make multiple inferences, compari- sons, and contrasts that are both detailed and precise. Deal with unfamiliar ideas and generate abstract categories for interpreta- tions. Level 1b Level 1a Level 2 Level 3 Level 5 Level 6 Level 4 PISA reading literacy proficiency levels

35. Reading literacy – Example item 35 Question: TELECOMMUTING Proficiency level 3

36. 36 Question: TELECOMMUTING-Continued Percentage of students answering correctly in 2009: OECD average: 52% U.S.: 55% Reading literacy – Example item

37. 37 Question: THE PLAY’S THE THING Proficiency level 6 Reading literacy – Example item

38. 38 Question: THE PLAY’S THE THING- Continued What were the characters in the play doing just before the curtain went up? Reading literacy – Example item Percentage of students answering correctly in PISA 2009: OECD average: 13% U.S.: 13%

39. Source: National Center for Education Statistics, 2013, http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_5a.asp.39http://nces.ed.gov/surveys/pisa/pisa2012/pisa2012highlights_5a.asp Higher than U.S. average Not measurably different from U.S. average Lower than U.S. average Of 34 OECD Countries, U.S.A. Ranks 17 th in Reading Literacy U.S.

40. Average is higher than U.S. average Average not measurably different from U.S. average Average is lower than U.S. average 40 U.S. 15-year-olds not measurably different from OECD average in reading literacy * = State avg. different than U.S. ** = State avg. different than OECD FL 492 MA 527 (*,**) CT 521 (*,**)

41. Massachusetts ConnecticutFlorida Number of education systems higher than state 3421 Number of education systems not measurably different than state 91617 Number of education systems lower than state 534527 41 Note: Comparisons include 65 education systems. State results in reading literacy

42. 42 Percentage higher than U.S. Percentage not measurably different from U.S. Percentage lower than U.S. In reading literacy, 8 percent of U.S. 15-year-old students scored at proficiency level 5 or above, not measurably different from OECD average * = State percentage different from U.S. ** = State percentage different from OECD CT 15% (*,**) MA 16% (*,**) FL 6% (*,**)

43. Of the 62 education systems that participated in PISA 2009 and 2012: o In 34 (including the U.S.) no measurable change in average scores o In 21 average scores increased  Germany was lower than U.S. and OECD averages in 2000; higher than U.S. and OECD averages in 2012 o In 7 average scores declined  Sweden was higher than OECD average and not different from U.S. in 2000; lower than both U.S. and OECD averages in 2012 43 No measurable change since 2009 in average reading literacy scores in more than half of PISA education systems, including the U.S.

44. Source: Among OECD Countries, U.S. has the 4 th Largest Achievement Gap Between High-SES and Low-SES Students for 2006 PISA – Science Literacy PISA 2006 Results, OECD, table 4.8b 44 U.S.

45. Source: Among OECD Countries, U.S. has the 5 th Largest Achievement Gap Between High-SES and Low-SES Students for 2009 PISA – Reading Literacy PISA 2009 Results, OECD, Table II.3.1 45 U.S. OECD

46. Source: PISA 2012 Results, OECD, Annex B1, Chapter 2, Table II.2.7a The U.S. ranks 26 th among 34 OECD Countries on the Percentage of Low-SES Students who are High-Performing PISA 2012 Mathematics Literacy U.S. OECD Note: High-performing, low-SES students are those who are in the bottom quarter of the ESCS in their country but perform in the top quarter across students from all countries after accounting for socioeconomic background. 46

47. Source: The U.S. Achievement Gap Between High-SES and Low-SES Students is Equivalent to Over Two Years of Schooling 2012 PISA – Mathematics Literacy PISA 2012 Results, OECD, Annex B1, Chapter 2, Table II.2.4a 47 U.S. OECD

48. For more information Contact: Dana Kelly NCES 202-219-7101 Dana.Kelly@ed.gov 48 PISA at NCES: http://nces.ed.gov/surveys/pisa/