1. American Federation of TeachersTIMSS 1 The American Federation of Teachers looks at TIMSS

2. American Federation of TeachersTIMSS 2 F International Association for the Evaluation of Education Achievement (IEA) F National Center for Education Statistics (NCES) F National Science Foundation (NSF) F 41 participating nations-Grade 8 F 26 nations-Grade 4 F 21 nations-Grade 12 Third International Mathematics and Science Study (TIMSS)

3. American Federation of TeachersTIMSS 3 What TIMSS looked at F Structure of education systems 3 expectations, supports, decision makers, stakes F Curriculum 3 rigor, focus, specificity, authority, expectation F Delivery of instruction 3 goals, structure, and content of lessons F Student Achievement 3 9-year-olds 3 13-year-olds 3 end of secondary school

4. American Federation of TeachersTIMSS 4 Sampling/validity F More than $2 million to validate sampling F Samples included: 3 representative regions 3 metropolitan and non-metropolitan sites 3 socioeconomic variety 3 demographic variety 3 different German states and types of schools 3 different size communities and schools in Japan F Achievement of video students is very similar to country as a whole.

5. American Federation of TeachersTIMSS 5 TIMSS: Grade 4 When comparing achievement in 26 TIMSS countries, U.S. fourth grade students are: F above the international average in mathematics.  above the international average in science. NCES, 1997. Third International Mathematics and Science Study. Pursuing Excellence

6. American Federation of TeachersTIMSS 6 Who do U.S. fourth graders outperform? MATHEMATICS F Latvia Greece F Scotland Iran F Thailand Cyprus F England Iceland F Portugal F Norway F New Zealand F Kuwait NCES, 1997. TIMSS. Pursuing Excellence SCIENCE F England Latvia F Canada Israel F Singapore Iceland F Slovenia Greece F Ireland Portugal F ScotlandCyprus F Hong Kong Thailand F Hungary Iran F New Zealand Kuwait F Norway

7. American Federation of TeachersTIMSS 7 Whose 4th graders do significantly better than U.S. students? NCES, 1997. TIMSS. Pursuing Excellence MATHEMATICS F Singapore F Korea F Japan F Hong Kong F Netherlands F Czech Republic F Austria SCIENCE F Korea

8. American Federation of TeachersTIMSS 8 TIMSS: Grade 8 When comparing students in 41 countries, U.S. eighth grade students are: F below the average in mathematics F above the average in science SOURCE: National Center for Education Statistics. Third International Mathematics and Science Study. Pursuing Excellence

9. American Federation of TeachersTIMSS 9 Whose 8th grade students scored significantly better than those of the U.S.? MATHEMATICS SingaporeFrance KoreaHungary JapanRussia Hong KongAustralia BelgiumIreland Czech RepublicCanada Slovak Republic Switzerland Netherlands Slovenia Bulgaria SCIENCE Singapore Czech Republic Japan Korea Bulgaria Netherlands Slovenia Austria Hungary SOURCE: National Center for Education Statistics. Pursuing Excellence

10. American Federation of TeachersTIMSS 10 TIMSS: Population 3 Students at the end of secondary school U.S. grade 12 Ranges from grade 9 to grade 14 Four reports F General knowledge of mathematics F General knowledge of science F Advanced mathematics F Physics

11. American Federation of TeachersTIMSS 11 TIMSS: Population 3 General Knowledge of Mathematics Significantly better F Netherlands F Sweden F Denmark F Switzerland F Iceland F Norway F France F New Zealand F Australia F Canada F Austria F Slovenia F Germany F Hungary Same as U.S. F Italy F Russian Federation F Lithuania F Czech Republic F U.S. Significantly lower F Cyprus F South Africa American Federation of Teachers SOURCE: NCES,1998. Pursuing Excellence. TIMSS

12. American Federation of TeachersTIMSS 12 Scale score comparison General Mathematics Knowledge F Netherlands560 F TIMSS average 500 F U.S. 461 F South Africa 356 SOURCE: NCES,1998. Pursuing Excellence. TIMSS

13. American Federation of TeachersTIMSS 13 TIMSS Population 3 General Knowledge of Science Significantly better F Sweden F Netherlands F Iceland F Norway F Canada F New Zealand F Australia F Switzerland F Austria F Slovenia F Denmark Same as the U.S. F Germany F France F Czech Republic F Russian Federation F United States F Italy F Hungary F Lithuania Significantly lower F Cyprus F South Africa American Federation of Teachers SOURCE: NCES,1998. Pursuing Excellence. TIMSS

14. American Federation of TeachersTIMSS 14 Scale Score Comparison General Knowledge of Science F Sweden559 F TIMSS average500 F United States480 F South Africa349 SOURCE: NCES,1998. Pursuing Excellence. TIMSS

15. American Federation of TeachersTIMSS 15 How do our best 4th grade students do? The TIMSS top 10 percent in math

16. American Federation of TeachersTIMSS 16 How do our best 4th grade students do? The TIMSS top 10% in science

17. American Federation of TeachersTIMSS 17 How do our best 8th grade students do? The TIMSS top 10 percent

18. American Federation of TeachersTIMSS 18 How do our other 4th grade students do? The TIMSS top quartile/top half in math

19. American Federation of TeachersTIMSS 19 Percentage of 8th grade students in the international top quartile and half in math

20. American Federation of TeachersTIMSS 20 Variation in Performance: Grades 7/8 The international average in 8th grade mathematics is 30 points higher than at 7th grade. If 30 points is roughly a year’s difference: F the average U.S. 8th grader is more than a year behind Russian, Hungarian and French students in math; F more than two years behind Czech and Belgian students; F more than three years behind Japanese, Korean and Singaporean students. The scores of students at the U.S. median are similar to those of Japan’s lower 25%. SOURCE: Beaton et al, 1996, Mathematics Achievement in the Middle School Years

21. American Federation of TeachersTIMSS 21 Advanced math: Top 5 and 10 percent Top 10% Mean F Slovenia* 629 F France 612 F Canada 567 F TIMSS 554 F U.S. 485 F Czech Rep. 485 *Sampling problem Top 5% Mean F Slovenia* 664 F France 645 F Canada 620 F TIMSS 601 F U.S. 543 SOURCE:Mullis et al, 1998. Mathematics and Science Achievement in the Final Year of Secondary School. IEA

22. American Federation of TeachersTIMSS 22 Physics: Top 5 and 10 percent Top 10% Mean F Slovenia* 652 F Sweden 630 F TIMSS 533 F Canada522 F U.S. 451 Top 5% Mean F Slovenia* 698 F Sweden 678 F TIMSS 583 F Canada 574 F U.S. 485 SOURCE: Mullis et al, 1998. Mathematics and Science Achievement in the Final Year of Secondary School. IEA

23. American Federation of TeachersTIMSS 23 U.S. Fourth Grade Performance Strengths/ Weaknesses Mathematics Y Above the international average Whole numbers Fractions/proportionality Data and probability Geometry Patterns, relations, functions Y Below the international average Measurement, estima- tion, and number sense Science Y Above the international average Earth science Life science Physical science Environmental issues and the nature of science SOURCE: NCES, 1997. Pursuing Excellence.

24. American Federation of TeachersTIMSS 24 U.S. Grade 8 Performance: Strengths/ Weaknesses SOURCE: National Center for Education Statistics, 1996. Pursuing Excellence. Math F U.S. students better in: Fractions and number sense Data representation F U.S. students weaker in: Measurement Geometry Comparing only topics they taught does not relative standings. Science F U.S. students better in: Earth science Life science Environmental F U.S. students weaker in: Chemistry Physics U.S. teachers say significantly change the

25. American Federation of TeachersTIMSS 25 How do other U.S. 4th graders do? The TIMSS top quartile in science

26. American Federation of TeachersTIMSS 26 Comparative Performance at Grades 4/8 Science Mullis et al. Mathematics Achievement in the Primary School Years. IEA NCES. TIMSS. Pursuing Excellence FOURTH GRADE F Korea*** F Japan F United States F Austria F Australia F Netherlands F Czech Republic F England F Canada F Singapore F Slovenia F Ireland u Scotland ***Statistically above all other countries EIGHTH GRADE F Singapore F Czech Republic F Japan F Korea F Netherlands F Slovenia F Austria F Hungary F England F Australia F Ireland F United States

27. American Federation of TeachersTIMSS 27 U.S. Performance: Grade 4 and Grade 8 Mullis et a.l, 1997. Mathematics Achievement in the Primary School Years The U.S. is the only country participating in the 4th grade TIMSS whose math standing falls from above average at fourth grade to below average at eighth grade.

28. American Federation of TeachersTIMSS 28 TIMSS: Population 3 Advanced Math Compared to All U.S. Calculus students Significantly better F France F Russian Federation F Switzerland F Denmark F Cyprus F Lithuania Significantly lower F Germany F Austria Compared to Only U.S. AP Calculus students Significantly better F France Significantly lower F Slovenia F Italy F Czech Republic F Germany F Austria SOURCE: NCES,1998. Pursuing Excellence.

29. American Federation of TeachersTIMSS 29 Scale score comparison Advanced math F France 557 F U.S. AP513 F TIMSS501 F U.S. Calculus 492 Pre-calculus 442 F Austria436 SOURCE: NCES,1998. Pursuing Excellence. TIMSS

30. American Federation of TeachersTIMSS 30 The crème de la crème: math F The average score of U.S. students is lower than that of their peers at both ends of the scale even though the difference between top and bottom is similar to that of most countries. F The most advanced 5% of U.S. math students score similarly to 10 to 20% of the age cohort in most of the other countries. SOURCE: NCES,1998. Pursuing Excellence

31. American Federation of TeachersTIMSS 31 TIMSS: Population 3 Advanced science: physics Compared to all physics students Significantly better F Norway F Sweden F Russian Federation F Denmark Significantly lower F Austria SOURCE: NCES,1998. Pursuing Excellence TIMSS Same as U.S. F Slovenia F Germany F Australia F Cyprus F Latvia F Switzerland F Greece F Canada F U.S. F France F Czech Republic

32. American Federation of TeachersTIMSS 32 Scale score comparison Physics F Norway 581 F TIMSS504 F U.S. AP474 F Austria435 F U.S. Non-AP423 American Federation of Teachers SOURCE: NCES,1998. Pursuing Excellence. TIMSS

33. American Federation of TeachersTIMSS 33 WARNING! No single finding of the TIMSS study by itself can explain why a country’s students do or do not do well. It is essential to see all the pieces of the puzzle and how they fit together--to figure out why, when some things look the same on the surface, there are huge differences in results; or why, when things seem quite different, results are quite similar.

34. American Federation of TeachersTIMSS 34 F The lesson profiles of France and Spain are similar. Yet, French students score significantly better than Spanish students. F Swiss lessons emphasize students’ responsibility for their own learning in contrast to very teacher-directed instruction in France. Both do well.  Canada’s math curriculum has more topics than the U.S. curriculum. Her students score significantly better in math. Schmidt et al, 1996. Characterizing Pedagogical Flow. For example...

35. American Federation of TeachersTIMSS 35 We must compare, conjecture, and ask what made the difference. Was it:  the subject matter knowledge of teachers? F the amount of time students worked alone? F the quality of textbooks and resources? F the presence of a common core curriculum? F having stakes for student achievement? F training? What patterns emerge from data for all the student populations and all the reports?

36. American Federation of TeachersTIMSS 36 What sets us apart from countries whose students scored significantly better? F no nationally defined curriculum F less advanced mathematical content F disruption of the educational process F no rewards or sanctions for academic performance F weak induction process for teachers F little opportunity for professional interaction with colleagues

37. American Federation of TeachersTIMSS 37 Teaching out of field The U.S. Commission on Teaching and America’s Future reports: F nearly one-fourth of all secondary teachers do not even have a college minor in their main teaching field, including; 3 30% of mathematics teachers 3 56% of physical science teachers. F These proportions are much higher in high-poverty schools and in lower track classes. What Matters Most: Teaching for America’s Future. 1996.

38. American Federation of TeachersTIMSS 38 Teaching and America’s Future recommendations F Get serious about standards, for both students and teachers. F Reinvent teacher preparation and professional development. F Overhaul teacher recruitment and put qualified teachers in every classroom. F Encourage and reward teaching knowledge and skill.  Create schools that are organized for student and teacher success. Source: National Commission on Teaching & America’s Future, 1996. What Matters Most: Teaching for America’s Future

39. American Federation of TeachersTIMSS 39 Possible Reasons for Our Downward Slide from 4th to 8th grade H Less rigorous content H Unfocused curriculum H Out-of field teaching H Lack of a common curriculum H Little incentive for high achievement

40. American Federation of TeachersTIMSS 40 Why is the U.S. relatively worse at 12th grade than 8th? Could it be: F amount of instruction? F course-taking in final year? F curriculum? F outside of school activity? F existence of consequences?

41. American Federation of TeachersTIMSS 41 How did course-taking in the final year affect end-of-secondary TIMSS results? F U.S. students not taking math in grade 12: 34% F Canadian students not taking math in final year: 46% F At least one third of students are no longer taking math in Iceland, Netherlands, Switzerland.

42. American Federation of TeachersTIMSS 42 Other Findings from end-of-secondary TIMSS F Strong relationship between calculator use and achievement. F Computer/achievement relationship is different. F Unlike other countries, U.S. advanced math and science students did not report having more than 5 hours of instruction a week. F Virtually no single-step or single-stage problems on TIMSS or NAEP. SOURCE: NCES,1998. Pursuing Excellence. TIMSS

43. American Federation of TeachersTIMSS 43 Employment and School Climate F Hours per day working at a paid job. (3 or more hours) U.S. 55% TIMSS 18% F Had something stolen at school in the month prior to taking TIMSS U.S. 24% TIMSS 13% F Was threatened by another student in the month preceding TIMSS U.S. 11% TIMSS 7% SOURCE: NCES,1998. Pursuing Excellence. TIMSS

44. American Federation of TeachersTIMSS 44 What do we make of it? F Countries having higher achievement at the end of school than at 8th grade tended to have older students at the end of school than countries whose standing was relatively worse at the end of school. F Countries where more students were taking math in their final year of secondary school were not more likely to have a better position relative to other countries than they did in 8th grade. NCES, 1998. Pursuing Excellence. TIMSS.

45. American Federation of TeachersTIMSS 45 First in the World Consortium  Strong foundation in the middle grades  Challenging curriculum and higher expectations in high school  Better prepared math and science teachers (all have majors or minors in what they teach)  Significant professional development opportunities for all. Learning networks to share best practices and teaching techniques. Incentives to work with experts and take significant course work. Source: First in the World Consortium

46. American Federation of TeachersTIMSS 46 Curriculum The United States is one of a small number of countries whose curriculum is not determined nationally or at least “strongly advised” nationally. Most countries can identify a common core of knowledge that all students are expected to learn. Schmidt, 1996. A Splintered Vision.

47. American Federation of TeachersTIMSS 47 Curriculum in the early years F More topics in both U.S. math and science than others F Content similar to the rest of the world in math F Science is more focused in 4th grade than in 8th. F We do less well in physical science even at this level. SOURCE: William Schmidt. TIMSS Curriculum Study and NCES, 1997. Pursuing Excellence.

48. American Federation of TeachersTIMSS 48 F Between fourth and eighth grades, other countries introduce six to eight math and eight to eleven science topics in a focused way. F Most of the U.S. curriculum is still unfocused and splintered. Only one of the topics introduced in U.S. math is done so with focused attention. SOURCE: William Schmidt TIMSS Curriculum Study Curriculum in later years

49. American Federation of TeachersTIMSS 49 Grade 8 curriculum: more findings F U.S. curriculum is less rigorous than in other countries; F covers more topics per year than most countries; F is not as focused as in Germany and Japan in math.  In science, topic coverage is similar to other countries. Schmidt, 1996. A Splintered Vision.

50. American Federation of TeachersTIMSS 50 Level of 8th grade content by average international placement F 8th grade F 9th grade F 7th grade F Germany F Japan  United States NCES, 1996. TIMSS Videotape Study web site: http:\\ed.gov/NCES/TIMSS/html

51. American Federation of TeachersTIMSS 51 What makes strong mathematics lessons? The Videotape Classroom Study analyzed: F teachers’ goals for lessons F treatment of concepts and applications F the presence of alternative solution methods F how mathematical principles, properties and definitions were used F whether proofs were included F whether concepts were connected F the kinds of tasks assigned

52. American Federation of TeachersTIMSS 52 Four Kinds of Goals F Mathematical Skills how to solve specific kinds of problems, use standard formulas... F Mathematical Thinking exploration, development, comprehension of mathematical concepts; multiple solutions F Social/Motivational non-mathematical F Test Preparation

53. American Federation of TeachersTIMSS 53 Teachers’ goals: skills/solving specific problems vs. thinking and understanding mathematics

54. American Federation of TeachersTIMSS 54 Structure and delivery of lessons F Characteristic lesson scripts s U.S. and German teachers present a procedure or skill and then have students apply it. s Japanese teachers have students solve a problem first, then draw on student thinking to develop understanding of a concept. F Coherence and U.S. lessons: s switched topics more often, s did not link the changes, s had more “irrelevant diversions,” and s more interruptions from outside. SOURCE: NCES, 1996. Pursuing Excellence

55. American Federation of TeachersTIMSS 55 What could 4th grade students in the top half in math answer correctly? Here is a number sentence. 2000 +  + 30 + 9 = 2739 What number goes in the  to make this sentence true? Answer: ______________ SOURCE: Mullis et al, 1997. Mathematics Achievement in the Primary School Years

56. American Federation of TeachersTIMSS 56 What could fourth grade students in the top quartile in math answer correctly? 25 x 18 is more than 24 x 18 How much more? A.1 B.18 C.24 D.25 SOURCE: Mullis et al, 1997. Mathematics Achievement in the Primary School Years. IEA

57. American Federation of TeachersTIMSS 57 What could 9-year-olds in the top 10 percent in math answer correctly? There are 10 girls and 20 boys in Juanita’s class. Juanita said that there is one girl for every two boys. Her friend Amanda said that means 1/2 of all the students in the class are girls. How many students are there in Juanita’s class? F Is Juanita right? Answer___ F Use words or pictures to explain why. F Is Amanda right? ___ F Use words or pictures to explain why. SOURCE: Mullis et al, 1997. Mathematics Achievement in the Primary School Years. IEA

58. American Federation of TeachersTIMSS 58 Questions that differentiated 8th grade students at various marker levels SOURCE: Beaton et al, 1996. Mathematics Achievement in the Middle School Years Top 10% Top halfLower half Percent Distance on map (a)Subtraction If the price of a Using the scale 6 0 0 0 can of beans is on a map - 2 3 6 9 raised from 60 ( 1cm = 1km) cents to 75 cents find the distance (b) Fractions what is the per- between two cent increase given cities. Write a fraction that the price? is larger than 2/7.

59. American Federation of TeachersTIMSS 59 Grade 8 Item comparisons SOURCE: Beaton et al., 1996. Mathematics Achievement in the Middle School Years Lower Half Top Half Top 10% O  OO   OOO    If m represents Juan has 5 fewer Which set of shapesa positive num-hats than Maria, and is arranged in theber, which ofClarissa has 3 times same pattern?these is as many hats as Juan.            equivalent toIf Maria has n hats,          m+m+m+m?which of these repre-        m+4sents the number of            4mhats that Clarissa has? m4 5-3n3n-5 4(m+1) 3n3(n-5) n - 5

60. American Federation of TeachersTIMSS 60 End of Secondary Math Literacy F From a batch of 3000 light bulbs, 100 were selected at random and tested. If 4 of the light bulbs in the sample were found to be defective, how -many defective light bulbs would be expected in the entire batch?

61. American Federation of TeachersTIMSS 61 End of secondary Math Literacy F Using the set of axes below, sketch a graph which shoes the relationship between the height of a person and his/her age from birth to 30 years. Be sure to label your graph, and include a realistic scale on each axis. (Picture of 11 x 16 grid)

62. American Federation of TeachersTIMSS 62 End of secondary Advanced Students F The vertices of the triangle PQR are the points p(1,2), Q(4,5) and R(-4,12). Which one of the following statements about triangle PQR is true? A. PQR is a right triangle with the right angle  P. B. PQR is a right triangle with the right angle  Q. C. PQR is a right triangle with the right angle  R. D. PQR is not a right triangle.

63. American Federation of TeachersTIMSS 63 End of Secondary Advanced math students F See p. 157 and scan in question box

64. American Federation of TeachersTIMSS 64

65. American Federation of TeachersTIMSS 65 Write down two different things that people can do to help reduce air pollution. SOURCE: Martin et al, 1997. Science Achievement in the Primary School Years. IEA In science, the upper half of fourth graders could generally answer this correctly

66. American Federation of TeachersTIMSS 66 In science, upper quartile fourth graders could generally answer this correctly SOURCE: Martin et al, 1997. Science Achievement in the Primary School Years. IEA The picture shows two forms of sugar--solid cubes and packets of loose sugar. One cube has the same mass of sugar as one packet. Which of the two forms of sugar will dissolve faster in water?__________ Give a reason for your answer.

67. American Federation of TeachersTIMSS 67 Item 4th graders in the top 10 percent in science were likely to answer correctly To find out whether seeds grow better in the light or dark, you could put some seeds on pieces of damp paper and A. keep them in a warm, dark place B.keep one group in a light place and another in a dark place C.keep them in a warm, light place D. put them in a light or dark place that is cool SOURCE: Martin et al, 1997. Science Achievement in the Primary School Years. IEA

68. American Federation of TeachersTIMSS 68 Treatment of mathematical concepts In U.S. 8th grade math, concepts were developed much less frequently than in Germany and Japan.

69. American Federation of TeachersTIMSS 69 Structure and delivery of lessons F Characteristic national lessons uU.S. and German teachers present a procedure or skill; then students apply it.  Japanese teachers have students solve a problem first, then draw on student thinking to develop understanding of a concept.  Coherence and U.S. lessons uswitched topics more often udid not link parts of the lesson uhad more “irrelevant diversions”  more interruptions from outside SOURCE: NCES, 1996. Pursuing Excellence

70. American Federation of TeachersTIMSS 70 Classwork and Seatwork F Japan switches back and forth between the two and has shorter segments. (Stevenson & Stigler, The Learning Gap) F U.S. and German seatwork focuses on routine procedures; Japan’s seatwork is thinking- oriented.

71. American Federation of TeachersTIMSS 71 Frequency of Alternative Solution Methods Grade 8 math

72. American Federation of TeachersTIMSS 72 Homework Comparing Germany, Japan, and the U.S.: sThe U.S. has about as much homework as others. sThe U.S. and Germany spend more class time sharing homework than Japan. sOnly the U.S. works on homework during class and counts it in grades.

73. American Federation of TeachersTIMSS 73 Linking was important for lesson coherence F There are times when teachers want students to understand ideas in relation to each other. If a teacher in the videotape study made a specific connection to tie ideas together during a lesson, researchers called it a “link.” F Remember when Marie said...

74. American Federation of TeachersTIMSS 74 Percent of grade 8 math lessons that include explicit linking by the teacher

75. American Federation of TeachersTIMSS 75 Expectations of classroom talk for students U.S. Japan  more yes/no or  much more likely to short answer questions include explana- tions  define terms and state  clarify and ruleselaborate Stevenson & Stigler, 1992. The Learning Gap

76. American Federation of TeachersTIMSS 76 Lesson characteristics that contribute to mathematical quality F Coherence of sequencing F Way in which examples contribute to lesson’s central concept F Type of reasoning required of students  Increase in complexity of problems NCES, 1996. Pursuing Excellence

77. American Federation of TeachersTIMSS 77 Mathematical quality of lessons When lessons were rated for the quality of math content, 87 percent of U.S. lessons were given the lowest rating.

78. American Federation of TeachersTIMSS 78 Proofs and Deductive Reasoning

79. American Federation of TeachersTIMSS 79 Other TIMSS findings F What is written during a lesson: In Japan, 80% of what is written is still visible at the end of class, compared to about 20% of what has been written in American classrooms. FClass time: U.S. students spend more time in math class than their peers. F Homework: Only U.S. students spend time working on the next day’s homework in class. F TV habits: Students in other countries report watching as much TV as their U.S. peers. TIMSS Videotape Study web site: http:\\www.ed.gov/NCES/TIMSS/html NCES, 1996. Pursuing Excellence

80. American Federation of TeachersTIMSS 80 Grouping F In the U.S., 8th grade students are generally put into different math classes on the basis of ability. The content differs for different classes. F In Germany, although tracked to different schools, students study the same content, but the rigor varies. F In Japan, classes are heterogeneous through eighth grade; everyone studies the same thing. NCES, 1996. Pursuing Excellence

81. American Federation of TeachersTIMSS 81 REMEMBER! No single finding of the TIMSS study by itself can explain the achievement level of a country’s students. It takes a whole coherent system.