1. Evaluating the Implementation of Singapore Math: A Large-Scale Study of 21 Elementary Schools James Badger Dianna Spence North Georgia College & State University NCTM Regional Conference 2009 Thursday, November 19

2. Agenda Singapore Math Overview & Examples Dianna Overview of Research & Methodology James Quantitative Findings Student Performance (Test Scores) Dianna Conceptual Framework & Qualitative Findings (Surveys, Observations, Interviews) James

3. What Is Singapore Math? Curriculum based on elementary mathematics teaching techniques used in Singapore Initial curriculum: “Primary Mathematics” Created in 1981 Developed by CDIS (Curriculum Development Institute of Singapore) Revisions 1992: stronger problem-solving focus (2 nd Ed.) 1999: reduced content (3 rd Ed.) 2001 & forward: adapted for U.S.

4. Why Singapore Math? Trends in International Math/Science Study Singapore 4 th graders consistently outperforming 4 th graders in other countries TIMSS: Mean Score, 4 th Grade Math COUNTRY19952003 Singapore590594 Hong Kong 557575 Japan567565 Netherlands549540 Latvia499533 England484531 Hungary521529 U.S.518518 Cyprus475510 Australia495499 New Zealand 469496 Scotland493490 Slovenia462479 Norway476451 Source: http://nces.ed.gov/timsshttp://nces.ed.gov/timss

5. Characteristics of Singapore Math Concrete  pictorial  abstract approach for each concept Strong emphasis on place value Repetitive drill minimized: topics are sequenced to reinforce/apply skills Problem solving based on conceptual approach rather than memorization of rules, “clue words”

6. Hallmark Strategies of Singapore Math Number bonds operations and part-whole relationships Mental math leverages and reinforces place value Bar models helps conceptualize arithmetic operations, fractions, ratios, algebraic thinking 9 27 6,325 + 400 = 6,725 “12 of Jack’s marbles are red, which is 2/9 of his collection…”

7. Example: Place Value Disks Thousands Hundreds Tens Ones. 110 100 1111111111 537 + 184 10 100 10 100 2 7 1

8. Example: Bar Modeling “12 of Jake’s marbles are red, and these make up 2/9 of his collection. How many marbles in Jake’s collection are not red?” 666666666 126 x 7 = 42 Whole collection ? Answer: 42 marbles in Jake’s collection are not red.

9. Classroom Best Practices Concrete  Pictorial  Abstract Emphasis on place value, mental math Conceptual approach, not rule-based Spiral approach to topics 3 + 4 34

10. Research Questions 1. Has the implementation of Singapore Math resulted in higher student math scores? 2. Has the implementation of Singapore Math had a positive impact on student interest and/or confidence in mathematics? 3. Has the implementation of Singapore Math resulted in measurable changes in the teachers’ attitudes toward mathematics? 4. Is there fidelity in the implementation of the Singapore Math curriculum? 5. How do elementary teachers implement the Singapore Math curriculum?

11. Research Design County-wide implementation in a school district in the Southeastern U.S. Research Setting 21 experimental elementary schools Every elementary school in the county All K-4 teachers used Singapore Math (first year) 3 control schools From another county with similar demographics State-approved curriculum (no Singapore Math) Participants One teacher in each grade (K-4) from each of the 24 schools volunteered to participate

12. Qualitative and Quantitative Data i. Interviews with teachers & students ii. Participating teachers’ journals (4 times) iii. Classroom observations iv. Teacher and student survey – fall/spring v. Video-taping of mathematics lesson (4 times) – analysis using TPR (Teaching Performance Record) vi. Standardized test scores

13. Quantitative Findings: Standardized Test Scores What standardized test scores did we examine? State criterion-reference test: Criterion-Reference Competency Test (CRCT) Nationally norm-referenced test: Iowa Test of Basic Skills (ITBS) What patterns did we look for? Degree of increase or decrease in mean score Increase or decrease in students meeting minimum requirements Increase or decrease in percentile rankings

14. Student Test Performance: Things to Keep in Mind Testing occurred during most teachers’ first year using new curriculum Most students in higher grades (e.g., 3 rd and 4 th ) had not previously been taught Singapore Math Data we are really interested in will not be available for 3-4 more years.

15. Student Performance: CRCT School Mean Math Score by Grade

17. Student Performance: CRCT Percent Change in Mean Math Score

21. Student Performance: CRCT Students Meeting Min. Requirements

23. Students Meeting CRCT Math Req.’s Change in Percentage Points

27. Student Performance: ITBS Mean Percentile Ranking in Math

28. Student Performance: ITBS Change in Mean Percentile Ranking

31. Fidelity of Curriculum Implementation (O’Donnell, 2008) Curriculum potential Teaching Curriculum-in-use Adaptation Theoretical Framework

32. Fidelity of Curriculum Implementation (O’Donnell, 2008) Guiding Questions Curriculum profile What are the critical components of the curriculum? What ranges of variations are acceptable? What does it mean to implement the program with fidelity (as defined by school administrators and county supervisors)? Teaching How does one distinguish good teaching and fidelity of implementation to good teaching practices prompted by the curriculum material? Curriculum-in-use How is the curriculum and the perceived curriculum viewed and implemented by teachers? How are curriculum materials and instruction mutually supportive and reinforcing? Adaptation Does the curriculum promote variation and adaptation of curriculum implementation?

33. 1. Survey Results Teachers, Kindergarten in particular, indicated a stronger affinity for and liked teaching mathematics at the end of the 2008-2009 school year than they had previously reported. Content knowledge of mathematics is important for effective teaching: teachers report some degree of understanding and confidence in teaching mathematics.

34. Survey Results, cont. A degree of satisfaction with the training and resources for mathematics teaching in 2009 – i.e. Singapore Math training and mentoring initiatives were apparently noticed and appreciated by many teachers. Echoed in interview and journal data

35. 2. Interview Results A fluid integration of the new curriculum: a consequence of the training provided by the county and ongoing support delivered by school administration Teachers reported manipulatives frequently integrated in the classroom - value discs and number bonds cited as fostering learning

36. Interview Results, cont. Teachers report students possessed a deeper understanding of mathematical concepts with the Singapore Math curriculum. Teachers claimed that they have higher expectations of students in Singapore Math. Parents’ reactions to Singapore Math ranged from enthusiasm to frustration.

37. 3. Journal Results Teachers wrote that students liked using place value disks a helpful hands-on manipulative to assist students grasp the concept of place value. Teachers wrote that students enjoyed activities, games, and manipulatives. - the students showed enthusiasm in class. - Students described these parts of math class as “fun”.

38. 4. Classroom Observation Some teachers tended to emphasize low-level cognitive processes in their instruction rarely asked students to draw associations to real-world contexts accountability pressure and time constraints? preponderance of teacher instruction at the expense of higher cognitive instruction, deeper questioning, and more occasions for cooperative student learning?

39. Second Year, 2009-2010 No design changes in the second year of the study – i.e. same data collection instruments, teachers volunteer to participate, fifth grade classes added, compare first and second year data. Data collected in the second year will - Determine student comprehension and achievement, fidelity of curriculum implementation.

40. Theoretical Framework: Fidelity of Curriculum Implementation (O’Donnell, 2008) Curriculum potential Teaching Curriculum-in-use Adaptation CHART Findings in Context Sources of Data Survey Data Journal Data Interview Data Observation Data Standardized Test Scores

41. Preliminary Observations Teacher training and support are essential Not a “drop-in” solution, especially at higher grades (need phased approach) Parent “buy-in” is important Will take time to see full impact