1. It Pays to Compare: Effectively Using Comparison to Support Student Learning of Algebra Bethany Rittle-Johnson Jon Star

2. IES Conference 20082 Our approach to improving students’ mathematics learning Identify instructional practices used in exemplary and typical classrooms Use cognitive science literature to focus on practices most likely to help student learning Experimentally evaluate impact of the instructional practice on student learning and develop instructional guidelines

3. IES Conference 20083 Potential of comparison Mathematics Education: Central tenet of reform efforts; used by teachers Cognitive Science: A fundamental learning mechanism

4. IES Conference 20084 Central tenet of math reforms Students benefit from sharing and comparing solution methods “nearly axiomatic”, “with broad general endorsement” (Silver et al., 2005) Noted feature of ‘expert’ math instruction Present in classrooms in high performing countries such as Japan and Hong Kong (Ball, 1993; Fraivillig, Murphy, & Fuson, 1999; Huffred-Ackles, Fuson, & Sherin Gamoran, 2004; Lampert, 1990; Silver et al., 2005; NCTM, 1989, 2000; Richland et al 2007; Stigler & Hiebert, 1999)

5. IES Conference 20085 Used in some Algebra textbooks Sobel, M.A., Maletsky, E. M., Lerner, N., & Cohen, L.S. (1985) Algebra One, Harper and Row Inc.

6. IES Conference 20086 But does comparison improve student learning? No evidence that comparison improves student learning in mathematics Cognitive science research suggests that it should…

7. Comparison in cognitive science “The simple, ubiquitous act of comparing two things is often highly informative to human learners…. Comparison is a general learning process that can promote deep relational learning and the development of theory-level explanations” (Gentner, 2005, pp. 247, 251)

8. IES Conference 20088 Fundamental learning mechanism Lots of evidence from cognitive science ◦ Identifying similarities and differences in multiple examples is an important pathway to flexible, transferable knowledge Mostly laboratory studies Rarely done with school-age children or in mathematics (Catrambone & Holyoak, 1989; Gentner, Loewenstein, & Thompson, 2003; Gick & Holyoak, 1983; Kurtz, Miao, & Gentner, 2001; Loewenstein & Gentner, 2001; Namy & Gentner, 2002; Oakes & Ribar, 2005; Schwartz & Bransford, 1998)

9. IES Conference 20089 Does comparison support math learning? Goal of our IES grant ◦ Investigate whether comparison can support conceptual and procedural knowledge of equation solving (and estimation) ◦ Explore what types of comparison are most effective ◦ Experimental studies in intact classrooms

10. IES Conference 200810 Why equation solving? Often students’ first exposure to abstraction and symbolism of mathematics Area of weakness for US students (Blume & Heckman, 1997; Schmidt et al., 1999) According to NCTM and National Math Panel Report, linear equation solving should be a focal point of math instruction in middle school Although real-world contexts and informal solution methods are powerful for simple problems, equations and equation solving are more effective for complex problems (Koedinger, Alibali & Nathan, 2008)

11. IES Conference 200811 Multiple methods for solving equations Method #1: 3(x + 1) = 15 3x + 3 = 15 3x = 12 x = 4 Method #2: 3(x + 1) = 15 x + 1 = 5 x = 4 ◦ Some are better than others ◦ Students tend to memorize only one method ◦ Example: Solving 3(x + 1) = 15

12. IES Conference 200812 Study 1 Research question: Does comparing solution methods improve equation solving knowledge? Research design: Randomly assigned to: ◦ Comparison condition  Compare and contrast alternative solution methods ◦ Sequential condition  Study same solution methods sequentially Rittle-Johnson, B. & Star, J.R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology.

13. IES Conference 200813 Translation to the classroom Students study and explain worked examples with a partner Based on core findings in cognitive science -- the advantages of: ◦ Worked examples (e.g. Sweller, 1988) ◦ Generating explanations (e.g. Chi et al, 1989; Rittle-Johnson, 2006) ◦ Peer collaboration (e.g. Fuchs & Fuchs, 2000)

14. IES Conference 200814 Comparison condition

15. IES Conference 200815 Sequential condition next page

16. IES Conference 200816 Predicted outcomes Students in comparison condition will make greater gains in: ◦ Procedural knowledge, including success on novel problems ◦ Procedural flexibility (e.g. use more efficient methods; evaluate when to use a procedure) ◦ Conceptual knowledge (e.g. equivalence)

17. IES Conference 200817 Study 1 Method Participants: 70 7th-grade students and their math teacher Design: ◦ Pretest - Intervention - Posttest ◦ Replaced 2 lessons in textbook ◦ Intervention occurred in partner work during 2 1/2 math classes Intervention: ◦ Randomly assigned to Compare or Sequential condition ◦ Studied worked examples with partner ◦ Solved practice problems on own

18. IES Conference 200818 Procedural knowledge assessment Equation Solving ◦ Intervention: 1/3 (x + 1) = 15 ◦ Posttest Familiar: -1/4 (x – 3) = 10 ◦ Posttest Novel: 0.25 (t + 3) = 0.5

19. IES Conference 200819 Procedural flexibility Use of more efficient solution methods on procedural knowledge assessment Knowledge of multiple methods ◦ Solve each equation in two different ways ◦ Evaluate methods: Looking at the problem shown above, do you think that this way of starting to do this problem is a good idea? An ok step to make? Circle your answer below and explain your reasoning. (a) Very good way (b) Ok to do, but not a very good way (c) Not OK to do

20. IES Conference 200820 Conceptual knowledge assessment

21. IES Conference 200821 Gains in procedural knowledge F(1, 31) =4.49, p <.05

22. IES Conference 200822 Flexible use of procedures Solution MethodComparisonSequential Conventional.61 ~.66 Demonstrated efficient.17 *.10 Solution Method at Posttest (Proportion of problems) ~ p =.06; * p <.05 Comparison students more likely to use more efficient method and somewhat less likely to use the conventional method

23. IES Conference 200823 Gains in flexible knowledge of procedures F(1,31) = 7.73, p <.01

24. IES Conference 200824 Gains in conceptual knowledge No Difference

25. IES Conference 200825 Summary of Study 1 Comparing alternative solution methods is more effective than sequential sharing of multiple methods ◦ Improves procedural transfer and flexibility ◦ In mathematics, in classrooms

26. IES Conference 200826 Comparison can help: Now what? Replicated findings for fifth graders learning computational estimation Goal: Develop guidelines for using comparison to support mathematics learning Starting Point: Standard classroom practices

27. IES Conference 200827 What are teachers doing? US teachers use comparison in 8th grade math lessons (average 4 per lesson) Types of comparisons (used with equal frequency): 1.Compare two similar problems with same basic solution 2.Compare two moderately similar problems or solutions 3.Compare a problem to a mathematical rule or principle 4.Compare a problem to a non-mathematical situation (Richland, Holyoak & Stigler, 2004 analysis of TIMSS videos)

28. IES Conference 200828 What are teachers doing? May not be using comparison well ◦ Teachers, rather than students, initiate comparisons and make links between examples ◦ When they present multiple solutions, rarely provide support for or discuss comparisons ◦ Don’t know which types of comparison support learning  e.g. Comparisons to contexts from different domains rarely support learning in laboratory studies. (Richland, Holyoak & Stigler, 2004; Richaland, Zur & Holyoak, 2007; Chazan & Ball, 1999)

29. IES Conference 200829 What about Algebra I textbooks? Compare two similar problems with same basic solution method (Equivalent Equations) Bellman,A.E., Bragg,S.C., Charles, R.I., Hall,B., Handlin, W.G., & Kennedy, D. (2007) Algebra 1, Pearson Education Inc, Pearson Prentice Hall

30. IES Conference 200830 Hollowell, K.A., Ellis, W., & Schultz, J.E. (1997). HRW Algebra. Holt, Rinehart, & Winston. Algebra I textbooks Compare problems with different structures (Different Problem Types )

31. IES Conference 200831 Algebra I textbooks Sobel, M.A., Maletsky, E. M., Lerner, N., & Cohen, L.S. (1985) Algebra One, Harper and Row Inc. Compare different solution methods to same problem (Solution Methods)

32. IES Conference 200832 Comparison in Algebra 1 textbooks Type of ComparisonPercent of worked examples Equivalent equations (similar problems; same method) 33% Different problem types (diff probs, solved same way) 1% Solution methods (one problem solved in two ways) 19% None - single worked examples47% Informal analysis of 10 Algebra I textbooks - chapter on multi-step linear equations

33. IES Conference 200833 What should be compared? Variety of comparisons are being used in math classrooms What are benefits and drawbacks to different types of comparisons? ◦ Study 1 confirms that comparing solution methods aids learning, as suggested by expert teaching practices ◦ Cognitive science literature suggests that comparing two problems solved with the same solution method should benefit learning

34. IES Conference 200834 Study 2 Research question: What are the relative merits of comparing solution methods vs. comparing problems? Research design: Randomly assigned to: ◦ Compare solution methods ◦ Compare problems that:  Are very similar (Equivalent)  Have different problem features (Different problem types)

35. IES Conference 200835 Types of comparison Solution Methods (one problem solved in 2 ways) Problem Types (2 different problems, solved in same way) Equivalent (two similar problems, solved in same way)

36. IES Conference 200836 Study 2 Method Participants: 161 7th & 8th grade students from 3 schools (more diverse sample) Design: ◦ Pretest - Intervention - Posttest - 2 week Retention ◦ Replaced 3 lessons in textbook ◦ Randomly assigned to  Compare Solution Methods  Compare Problem Types  Compare Equivalent ◦ Intervention occurred in partner work ◦ Assessment adapted from Study 1

37. IES Conference 200837 Conceptual knowledge results F (2, 153) = 5.76, p =.004,  2 =.07 *

38. IES Conference 200838 Procedural knowledge results No differences, even on novel problem types

39. IES Conference 200839 Flexible use of procedures F (2, 153) = 4.96, p =.008,  2 =.06 *

40. IES Conference 200840 Flexible knowledge of procedures F (2, 153) = 5.01, p =.008,  2 =.07 *

41. IES Conference 200841 Summary Across studies, Comparing Solution Methods often supported the largest gains in conceptual knowledge, procedural knowledge and procedural flexibility ◦ Supported attention to multiple methods and their relative efficiency, which both predicted learning

42. IES Conference 200842 Guidelines for using comparison Provide a written record of examples ◦ Leverage current use of worked examples in textbooks Contrast important dimensions in the examples, such as problem features or solution methods ◦ Contrasting correct and incorrect solution methods can help too (Kelley Durkin, IES pre-doc research) Have students compare a familiar method to an unfamiliar method Invite comparisons by using common labels and prompting for specific comparisons, including efficiency of the methods Be sure students, not just teachers, are comparing and explaining Incorporate some direct instruction

43. What’s next? Teacher Professional Development for using comparison in Algebra I courses Type of comparison matched to prior knowledge and sequencing different types of comparison

44. IES Conference 200844 Acknowledgements For slides, papers or more information, contact: b.rittle-johnson@vanderbilt.edu Funded by a grant from the Institute for Education Sciences, US Department of Education Thanks to research assistants at Vanderbilt: ◦ Holly Harris, Shanelle Chambers, Jennifer Samson, Anna Krueger, Heena Ali, Kelley Durkin, Kelly Cashen, Calie Traver, Sallie Baxter, Amy Goodman, Adam Porter, John Murphy, Rose Vick, Alexander Kmicikewycz, Jacquelyn Beckley and Jacquelyn Jones And at Michigan State: ◦ Kosze Lee, Kuo-Liang Chang, Howard Glasser, Andrea Francis, Tharanga Wijetunge, Beste Gucler, and Mustafa Demir And at Harvard: ◦ Martina Olzog, Jennifer Rabb, Christine Yang, Nira Gautam, Natasha Perova, and Theodora Chang