1. Opportunities and Challenges for Diagnosing Teachers’ Multiplicative Reasoning NSF DR-K12 PI Meeting December 3, 2010 Andrew Izsák University of Georgia

2. Diagnosing Teachers’ Multiplicative Reasoning Andrew Izsák, Jonathan Templin, Allan Cohen University of Georgia Joanne Lobato San Diego State University Chandra Orrill University of Massachusetts Dartmouth Supported by the National Science Foundation under Grant No. DRL-0903411. The opinions expressed are those of the authors and do not necessarily reflect the views of NSF.

3. Existing Approaches to Assessing Teacher Knowledge Composite measures –Count college mathematics courses teachers completed –Use item response theory to measure Mathematical Knowledge for Teaching Case studies –Investigate topics such as subtraction with regrouping, arithmetic with fractions, and functions

4. Essential Features of DTMR Assessing mathematical knowledge of middle grades teachers Emphasize knowledge needed for teaching: –multiplication and division of fractions and decimals –proportional reasoning –using problem situations and drawn models to develop general methods Combine mathematics education research with psychometric research on an emerging class of models called Diagnostic Classification Models (DCMs) One of the first projects to develop a test for DCMs using STEM education research

5. Opportunities Curricular trends –Reform-oriented curricula –Common Core Standards Need measures suitable for tracking growth and change in PD Large literature –Multiple components of reasoning –Glean attributes DCMs are multi-dimensional models that (compared to MIRT) can be reliably estimated with smaller samples and shorter tests Expand range of psychometric models applied to STEM education research

6. Fraction Attributes Referent Units –Understand units to which numbers refer Partitioning –Using whole-number multiplication to guide partitioning Iterating –Interpret to mean A copies of Appropriateness –Identifying multiplication and division situations

7. Example: Referent Unit & Iterating I.The diagram can show. II.The diagram can show 1. III.The diagram can show. Which of the following interpretations are sensible?

8. The Mastery Profile (Fractions) DCMs estimate an attribute mastery profile for each teacher:

9. Anticipated Uses Use as formative assessment to inform PD Detect growth and change in PD characterized as increased proficiency with the attributes Determine distribution of attribute patterns in large samples of teachers Examine relationships between attribute patterns and enactment of curricular materials Examine relationships between attribute patterns and student achievement

10. Challenge: What are Workable Attributes? Existing examples from psychometrics –Steps in numeric algorithms –Branches of mathematics Our criteria for attributes –Written responses provide reliable information –Separable from one another –Separate teachers –Cut across topics Cannot translate research findings directly –Multiple cycles of using attributes to write items and interviewing teachers

11. Example: Composed Unit Reasoning One week Mr. Compton drove to a training course that required him to drive 8/3 the distance he usually drives to work. He noticed that 24 min. had passed when he had drive half way to the course. How long does it take Mr. Compton to drive to work? Mr. Vargas gave the following task to his students: One student tried to model the problem using two number lines as shown but is stuck. How could you help the student? 0 min 0 km 24 min

12. Challenge: Item Design Machine scoreable –Multiple-choice –Constructed response items Case studies often rely on observed strategies to make inferences –Composed unit reasoning –Fractions as multiplicative relationships Numeric computation should not help find correct response –Teachers’ use of computation obscures access to their reasoning with quantities

13. Challenge: Item Design (Cont.) Attention to pedagogy can drive responses –Teachers are not always comfortable evaluating students –How teachers would teach a topic and what they know about a topic are not the same –Teachers can deflect mathematical issues by appealing to what their students can understand Drawings –Teachers do not always interpret diagrams in ways that we intend

14. Challenge: Balancing Constraints Psychometric modeling and interviewing teachers to investigate constraints that include –Grain-size of attributes –Item design –Number of attributes –Number of items per attribute –Number of items (test length) –Sample size

15. Proportional Reasoning Attributes Covariation and Invariance –Multiplicative relationship invariant as quantities co- vary Connections between Ratios and Fractions –Conceptual links between ratios and fractions Appropriateness –Direct proportion vs inverse proportion –Direct proportion vs linear relationship Multiplicative Object

16. Summary Develop one of the first tests for use with Diagnostic Classification Models (DCMs) based on STEM education research Opportunities –Assess select aspects of teachers’ multiplicative reasoning –Detect growth and change during professional development Challenges –Interpret the term “attribute” for mathematics education research on multiplicative reasoning –Design items that get at fine-grained aspects of multiplicative reasoning using drawn models

17. Expanded Fractions Attributes

18. Expanded Proportional Reasoning Attributes

19. Probability of Correct Response

20. The Q Matrix ItemReferent Units PartitionIterateApprop. Circles1010 Item 20101 Item 31000 Item 40110 … Item jq j1 q j2 q j3 q j4 …