1. Katherine L. McEldoon & Bethany Rittle-Johnson

2. Project Goals Develop an assessment of elementary students’ functional thinking abilities, an early algebra math skill Develop a model of knowledge progression 2

3. Functional Thinking A type of mathematical thinking which focuses on the relationship between two (or more) varying quantities, specifically the kinds of thinking that lead from specific relationships to generalizations of that relationship across instances. (Smith, 2008) Encapsulates important core components of early algebraic reasoning, such as generalization and covariation. (Carraher, Martinez, & Schliemann, 2008) 77 Out = (In x 2) + 1 Y = 2X + 1 The table shows how the “In” numbers are related to the “Out” numbers. When a 38 goes in, what number comes out? A.41B.51 C. 54D. 77 3

4. Functional Thinking Performance – Grade 4 The table shows how the “In” numbers are related to the “Out” numbers. When a 38 goes in, what number comes out? A.41 B.51 C.54 D.77 National Assessment of Educational Progress (NAEP), National Performance results in Mathematics at Grade 4; 2007 4

5. Function Tables Focus: Functional Tables Determining Values and Rules Typical Tasks (Carraher & Earnest, 2001; Schliemann & Carraher, 2000) Fill in the missing values in this table What is the rule for this table? Asked to select a rule from several choices Asked to write the rule verbally or symbolically 5 Column A Column B 26 37 48 59 610 1418 2125 4145

6. Column A Column B 26 37 48 59 6 14 21 41 Function Table Competencies Within function table problems, we isolated required competencies and used this as a basis for our assessment Loosely hypothesized order of difficulty:  Apply a Given Rule (prerequisite)  Determine Next Y value in Sequence  Determine Near Y value in Sequence  Determine Far Y value in Sequence  Recognize a Rule (verbal/symbolic)  Generate Rule Verbally  Generate Rule Symbolically 6 Column B = Column A + 4 B = A + 4

7. Wilson’s Construct Modeling Approach Wilson’s Four Building Blocks 1) Construct Map 2) Item Design  3) Item Score 4) Measurement Model Assess the student performance data to evaluate your construct map and items 7 Item Design Item Score Measure ment Model Construct Map

8. Item Design: Assessment We designed items that tapped each of these competencies Modified from e.g. Blanton; Schliemann; Warren, Cooper & Lamb Items varied in operation used in underlying function 33 responses to 11 items 16 of which had an additive underlying function Y = X + 2 10 had a combination underlying function Y = 2X + 2 8

9. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 9

10. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 10 Column A Column B 26 37 48 59 6 14 21 41

11. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 11 Column A Column B 26 37 48 59 6 14 21 41

12. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 12 Column A Column B 26 37 48 59 6 14 21 41

13. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule – Verbal & Symbolic 6. Generate Rule Verbally 7. Generate Rule Symbolically 13 What is a rule used in the table above to get the numbers in column B from the numbers in column A? A)Multiply the number in column A by 2. B) Divide the number in column A by 2. C) Subtract 2 from the number in column A. D) Add 2 to the number in column A.

14. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 14 Column A Column B 26 37 48 59 6 14 21 41 “The rule is that you add 4 to the A number to get the B number” What is a rule for figuring out what number belongs in column B?

15. Item Design: Assessment We developed items that tapped each of these competencies  Apply a Given Rule  Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 15 Column A Column B 26 37 48 59 6 14 21 41 “B = A + 4” Write this rule as a number sentence, using “A” to stand for any number in column A and “B” to stand for any number in column B.

16. Item Scores: Coding Coding Each response only tapped one competency Each was coded as correct or incorrect 16 Item Design Item Score Measure ment Model Construct Map

17. Data Collection: Procedure 231 second through sixth grade students Middle class suburban community Predominantly Caucasian population During one 40 minute class period 17

18. Measurement Model Based on Item Response Theory Item Response Theory encompasses a set of ways to mathematically model how both Student Ability Estimate and Item Difficulty are related to a student’s Item Responses It is a useful methodology to use when evaluating an assessment instrument both in terms of its ability to accurately estimate student ability but it also give metrics of the quality of each item on the instrument. 18

19. Measurement Model Wright Map An Wright map generated by a Rasch model (a type of item response model) and was used in this evaluation Logit Scale (log-odds ratio) Student Ability Estimates Item Difficulties 19 --------------------------------------------------------------- 5 XX| XXXX| XX| 4 XXXXXX| XXXXXXXX| XXXXXX| XXXXXXXXXXXXXXXX| 3 XXXXXXXXXXXX| XXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXX| XXXXXXXXXXX| 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L4 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 0 XXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L2 L2 -2 XXXXXXXXXXXXXXXXXXXX|L2 L2 L2 XXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXX| -3 XXXXXXXXXXXXXX| XXXXXXX|L1 L1 L1 XXXX| XXX|L1 L1 -4 X| X| XX| | ================================================================== Student Ability ScoresItem Difficulty Scores

20. Wright Map An Wright map was generated by a Rasch model (a type of item response model) and was used in this evaluation Item difficulties based on the Wright maps were used in the development of our Construct map 20 XXXXXXXX| XXXXXXXXXXX|Item E 3 XXXXX-3-Item C Item D XXXXXXXXXX| XXXXXXXXXXXX| 2 XXXXXXXXX-2- XXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|Item B XXXXXXXXXXXXXXXXXXXXXXXXXX| 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|Item A Student Ability ScoresItem Difficulty Scores This item has a difficulty level of.98, meaning that the average student has a ~0.47 probability of getting it correct This item difficulty is 3.1logits, or the average student has a ~0.28 probability of getting it correct Measurement Model Wright Map

21. XXXXXXXXXXXXXXXXXX| | XXXXXXXXXXX| | 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXX| | 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|7-16 | XXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXX|6-6 | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|6-14 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-15 | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-13 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|3-12 | XXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|3-4 3-5 | -2 XXXXXXXXXXXXXXXXXXXX|5a-9 2-10 2-11 | XXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXX| | -3 XXXXXXXXXXXXXX| | XXXXXXX|1-3 1-7 1-8 | XXXX| | XXX|1-1 1-2 | -4 X| | X| | XX| | ======================================================================================= Each 'X' represents 0.4 cases =======================================================================================0 21 7) Generate Rule Symbolically 6) Generate Rule Verbally 5) Recognize a Rule 4) Determine Far Y value in Sequence 3) Determine Near Y value in Sequence 2) Determine Next Y value in Sequence 1) Apply a Given Rule Wright Map Addition Functions

22. Wright Map Combination Functions ---------------------------------------------------------------------------------------| 4 XXXXXXXXX| | XXXXXXXX| | XXXXXXXXXXX|4-8 | 3 XXXXX|3-6 4-7 6-9 7-10 | XXXXXXXXXX| | XXXXXXXXXXXX| | 2 XXXXXXXXX| | XXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXX| | 1 XXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|2-5 | XXXXXXXXXXXXXXXXXXXXXXXXXX| | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5b-4 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5a-3 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -2 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXX|1-2 | XXXXXXXXXXXXXXX| | -3 XXXXXXXXX| | XXXXXXXXXXX|1-1 | XXXXXXX| | -4 XXXXXXXXX| | ======================================================================================= Each 'X' represents 0.4 cases 22 7) Generate Rule Symbolically 6) Generate Rule Verbally 5) Recognize a Rule 4) Determine Far Y value in Sequence 3) Determine Near Y value in Sequence 2) Determine Next Y value in Sequence 1) Apply a Given Rule

23. Construct Map A representation of the continuum of knowledge that people are thought to progress through for the target construct (Wilson, 2005) Placed competencies into a hierarchy based on We used item difficulty scores from IRT measures Their clumping on the Wright maps From theory 23

24. Mapping of Competencies into Construct Map Levels Level Description Competencies Level 4: Generate Symbolic Rule - Generate an explicit symbolic rule Level 3: Generate & Use Verbal Rule - Generate an explicit verbal rule - Complete a function table with missing values Level 2: Recognize Rule & Determine Next - Select a correct rule out of several choices - Determine the next Y value in a function sequence Level 1: Apply Rule - Use a given rule to determine new Y values 24 7. Generate Rule Symbolically 6. Generate Rule Verbally 5. Recognize a Rule (verbal/symbolic) 4. Determine Far Y value in Sequence 3. Determine Near Y value in Sequence 2. Determine Next Y value in Sequence 1. Apply a Given Rule

25. Benefits of a Construct Modeling Approach First, it elucidated the relative difficulty of functional thinking abilities, and at times this was not in line with our predictions. Second, the resulting assessment is a criterion referenced measure which is particularly appropriate for assessing Students’ ability estimate levels Learning gains from an intervention 25

26. Summary Identified key competencies that are important for elementary-level functional thinking, with a focus on function table problems These competencies were then incorporated into an assessment Student performance data was used to develop a construct map, or proposed knowledge progression, of elementary-level functional thinking abilities The resulting construct map provided insight into the acquisition of functional thinking knowledge in elementary-school students This can be used as a research tool, and to guide instructional sequences for students 26

27. Thank you For more information: http://peabody.vanderbilt.edu/earlyalgebra.xml 27 The first author is supported by a predoctoral training grant provided by the Institute of Education Sciences, U.S. Department of Education, through Grant R305B040110 to Vanderbilt University. The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education.

28. 28

29. Wright Map - Multiplication --------------------------------------------------------------------------------------- 5 XXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXX| | 4 XXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | 3 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|7-7 | XXXXXXXXXXXXXXXXXX|3-4 | 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-3 3-5 | XXXXXXXXXXXXXXXXXX|6-6 | 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5-1 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|2-2 | XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXX| | -3 XXXXXXXXXXXXXXXXXXXXXXXXXX| | ======================================================================================= Each 'X' represents 0.3 cases ======================================================================================= 29 1.Apply a Given Rule 2.Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule (symbolic) 6. Generate Rule Verbally 7.Generate Rule Symbolically