Measuring and Interpreting Test Results for Teaching Core Standard Expectations

From a very recent research report Knowledge of mathematics is crucial to educational and financial success in contemporary society and is becoming ever more so. High school students’ mathematics achievement predicts college matriculation and graduation, early-career earnings, and earnings growth

Elementary school students’ knowledge of fractions and division uniquely predicts their high school mathematics achievement, even after controlling for a wide range of relevant variables suggesting that efforts to improve mathematics education should focus on improving students’ learning in those areas.

Robert S. Siegler, Greg J. Duncan, Pamela E. Davis- Kean, Kathryn Duckworth, Amy Claessens, Mimi Engel, Maria Ines Susperreguy, and Meichu Chen, Early Predictors of High School Mathematics Achievement, Psychological Science, June 14, 2012.

Close examination of test questions, and student responses including to the distracters, can tell us a tremendous amount about these issues. This is what I would like to discuss today.

Close examination of test questions, and student responses including to the distracters, can tell us a tremendous amount about these issues. This is what I would like to discuss today.

Grade 3

Percent choosing each answer Percent CorrectA %B %C %D %

Comments It is literally impossible for this to happen if students actually do understand the area model. So the 91% correct response to the first means something entirely different.

The First Problem What of (B) (30%)? It seems that these children counted the number of WHITE regions A B C D 11% 30% 53% 5%

The Second Problem What of (A) (38%)? It seems that these children also counted the number of WHITE regions In both situations, it is clear that they had very little understanding that the model worked with EQUAL AREAS A B C D 38% 44% 8% 10%

And look at the results for division of fractions.

Grade 6 Percent choosing each answer Percent CorrectA %B %C %D %

The First Problem Note the number of students who simply multiplied (B). 9% Added top and bottom Too many checked (D), so we could not measure other errors. These are issues with the teaching of the subject. A B C D 9% 24% 37% 31%

The Second Problem 11%, about the same as before added top and bottom. Too many checked (D), so we could not measure other errors, including the dominant “multiplication” These again are issues with the teaching of the subject. A B C D 8% 37% 11% 44%

Comments Here, the distracters tell us quite a bit about what is going on. Things would have been clearer in the second example if there had been a response (a/b)/(c/d) = (ac/bd) as was the case with the first example. But it appears safe to suggest that this lack was the reason for the increase in “not given” in the second question.

Grade 6 Percent choosing each answer Percent CorrectA %B %C %D %

The First Problem The correct answer is (D), which is unfortunate since it seems that a common strategy here is to check (D) when the students don’t know. so we could not measure errors, though they seem to all be in placing the decimal – a problem with understanding magnitude. A B C D 27% 22% 6% 45%

The Second Problem Note that more answered (B) than the correct answer (C) (A) and (B) both show a lack of understanding of magnitude. Too many checked (D), so we could not measure other errors. These again are issues with the teaching of the subject. A B C D 7% 36% 34% 24%

And look at the results for fraction addition.

Grade 5

The First Problem The correct answer is (D) which a simple size estimate shows which is somewhat unfortunate. (A) could have just been a simple arithmetic error, but again ¾ + 2/7 > ¾ + ¼ = 1, so it should not have happened. But what of (B)? A B C D 18% 37% 9% 37%

The Second Problem Note that more answered (A) than the correct answer (C) (A) shows the relatively surprising error of adding top and bottom separately on the frational parts of the mixed number These again are issues with the teaching of the subject. A B C D 42% 4% 34% 20%

We have huge reasons to make sure students learn fractions and division completely and carefully. But, for mathematical reasons, student difficulty with fractions and division rest on earlier problems with place value

But these difficulties actually start with PLACE VALUE as it is taught in the earliest grades.

Grade 1 Percent choosing each answer Percent CorrectA %B %C %D % As one can see, the percent correct is not always a good measure of what is going on. But the distracters tell us quite a bit.

Grade 2 Percent CorrectA %B %C %D % The distracters here give very important, troubling but consistent information.

This is an inevitable consequence of lessons like the following

Typical U.S Lesson on Place Value Note Focus on Manipulatives Linear model For 10’s, area for 100’s, volume for 1000’s. This is Illogical and confusing What represents 10,000?

This is codified in the U.S. curriculum to the extent that if a third grade text does not have this lesson it will typically be rejected as being mathematically insufficient.

By contrast here is how this topic is handled in the high achieving countries

First Grade Russian Text: Place Value

First Grade: Russian Text: Models for Place Value. Especially note use of Decimeters for putting (2 place) Place value on number line

Second Grade: Russian Text. Note consistency Of models for higher Places and tight focus

E Even 1000’s are Consistent Consistent models make Comparison easier. Note attention to Comparisons

“Bundles” In Core Standards

This should warn us that there is much, much more going on in Core Standards than one might think. I believe care is necessary in choosing our Core Standard “experts.”