Travelling the road to expertise: A longitudinal study of learning Kaye Stacey University of Melbourne Australia

A journey with 3204 students as they learn to understand decimal notation over seven years (Grades 4 – 10) repeating 4,08 Grades 4 – 6 “primary”; Grades 7 – 10 “secondary”

Our decimals work in summary Understanding how students think about decimals Tracing students’ progress in the longitudinal study Looking at teaching interventions Creating computer games using intelligent tutoring and AI (Bayesian nets) Developing CD and website for teachers Thanks to Vicki Steinle Liz Sonenberg Ann Nicholson Tali Boneh Sue Helme Nick Scott Australian Research Council many U of M honours students Dianne Chambers teachers and children providing data

Why decimals? Practical Importance Is my blood alcohol over 0.05% or not? Is my p value over 0.05? Links to metric measuring Fundamental role of number in mathematics (e.g. understanding 0) Known to be complex* with poor learning A case study of students’ growth of understanding, which was able to start from a good research base (incl M.Swan) * place value, fractions, density of real numbers etc

In preparation for the journey: Who? 3204 students (12 schools, all SES, volunteer teachers) Transport: ordinary teaching Territory and map – see later The destination – “understanding decimals”

The destination: understanding decimal notation Why is such a simple rule as rounding hard to remember? A convention carrying distributed intelligence

Sample Cohort study aimed to follow as many students as possible for as long as possible 1079 students first tested in primary – nearly 60% followed to secondary school over 600 students completed 5,6 or 7 tests (i.e. followed into a third or fourth year) Quantitative analysis of longitudinal data conducted by Dr Vicki Steinle (PhD thesis)

Sample data Two tests per year – one per “semester” 9862 tests completed Students tracked for up to 7 semesters Tests average 8.3 months apart. Absentees not chased.

Interconnections between the map and the mapping tool “It is only by asking the right, probing questions that we discover deep misconceptions, and only by knowing which misconceptions are likely do we know which questions are worth asking”, (Swan, 1983, p65). The longitudinal study uses one type of question: which of two decimals is larger? e.g or 0.75 (CSMS item, 1981) 4.8 or 4.63(Resnick et al, 1989)

Similar items – different success Select the largest number from 0.625, 0.5, 0.375, 0.25, Correct: 61% Select the smallest number from 0.625, 0.5, 0.375, 0.25, Correct: 37% Why such a large difference? Foxman et al (1985) Results of large scale “APU” monitoring UK. All sets given here as largest to smallest; not as presented.

Common patterns in answers LargestSmallest correct well known error “longer-is-larger” identified 1980s “shorter-is-larger”

Persistent patterns Select the smallest number from 0.625, 0.5, 0.375, 0.25, predicted from other responses by our sample Our sample is typical

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Longer-is-larger expert Map prior to 1980 Early explorers Comparison used by Brueckner 1928

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Map by 1990 Early explorers Longer-is-larger Shorter-is-larger expert Use patterns of responses to sets of comparison items – Sackur, Resnick, Peled and others

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Map for this longitudinal study Longer-is-larger (L) Shorter-is-larger (S) expert (A1) other A U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1

Decimal Comparison Test (DCT2) We now have better version set of similar items set of similar items

Decimal Comparison Test (DCT2) We now have better version set of similar items set of similar items

Complex test, easy to complete Several items of each type Codes require consistent responses Items within types VERY carefully matched

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 whole number analogy 4.8, 4.63 additional information – zero makes small “fat columns” 4.71 has 71 tenths, has 82 hundredths Longer-is-larger

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 “unclassified”: we don’t know everything wrong – quite smart!

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 expert on comparison task truncation – no meaning for later dec pl; failed algorithms – e.g. comparison of space and Only known to be OK on easy items on DCT2 – maybe not OK on harder items vs

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Shorter–is-larger analogies with fractions or negatives (e.g. 0.4 < 0.3) some place value considerations – all thousandths smaller than all hundredths e.g greater than Distinguish behaviour and way of thinking

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Shorter–is-larger analogies with fractions or negatives (e.g. 0.4 < 0.3) some place value considerations – all thousandths smaller than all hundredths e.g greater than Distinguish behaviour and way of thinking Can subdivide these with improved DCT

Example of S thinking Courtney - a “text book” case derived from research interviews – not too obvious for our student teachers Hidden Numbers Making the biggest and smallest numbers Number Between

S behaviour: “through the looking glass” < 5.62 (because larger whole number, so reverse) The mirror is a powerful metaphor underpinning some everyday and mathematical concepts (Lakoff & Johnson) Fractions (and hence decimals) and negative numbers as “mirror images” of whole numbers, so everything is reversed (Stacey et al, PME, 2001) negative is an additive inverse reciprocal is a multiplicative inverse Can even lead to getting all comparisons wrong (U2)

S behaviour: false number line analogies ThHTUthth Further right means smaller “further right” than 5.62 Consequences of 0 and Units as “mirror position” 0 vs 0.6 Further confusion with 1 as “mirror position” for fractions Some S-like students get classified into A by DCT2

Characteristics of test Reliable (56% of students in same code after one semester!) Generally agrees with interviews Weakness is in diagnosing expertise with consequence that all our estimates of expertise are overestimates some “experts” follow rules without understanding some “experts” cant do other tasks (e.g. could reshelve books in library, but don’t know metric properties of decimals) Test has been improved over life of study – now have improved versions – cycle of improvement

Examples of student journeys

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1

Where are the students in each grade?

Test- focussed prevalence

Prevalence of L codes L drops exponentially (L = 440exp(-0.45*grade)) L2 about 5% in Grades 5-8: some just accumulating facts, not changing concepts “just a few little things still to learn”

Prevalence of A codes by grade One quarter expert at Grade 5, one half in next 4 years, one quarter never An issue for adult education e.g. “death by decimal” Remember these are overestimates of expertise! A1 = expert

Prevalence of A codes by grade Note: 10% in the non-expert A category / 4.45 is a difficult item – more about these later A1 = expert

Prevalence of S codes 10-15% Peaks in early secondary - probably curriculum effect * This graph was incorrect in the original, and is corrected here.

Within the S region Around 5% in S1 in all grades 0.6 < 0.7  0.5 <0.125 x Around 10% in S3 in all grades (more Grade 8) 0.6 < 0.7 x 0.5 <0.125 x Early studies did not ask the S3 question! S3 Possibilities: analogy with fractions (one sixth, one seventh) analogy with negative numbers (-6, -7) doesn’t include any place value considerations

Another look at prevalence Which towns are most visited?

Another look at prevalence Previously “test-focussed prevalence” - how many are at each town at each time? Also “student-focussed prevalence” – how many students visit each town sometime on their journey? S.F.P. > T.F.P. S.F.P. measured over time – in primary school (Gr 4 - 6) and in secondary school (Gr 7 – 10) S. F. P. increases if you make more frequent observations, so reports are under-estimates.

Student-focussed prevalence SFP of A1 is 80% SFP of non-expert codes 15% - 30+% SFP primary different to SFP secondary

Differences between TFP & SFP TFP S < 25% SFP S = 35% TFP S1 = 6% (most grades) SFP S1 (pri) = 17% SFP S1 (sec) = 10% TFP estimates, SFP under-estimates

Persistence Where do students stay the longest?

Questions about persistence Persistence = percentage of students who retest in the same code after 1 semester Persistence tells us: Which towns are the most attractive, in the sense of hard to leave? Do students stay in one place for a long time, or do they move around? Do experts stay as experts?

Persistence of A1 A1 is most persistent (Hurrah!) 90% retest as A1 next semester, 80% of A1 always stay A1 20% of “experts” don’t stay as expert – less than lasting understanding Quick instruction before the test is better than nothing! A1 students often go to A2 lucky guesses with truncation strategies which fail to give comparison decision (e.g vs 4.45) different repairs to faulty algorithms (Brown & vanLehn “bug migration” 1982) or faulty concepts

Persistence over 1 semester Persistence of L1 around 30% Persistence of A2 and S3 increases with age MERGA 2005: Sec S about 40% more likely to remain in S than Pri S (stat. sig.) Effect holds across schools Conclusion: Naïve L ideas are challenged by secondary school but S and non-expert A ideas are supported by school practices (e.g. always rounding off) and new ideas in curriculum.

Persistence over 1, 2, 3, 4 semesters About 35% of L,S,U retest same after one semester About 15% retest same after 2.5 years Schooling is not impacting on ideas!

Proximity to Expertise Which town is the best place to be? From which non-A1 code is it most likely that a student who changes code* will become an expert on the next test? * Proximity is independent of persistence

Proximity to Expertise

General hierarchy A > U > S > L A2 is nearest (but maybe VERY LITTLE place value understanding ) U1 (unclassified) is high – why is not having a definite misconception better? * Proximity is independent of persistence

Proximity to Expertise S better than L in primary ( as inferred from previous research) L better than S in secondary – why? S pri student has 60% more chance of moving to expertise than S sec L pri student has 30% less chance of moving to expertise than L sec * Proximity is independent of persistence

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Proximity to A1 (primary)

A1 expert A2 A3 L2 L1 L4 S1 S5 S3 U2 U1 Proximity to A1 (secondary)

Lessons about learning (1) Variation in age when expertise is attained Too many never make it Many with little understanding hide it (e.g. A2) Schooling not IMPACTING on fundamental ideas – 15% persistence over 4 semesters Harder to shake ideas of students with a specific misconception – students need to be shown they have something to learn Importance of looking at prevalence in both ways

Lessons about learning (2) Different misconceptions have different causes and are impacted differently by the learning environment L1 - naïve, first guess without teaching, decreases in prevalence S – supported by features in the curriculum, operating at deep psychological level, so reinforced especially in secondary school

“Just a few little things left to learn” Contrast between orientation to learning principles vs accumulating facts expert: a few math’l principles requiring mastery of a web of complex relations between them some students and teachers: a large number of facts to learn with weak links between them Maths education has a challenge to properly deal with this accumulated facts approach for research (DCT2 was weak here) for improving teaching and learning of such teachers and such students

How to use a detailed analysis of students’ thinking Study grappled with what grain size of detail is useful – eventually worked with two (but both finer and coarser possible) Coarse analysis is useful for human teaching Fine analysis is useful for machine teaching several games using artificial intelligence to present the items from which a student can see that there is something for him/her to learn learn something new. Good results from just a little attention to this.

Flying Photographer

Hidden Numbers Features of games and other instruction: students need to find out that they don’t know everyone needs to learn about the same principles different students need to learn in the context of different items

Their teaching No special treatment (but see our website & papers for many suggestions) Usually start decimals around Grade 4, e.g. as alternative notation for tenths Common models MAB, area (but better to use length) Often restricted to one or two place decimals for a long time (Brousseau and others comment on this) Rounding to 2dp standard in later years 0.13

Better to use length Stacey, Helme, Archer, Condon – Ed. Studies Math.(2001) Accessibility leads to better retention and classroom discussion

Thank you