1. Using Assessment to Inform Instruction: Cautions & Recommendations Karen Wixson, Univ. of Michigan Sheila Valencia. Univ. of Wash. IRA, Toronto, May, 2007

2. Why should we be cautious when using test scores to guide instruction?

3. Problems with Test Scores Teaching the test The issue of standards--alignment? Ceilings and floors Masking individual differences

5. NAEP Scores vs State Scores NY Times 11/05

6. Ceilings and Floors

7. Johnny cant read because Johnny needs phonics By Harold Hochstatter Chairman of Senate Education Committee Seattle Times Op Ed, Feb. 25, 1998

9. Word Id Comp. Vocab Fluency Sandy Tomas Trang Sandy Trang Trang Joey Martin JesseAndrew Andrew Jesse Jesse

10. H & T %ile CWP M Acc (%) Rate (wpm) Exp. (1-4) ITBS (NCE) Dibels >75151 140 89 99 169 142 3434 70 48 No risk 50-74118 128 94 93 126 137 2323 42 64 No risk 25-4990 97 93 96 97 99 2323 45 36 Some risk 0-2488 45 94 96 94 47 2121 66 31 At risk Grade 4 Student Profiles

11. What does all this mean? We cant trust any single piece of information to screen perfectly or guide instructional decisions Need an inquiry approach to using assessment information

12. What is an Inquiry Approach? All assessment info is viewed as a springboard to understanding student learning--not as static facts We need to use easily accessible information and questions to help figure out where to go with instruction

13. Easily Available Information State/district test results WCPM Running Records Writing samples Spelling tests/samples

14. How do we use this information? Look at available evidence together Check to see if information is consistent –With itself –With classroom evidence –With teacher judgment

15. Asking Questions If ALL evidence points to student doing well, ask what, if any, confirming information is needed to plan instruction If ALL evidence points to problems, ask what additional information is needed to plan instruction If evidence is inconsistent, ask what might be going on and what information is needed to clarify before planning instruction

16. Ongoing Questioning With each additional piece of information ask: Does this fit with what we already know about the student? –If yes, do we have all the information we need to plan instruction –If not, generate and test hypotheses about whats going on

17. How do we get more information easily? High utility assessments –IRI or IRI-like procedures –Running Records Work samples/observations –Writing –Spelling –Oral language

18. The Many Uses of IRI-like Procedures Much more than a set of scores Qualitative Analyses –Comprehension Oral reading, silent reading, listening Literal and inferential comprehension questions Retellings –Word recognition Comparison of word rec in and out of context Word rec strategies –Vocabulary –Fluency

19. Probing Retelling/Comp Performance Write/tell about the story you just read in your own words. The following sentences tell you something about the story you just read. Read (listen to) them and write/tell some more about the story. Place a check next to each idea that you think was in the story you just read.

20. Probing Strategy Usage with Think Aloud S: (reads title) Space Ship Earth T: What were you thinking when you read the title? S: A space trip to earth. (Reads text haltingly). Boy! I had a lot of trouble with that. T: What makes you think you had trouble? S: I kept messing up. T: What do you mean by messing up?

21. Think Aloud (contd) S: I kept reading sentences over. T: What do you think caused you to read over like that? S: Not understanding it. T: OK, did any of the words give you trouble? S: No

22. Pursuing Leads S: Harry lik..liked to walk to the store. He like to hop on the ker, cr, crub, crub, kir, crib, crib.. (22 sec on word) S: Mom is teaching me the sound of the letters but theres all kinds of sounds for the u T: I can hear that you are trying some out S: (Spontaneously goes on trying to sound out word), cube, kirb, crub, cr.. T: The word is curb S: Well, I would expect a er there because we have a chart in class that has the er sound…

23. Probing Word Recognition Reader Text Analysis Could/cloudcloudsSC; uses mng/syn Then/tundra/ tundraherds Thunder- heads multisyllabic;uses grapho-phonic motorlessmotionlessMulti; suffix Torn/tornadotornadoSC; using mng. Unc/oncingoncomingMulti; prefix

24. Probing Vocabulary Observe performance on voc questions; add/ask new voc questions Observe oral reading errors for evidence of words not in speaking voc Probe understanding of words that are questionable--ask for synonyms, use in sentence, etc.

25. Example Student #1 Grade 4 –WCPM 140 –ITBS 48NCE Inconsistent information leads us to ask: Why might a student who is so fast (fluent) have such low comprehension?

26. Student 1 Follow-up Informal Reading Inventory –Accuracy--99% –Expression--4 (very fluent) –Passage comp--below grade level What next? –Listening comprehension –Vocabulary questions

27. Student #1--Results and What to try? Listening comp and voc OK (classic word caller) SLOW DOWN If that doesnt work try comp and/or monitoring strategies

28. Example Student #2 Grade 4 ESL student –WCPM, 117 (low risk) –Comp/state test, <second grade –Writing sample--below proficient Inconsistent info leads us to ask: Why might a student who is low risk on wcpm perform poorly on comprehension and writing?

29. Student #2 Follow-up Informal Reading Inventory –Word rec in context, second grade –Expression, 2.5 (borderline disfluent) No evidence of monitoring for meaning What next? –Vocabulary –Oral language

30. Student #2--Results and What to try? Low voc and oral English Try: –Second grade materials that are interesting/familiar –Exposure to grade level content to build background –Instruction in decoding, vocab, and monitoring for meaning

31. Example Student #3 Grade 2 student –WCPM, 73 (some risk) –ITBS, 80NCE Inconsistent information leads us to ask: Why might a student who is struggling with oral reading fluency do so well in comprehension?

32. Student #3 Follow-up Informal Reading Inventory –Acc, 97% –Expression, 3 (fluent) What next? Why? –Isolated word rec –Rate with repeated readings

33. Student #3--Results and What to try? Isolated word rec low--classic slow kid Decoding in context is much better than decoding out of context Reading is very slow Hyp--needs word rec work and practice, practice, practice

34. Student #4 Grade 2 –WCPM, 90 (low risk) –ITBS, 39NCE –Spelling--still emergent Questions: Why might someone with this rate not do better in comp? What does poor spelling say about word rec?

35. Student #4 Follow-up Informal Reading Inventory –Accuracy, 87%, grade level passages –Rate, 102 –Expression, 1 (non-fluent) What next? –Listening comp –Voc –Sight vocab/decoding

36. Student #4-- Results and What to try? Listening comp and voc high Sight words/decoding low Over-relying on context so try –Word rec instruction (both sight words and decoding strategies) –Use first grade materials that are less familiar

37. In Closing We often spend more time giving and scoring tests and less time thinking about the information we have Instead, we need to spend more time thinking about the information we have and the additional information we need

38. Oral Reading Think Aloud S: (reads title) Space Ship Earth T: What were you thinking when you read the title? S: A space trip to earth. (reads haltingly) Boy! I had a lot of trouble with this. T: What makes you think you had trouble? S: I kept messing up.