2Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement
3Overview of Spelling-CBM Assessment Training Session Part of a training series developed to accompany the AIMSweb Improvement System.Purpose is to provide the background information and data collection procedures necessary to administer and score Spelling Curriculum Based Measurement (S-CBM).Designed to accompany:AIMSWeb Training Workbook: Administration and Scoring of Spelling CBM for Use in General Outcome MeasurementStandard Spelling ProbesAIMSweb Web-based SoftwareTraining Video
4Training Session Goals Brief review of Curriculum Based Measurement (CBM) and General Outcome Measurement (GOM).Its PurposeIts OriginsLearn how to administer and score Spelling - Curriculum Based Measurement (S-CBM) through applied practice.
5General Outcome Measurements from Other Fields Medicine measures blood pressure, weight, temperature, weight.Federal Reserve Board measures the Consumer Price Index.Wall Street measures the Dow-Jones Industrial Average.Companies report earnings per share.McDonald’s measures how many hamburgers they sell.Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6
6Common Characteristics of General Outcome Measures Simple, accurate, and reasonably inexpensive in terms of time and materials.Considered so important to doing business well that they are routine.Are collected on an ongoing and frequent basis.Shape/inform a variety of important decisions.
7CBM is Designed to Provide Educators With… The same kind of evaluation technology as other professions!Powerful measures that are:SimpleAccurateEfficient indicators of student achievement that guide/inform a variety of decisionsAdministration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6
8General Outcome Measures in Spelling Not a measure of performance on a particular lesson or unitA broader indicatorLevel of general spelling skills relative to peersDegree of progress in spelling relative to self or peers
9Things to Remember About General Outcome Measures in Spelling “Signs” of general achievementNot everythingImportant thingsStandardized testsAdministrationScoringInterpretationResearchedReliability and validityAccuracy of measurementSensitive to improvement in short periods of timeDesigned to be as short as possible (do-ability)Linked to decision making for promoting positive achievement
10Spelling CBM Traditional spelling tests versus S-CBM Shorter duration (2 minutes)Spelling words dictated carefully at set paceRandom sample from a large pool of graded spelling wordsCombination of wordsWords students have already learnedWords they will learn to spell in upcoming weeks or monthsExact and sensitive way to assess spelling improvementCorrect letter sequencesCorrect word spellings
12Summary of Spelling Conditions Timing2 minutesNew word dictated every 7 seconds (every 10 seconds: grades 1 & 2)ArrangementsGroupIndividualWhat is scoredNumber of letters spelled in correct sequenceNumber of words spelled correctly
13Standard DirectionsStudents have lined paper numbered 1–12 (Grades 1 & 2) or 1–17 (Grade 3-8)and a pencil.Say, “We’re going to take a 2-minute spelling test. I am going to say some words that I want you to spell on the sheet of paper in front of you. Write the first word on the first line, the second word on the second line and so on. I’ll give you   seconds to spell each word. When I say the next word, write it down, even if you haven’t finished the last one. You will receive credit for each correct letter written. Are there any questions? [Pause] Let’s begin.”Say the first word and start your stopwatch (or tape recorder)
14Standard DirectionsSay each word twice. Use homonyms in a sentence (provided)Say a new word every 10 or 7 secondsSay, “You should be on the fifth word which is…”Monitor students to ensure they are writing on the correct lineDo not respond to student questions during the testAfter two minutes, say, “Stop. Put your pencils down.”Good to begin with short practice test to reduce novelty.Shortened directions available for well-experienced students.
15Standard Assessment Spelling Lists Benchmark: 3 standards lists for each grade (1–8)Screen and identify students who are at riskMonitor progress and improvement in fall, winter, and springGuide program evaluation decisions & improve accountabilityProgress Monitor: 30 standard lists for each gradeProvide practical way to write individualized progress goalsMonitor progress and determine rate of improvementsProvide accountability data
17Administration and Scoring of Spelling-CBM What Examiners Need To Do…Before testing studentsWhile testing studentsAfter testing students
18BEFORE Testing Student needs: Numbered, lined paper with the exact number of lines (12 or 17)Saves timeReduces potential confusionMakes scoring easierAnswer sheets available as PDF documentsPencil
19BEFORE Testing Examiner needs List of words with time interval clearly markedIncludes sentences for homonymsImproves ease of scoringTiming device
20Gather Assessment Materials Download and review S-CBM ManualDownload and reproduce student answer sheetsDownload Standard Spelling Assessment ListsCurriculum independentAllow decision makingMinimize between-teacher, between-school differencesStopwatch or timerClock on the wall is inaccurate, inefficientTiming tapeAudio tape with recorded tones at prescribed intervalsAllows better attention to words and studentsList of students to assess
21Arrange Testing Environment Quiet, without distractionsMay be 1:1, small group, whole classroomConsider a simple practice testUse very simple wordsAllows students to adjust to paceShow student how they get credit for unfinished wordsBe sure you can monitorEspecially those students who “get lost” easilyBe available to point to the number of the next word
22Things to Keep in Mind Keep examiner out of the process Keep number of words said to minimumSome students try to spell everything saidOnly homonyms have sentences (say word, say sentence, repeat word)Roam and projectLoud enough for all students to hearBe sure head is turned toward studentsMonitor, monitor, monitorAlert for lost studentsBefore they become frustrated and quitBefore they call out that they are lostAvailable to point to the next number
23Things to Keep in Mind Say the number of the word periodically Younger students, say, “You should be on the fifth (tenth) word which is…”Older students find this intrusive, so don’t say the numbersAvoid answering questions during testingStudents may comment: You’re going too fast, I’m lost, What number are we on?Deal with this in practice testsPoint to orient studentAdhere to end of timingInterruptionsRe-administer later with another spelling listConsider that some students need small group or 1:1
24Helpful Hints Ensure a quiet and distraction-free testing environment Students must be able to hear the examinerOffer no assistance, even when you know the child is capable of moreComplete reliability checksBefore and after data collection with all examiners to ensure reliable administration and scoringAdditional reliability checks at least once a yearPrevent examiner “drift”Use same Benchmark examiner for the same students.Younger children tend to perform better with familiar examiners.• Make sure the testing environment is quiet and free from distractions. Students must be able to hear theexaminer during the Phonemic Segmentation Fluency task.• Complete reliability checks before data collection with all examiners to ensure reliable administration.Additional reliability checks should be done at least once/year for all examiners, even those familiar withtesting.• If possible, try to have the same examiner during each Benchmark period test the same students. Youngerchildren tend to perform better with familiar examiners. Periodic reliability checks will help with examiner“drift,” or a tendency to provide additional assistance when you know a child is capable of completingmore of each task.
25Things You Need DURING Testing Follow the standardized directions:Spelling-CBM is a standardized testAdminister the assessment with consistencyRemember it’s about testing, not teachingDon’t teach or correctNever practice using the standard spelling lists
26Things to Do AFTER Testing Score Immediately!Determine the words spelled correctlyCircle the number beside the words spelled correctlyAward full pointsDetermine the correct letter sequences for misspelled words (see next slide)More likely to show changeProvides diagnostic and positive feedback
27c a t 1 2 3 4 sequences correct Spelling Scoring Correct: Count each sequence of 2 lettersSpaces before and after the word count as “letters”Consider: break through versus breakthroughExample: catc a tsequences correct
28fisshing _ f i s s h i n g _ (8) cat _ c a t _ (4) Spelling Scoringstore _ s t o r e _ (6)stor _ s t o r e _ (6)fishing _ f i s h i n g _ (8)fisshing _ f i s s h i n g _ (8)cat _ c a t _ (4)catz _ c a t z _ (4)
29store _^s^t^o^r^e^_ 6 (6) stor _^s^t^o^r e _ 4 (6) Spelling Scoringstore _^s^t^o^r^e^_ 6 (6)stor _^s^t^o^r e _ (6)fishing _^f^i^s^h^i^n^g^_ 8 (8)fisshing _^f^i^s s h^i^n^g^_ 7 (8)cat _^c^a^t^_ (4)catz _^c^a^t z _ (4)
30Spelling Scoring, Continued Tricks:Use master sheet with all correct sequences pre-counted.Then you only need to count the scoring for the few incorrect sequences (errors) on each paper.A complete list of spelling rules can be found in the Appendix of your workbook.Please review and become more familiar with the more unusual errors.
31SummaryYou now have the building blocks to begin Spelling - CBM to ensure literacy growth.Practice to Automaticity — You’ll get more efficientGet Checked Out by a colleague using the Accuracy of Implementation Rating Scale (AIRS) for accuracy and efficiencyStay in Tune by periodically checking AIRS
35If the Student Misses a Few Words… Any missing single letter equals 2 missing CLS.Any 2 letters together that are missing equals 3 missing CLSAny incorrectly inserted letter equals 1 missing CLSNEXT — Score Practice List 2: Lester
36Answer: Lester WSC = 0 CLS = 52 NEXT — Score Practice List 3: Calvin