2. Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement

3. Overview of Spelling-CBM Assessment Training Session
Part of a training series developed to accompany the AIMSweb Improvement System.
Purpose is to provide the background information and data collection procedures necessary to administer and score Spelling Curriculum Based Measurement (S-CBM).
Designed to accompany:
AIMSWeb Training Workbook: Administration and Scoring of Spelling CBM for Use in General Outcome Measurement
Standard Spelling Probes
AIMSweb Web-based Software
Training Video

4. Training Session Goals
Brief review of Curriculum Based Measurement (CBM) and General Outcome Measurement (GOM).
Its Purpose
Its Origins
Learn how to administer and score Spelling - Curriculum Based Measurement (S-CBM) through applied practice.

5. General Outcome Measurements from Other Fields
Medicine measures blood pressure, weight, temperature, weight.
Federal Reserve Board measures the Consumer Price Index.
Wall Street measures the Dow-Jones Industrial Average.
Companies report earnings per share.
McDonald’s measures how many hamburgers they sell.
Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6

6. Common Characteristics of General Outcome Measures
Simple, accurate, and reasonably inexpensive in terms of time and materials.
Considered so important to doing business well that they are routine.
Are collected on an ongoing and frequent basis.
Shape/inform a variety of important decisions.

7. CBM is Designed to Provide Educators With…
The same kind of evaluation technology as other professions!
Powerful measures that are:
Simple
Accurate
Efficient indicators of student achievement that guide/inform a variety of decisions
Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6

8. General Outcome Measures in Spelling
Not a measure of performance on a particular lesson or unit
A broader indicator
Level of general spelling skills relative to peers
Degree of progress in spelling relative to self or peers

9. Things to Remember About General Outcome Measures in Spelling
“Signs” of general achievement
Not everything
Important things
Standardized tests
Administration
Scoring
Interpretation
Researched
Reliability and validity
Accuracy of measurement
Sensitive to improvement in short periods of time
Designed to be as short as possible (do-ability)
Linked to decision making for promoting positive achievement

10. Spelling CBM Traditional spelling tests versus S-CBM
Shorter duration (2 minutes)
Spelling words dictated carefully at set pace
Random sample from a large pool of graded spelling words
Combination of words
Words students have already learned
Words they will learn to spell in upcoming weeks or months
Exact and sensitive way to assess spelling improvement
Correct letter sequences
Correct word spellings

12. Summary of Spelling Conditions
Timing
2 minutes
New word dictated every 7 seconds (every 10 seconds: grades 1 & 2)
Arrangements
Group
Individual
What is scored
Number of letters spelled in correct sequence
Number of words spelled correctly

13. Standard Directions
Students have lined paper numbered 1–12 (Grades 1 & 2) or 1–17 (Grade 3-8)and a pencil.
Say, “We’re going to take a 2-minute spelling test. I am going to say some words that I want you to spell on the sheet of paper in front of you. Write the first word on the first line, the second word on the second line and so on. I’ll give you [10] [7] seconds to spell each word. When I say the next word, write it down, even if you haven’t finished the last one. You will receive credit for each correct letter written. Are there any questions? [Pause] Let’s begin.”
Say the first word and start your stopwatch (or tape recorder)

14. Standard Directions
Say each word twice. Use homonyms in a sentence (provided)
Say a new word every 10 or 7 seconds
Say, “You should be on the fifth word which is…”
Monitor students to ensure they are writing on the correct line
Do not respond to student questions during the test
After two minutes, say, “Stop. Put your pencils down.”
Good to begin with short practice test to reduce novelty.
Shortened directions available for well-experienced students.

15. Standard Assessment Spelling Lists
Benchmark: 3 standards lists for each grade (1–8)
Screen and identify students who are at risk
Monitor progress and improvement in fall, winter, and spring
Guide program evaluation decisions & improve accountability
Progress Monitor: 30 standard lists for each grade
Provide practical way to write individualized progress goals
Monitor progress and determine rate of improvements
Provide accountability data

16. An Example

17. Administration and Scoring of Spelling-CBM
What Examiners Need To Do…
Before testing students
While testing students
After testing students

18. BEFORE Testing Student needs:
Numbered, lined paper with the exact number of lines (12 or 17)
Saves time
Reduces potential confusion
Makes scoring easier
Answer sheets available as PDF documents
Pencil

19. BEFORE Testing Examiner needs
List of words with time interval clearly marked
Includes sentences for homonyms
Improves ease of scoring
Timing device

20. Gather Assessment Materials
Download and review S-CBM Manual
Download and reproduce student answer sheets
Download Standard Spelling Assessment Lists
Curriculum independent
Allow decision making
Minimize between-teacher, between-school differences
Stopwatch or timer
Clock on the wall is inaccurate, inefficient
Timing tape
Audio tape with recorded tones at prescribed intervals
Allows better attention to words and students
List of students to assess

21. Arrange Testing Environment
Quiet, without distractions
May be 1:1, small group, whole classroom
Consider a simple practice test
Use very simple words
Allows students to adjust to pace
Show student how they get credit for unfinished words
Be sure you can monitor
Especially those students who “get lost” easily
Be available to point to the number of the next word

22. Things to Keep in Mind Keep examiner out of the process
Keep number of words said to minimum
Some students try to spell everything said
Only homonyms have sentences (say word, say sentence, repeat word)
Roam and project
Loud enough for all students to hear
Be sure head is turned toward students
Monitor, monitor, monitor
Alert for lost students
Before they become frustrated and quit
Before they call out that they are lost
Available to point to the next number

23. Things to Keep in Mind Say the number of the word periodically
Younger students, say, “You should be on the fifth (tenth) word which is…”
Older students find this intrusive, so don’t say the numbers
Avoid answering questions during testing
Students may comment: You’re going too fast, I’m lost, What number are we on?
Deal with this in practice tests
Point to orient student
Adhere to end of timing
Interruptions
Re-administer later with another spelling list
Consider that some students need small group or 1:1

24. Helpful Hints Ensure a quiet and distraction-free testing environment
Students must be able to hear the examiner
Offer no assistance, even when you know the child is capable of more
Complete reliability checks
Before and after data collection with all examiners to ensure reliable administration and scoring
Additional reliability checks at least once a year
Prevent examiner “drift”
Use same Benchmark examiner for the same students.
Younger children tend to perform better with familiar examiners.
• Make sure the testing environment is quiet and free from distractions. Students must be able to hear the
examiner during the Phonemic Segmentation Fluency task.
• Complete reliability checks before data collection with all examiners to ensure reliable administration.
Additional reliability checks should be done at least once/year for all examiners, even those familiar with
testing.
• If possible, try to have the same examiner during each Benchmark period test the same students. Younger
children tend to perform better with familiar examiners. Periodic reliability checks will help with examiner
“drift,” or a tendency to provide additional assistance when you know a child is capable of completing
more of each task.

25. Things You Need DURING Testing
Follow the standardized directions:
Spelling-CBM is a standardized test
Administer the assessment with consistency
Remember it’s about testing, not teaching
Don’t teach or correct
Never practice using the standard spelling lists

26. Things to Do AFTER Testing
Score Immediately!
Determine the words spelled correctly
Circle the number beside the words spelled correctly
Award full points
Determine the correct letter sequences for misspelled words (see next slide)
More likely to show change
Provides diagnostic and positive feedback

27. c a t 1 2 3 4 sequences correct Spelling Scoring
Correct: Count each sequence of 2 letters
Spaces before and after the word count as “letters”
Consider: break through versus breakthrough
Example: cat
c a t
sequences correct

28. fisshing _ f i s s h i n g _ (8) cat _ c a t _ (4)
Spelling Scoring
store _ s t o r e _ (6)
stor _ s t o r e _ (6)
fishing _ f i s h i n g _ (8)
fisshing _ f i s s h i n g _ (8)
cat _ c a t _ (4)
catz _ c a t z _ (4)

29. store _^s^t^o^r^e^_ 6 (6) stor _^s^t^o^r e _ 4 (6)
Spelling Scoring
store _^s^t^o^r^e^_ 6 (6)
stor _^s^t^o^r e _ (6)
fishing _^f^i^s^h^i^n^g^_ 8 (8)
fisshing _^f^i^s s h^i^n^g^_ 7 (8)
cat _^c^a^t^_ (4)
catz _^c^a^t z _ (4)

30. Spelling Scoring, Continued
Tricks:
Use master sheet with all correct sequences pre-counted.
Then you only need to count the scoring for the few incorrect sequences (errors) on each paper.
A complete list of spelling rules can be found in the Appendix of your workbook.
Please review and become more familiar with the more unusual errors.

31. Summary
You now have the building blocks to begin Spelling - CBM to ensure literacy growth.
Practice to Automaticity — You’ll get more efficient
Get Checked Out by a colleague using the Accuracy of Implementation Rating Scale (AIRS) for accuracy and efficiency
Stay in Tune by periodically checking AIRS

32. Let’s Practice

33. Practice: Spelling
Open your workbooks to Spelling Practice List 1: Elida
Score this spelling test
Words spelled correctly
Correct letter sequences

34. Answer: Elida WSC = 13 CLS = 121

35. If the Student Misses a Few Words…
Any missing single letter equals 2 missing CLS.
Any 2 letters together that are missing equals 3 missing CLS
Any incorrectly inserted letter equals 1 missing CLS
NEXT — Score Practice List 2: Lester

36. Answer: Lester WSC = 0 CLS = 52
NEXT — Score Practice List 3: Calvin

37. Answer: Calvin WSC = 9 CLS = 109

38. Determining Inter-Scorer Agreement
Agreements / (Agreements + Disagreements) x 100
Example:
Examiner 1 = 90 CLS
Examiner 2 = 92 CLS
Inter-Scorer Agreement = 98%
NEXT — Score Practice List 4: Lucinda and Compute Inter-Scorer Agreement with a partner

39. Answer: Lucinda WSC = 2 CLS = 70
How is your Inter-Scorer Agreement?

40. The End