1. Curriculum Based Measurement Writing Progress Monitoring
Introducations
Presenters
Kathleen Begeny-Johnson & Jerry Purles
June 12, 2013

2. What is your Professional Role?
General Education Teacher
Special Education Teacher
School/District Administrators
Coaches/Consultants/Coordinators
Get to know the audience

3. Objectives
Learn how to monitor a student’s writing performance to indicate what the teacher needs to specifically instruct each student
Administer, score, and graph data from Curriculum Based Measurement – Writing (CBM-W)
Use CBM-W for data-based instructional decision making

4. What Is the Difference Between Traditional Assessments and Progress Monitoring (PM)?
Curriculum-Based Measurement (CBM) is one type of PM:
CBM provides an easy and quick method to gathering student progress
Teachers can analyze student scores and adjust student goals and instructional programs
Student data can be compared to teacher’s classroom or school district data
One type of progress monitoring, CBM, is an alternative to commercially prepared traditional assessments that are administered at one point in time. CBM provides teachers with an easy and quick method of obtaining empirical information on the progress of their students. With frequently obtained student data, teachers can analyze student scores to adjust student goals and revise their instructional programs. That way, instruction can be tailored to best fit the needs of each student.
One problem with traditional assessments is that student scores are based on national scores and averages. In fact, the students in a teacher’s classroom may differ tremendously from a national sample of students. CBM allows teachers to compare an individual student’s data to data on other students in their classroom. Schools or school districts may also collect normative data on the students within their own school or district to provide teachers with a local normative framework for interpreting scores. 4

5. Curriculum-Based Measurement (CBM) Overview
CBM was designed to be a set of simple, efficient standard procedures that
Are objective
Allow for comparison of students to peers and to grade-level benchmarks
Allow for repeated measurement
Show student growth
Serve as indicators of student performance and progress in academic areas

6. Scoring Categories Category Notation Definition
Correct Letter Sequence
CLS
Total Words Written
TWW
A count of the number of words written. A word is defined as any letter or group of letters separated by a space, even if the word is misspelled or is a nonsense word.
Words Spelled Correctly
WSC
A count of the number of words that are spelled correctly. A word is spelled correctly if it can stand alone as a word in the English Language.
Correct Writing Sequence
CWS
A count of correct writing sequences found in the sample. A correct writing sequence is defined as two adjacent writing units (i.e., word-word or word-punctuation) that are acceptable within the context of what is written. Correct writing sequence takes into account correct spelling, grammar, punctuation, capitalization, syntax, and semantics.
Correct Minus Incorrect Word Sequence
CIWS
Number of incorrect word sequences subtracted from number of correct sequence

7. CBM –W

8. University of Minnesota Research Institute for Problem Solving Finding Grade 1 (2009)
Measures
Scoring
Sentence Copying (3-5 min)
TWW, WSC, CWS, CIWS, CLS
Story Prompt (5 min)
TWW, WSC, CWS, CLS
Picture Word (3-5 min)
Photo Prompt (5 min)
CWS, CLS
Recent Research findings on Grade 1 from the University of Minnesota (RIPS)

9. Sample Narrative Prompt
TWW = 36

10. Sample Expository Prompts
Describe a game you like to play and tell why you like it.
Describe your favorite day of the week and tell why you like it.
Think of the most valuable think you own that was not bought in a store. Why is it important to you?
Think of something you made with your own hands. How did you do it? What would you differently next time?
A sentence that includes a “tell why” component (expository)

12. Sources for Prompts Research Institute on Progress Monitoring
Write Source
NWREL

13. How to Administer CBM - W

14. Select Materials Select type of measures based on
Grade level
Purpose of progress monitoring
What will provide you with the most meaningful information
Determine the number of prompts needed
How many weeks do you plan to monitor progress
How often do you plan to monitor progress
Create
That tap students background knowledge
The end of a mid-sentence (for narrative/story starters)
A sentence that includes a “tell why” component (expository)
TIP
Keep materials consistent throughout the monitoring period

15. Materials Needed for Written Expression CBM
CBM Writing Manual with story starters and scoring criteria
Story starter appropriate for the students grade level
Lined paper for student responses
Stop watch
Data recording sheet (optional)
Graph paper or computer graphing program
Written Expression CBM consists of presenting students with a story starter and then allowing students to write for a set amount of time. Written expression CBM is administered to an entire class of students at one time. The teacher scores individual Written Expression CBM samples after administration is complete. 15

16. General Finding within and Across Grades
Grade Level
Type of Prompt
Duration
Scoring Procedures
1-2
Sentence Copying
3-5 min
ww, wsc, cws
Picture-word
Narrative
3-4
Cws, CIWS
5-6
Cws, CIWS,
Expository
5 min
7-9
5-10 min
Cws, CIWS, %CWS
10-11
7-10 min

17. Narrative Prompts (story starters)
Selection of Prompts
Narrative Prompts (story starters)
Primary Narrative Prompts Grade 1-3
Intermediate Narrative Prompts Grade 4-6
(Included in the scoring manual)
The story starters should represent the experiences and background knowledge of the students
The prompts for Written Expression CBM should be simple in sentence structure and should tap age-appropriate background knowledge of the students being monitored. The prompts should also represent the experiences of a wide range of school-age students. For example, English-language learners may not know about certain holidays, students in Utah may not relate to the ocean, and not all students relate to sports. 17

18. Administering CBM Writing
Say these specific directions to the students:
“You are going to write a story. First, I will read a sentence, and then you will write a story about what happens next. You will have 1 minute to think about what you will write, and 3 minutes to write your story. Remember to do your best work. If you don't know how to spell a word, you should guess. “Are there any questions?” (Pause). “Put your pencils down and listen.”
“For the next minute, think about ... (insert story starter).” 18

19. Administering CBM Writing
After reading the story starter, begin your stop watch and allow 1 minute for students to “think.” (Monitor students so that they do not begin writing).
After 30 more seconds say: “You should be thinking about (insert story starter)” Let 30 more seconds pass . . .
At the end of the 1 minute say: “Now begin writing.” Restart your stopwatch (set for 3 minutes).
Monitor students' participation. If individual students pause for about 10 seconds or say they are done before the test is finished, move close to them and say “Keep writing the best story you can.” This prompt can be repeated to students should they pause again. 19

20. Administering CBM Writing
After 90 seconds say: “You should be writing about… (insert story starter).”
At the end of 3 minutes say: “Stop. Put your pencils down.” If students want to finish their story, they may do so on a separate piece of paper or continue on the same sheet after a slash mark has been inserted at the end of 3 minutes. 20

21. Administering CBM Writing
Testing Considerations
Testing not teaching
Best vs. fastest
Monitor student performance
Do not answer questions

22. Scoring Total Words Written (TWW)
What Is A Word?
Any letter or group of letters separated by a space is defined as a word, even if the word is misspelled or is a nonsense word.
The total number of words written are counted regardless of spelling or context.
Correct spelling, word usage, capitalization, and punctuation are ignored when calculating the number of words written.
Underline each word written when scoring
Students can score their own TWW

23. Example Scoring Total Words Written (TWW)
Sentence Total Words Written
The sky was blue TWW = 4
The sky was blew TWW = 4
I tuk a baf TWW = 4
I tuka baf TWW = 3
Iv graqz zznip TWW = 3
Remember - Any letter or group of letters separated by a space is defined as a word, even if the word is misspelled or is a nonsense word.
In your manual there are specific scoring procedures to follow

24. ______ ___ ____ __________ ___ ______
____ ___ ___ _____ ___ ____ ____ __
________ ______ ____ __ ____ ___ ______
__________ ____ ___ ____ ___ _______
___ ______ ____ _____ ___ _____ __ __
______ ___ ________ ______ ___ _ ______
TWW = 42

25. Scoring Words Spelled Correctly (WSC)
Number of correctly spelled words regardless of context
A word is counted correct if it can be found in the English language
Incorrectly spelled words are circled
WSC is calculated by subtracting the total number of circled words from the TWW
Teachers score the WSC

26. Scoring Words Spelled Correctly (WSC)
What is a correctly spelled word?
A word is spelled correctly if it can stand alone as a common word in the English Language regardless of context
Sentence WSC
Bill will reed the book. WSC = 5
And can not pake. WSC = 3
Daz ran down the road. WSC = 4

27. Found in English Language, regardless of context
Scoring WSC
Found in English Language, regardless of context
______ ___ ____ __________ ___ ______
____ ___ ___ _____ ___ ____ ____ __
________ ______ ____ __ ____ ___ ______
__________ ____ ___ ____ ___ _______
___ ______ ____ _____ ___ _____ __ __
______ ___ ________ ______ ___ _ ______
TWW = 42
WSC = 39

28. How to Score CWS EX: Nobody could see the trees of the forest .
Correct Word Sequences (CWS):
CWS is any two adjacent, correctly spelled words acceptable within the context of the sample to a native English speaker.
The teacher considers the units of writing and their relations to one another
The two words must be syntactically and semantically correct
Acceptable word sequence: “the car”
Unacceptable word sequence: “car eyebrow” ^ ^ ^
EX: Nobody could see the trees of the forest . ^ ^ ^
The third method for scoring Written Expression CBM is to calculate the number of correct word sequences (CWS). Before scoring a sample using this method, the teacher must read the entire sample. ^ ^ 28

29. How to Score CWS Correct Word Sequences (CWS):
Carat is placed ABOVE two words if it represents a CWS
“the ^ car”
Correct carats placed between:
Between any two correct word sequence
Between a word and the line at the beginning of a sentence
Between a word and the correct punctuation at the end of a sentence
A carat method is used during scoring. A carat is placed above two words if it represents as correct word sequence. Generally, correct carats are marked with a blue pencil.
When placing carats in a Written Expression CBM sample, correct carats are placed between any two correct word sequence, between a word and line at the beginning of a sentence, and between a word and the punctuation at the end of a sentence. 29

30. How to Score CWS Correct Word Sequences (CWS):
Carat is placed BELOW two words if it represents an incorrect word sequence
“car V eyebrow”
Incorrect carats placed between:
Any two incorrect word sequence
Between a misspelled circled word and non-circled word
Between an un-capitalized word and line at beginning of a sentence
Between an incorrect word and punctuation at end of a sentence
A carat is placed below the words if it represents an incorrect word sequence. Generally, incorrect carats are marked with a red pencil. 30

31. How to Score CWS
Correct Word Sequences (CWS):
Teacher must read entire sample before scoring
Vertical line placed where a sentence should end
^ Mary ^ asked ^ if ^ I ^ would ^ come ^ over ^. ^ I ^ said ^ no ^. CWS = 12
Judgment calls may have to be made about where sentences end: Make decision rules and stick to them when scoring.
Misspelled words are circled
When scoring, a vertical line is placed where a sentence should end. (Judgment calls may have to be made about where a sentence should end. Teachers should establish decision rules and apply those rules to all students’ samples). Misspelled words are circled. 31

32. How to Score CWS
^ It ^ was ^ dark ^ . ^ Nobody ^ could  seen  the ^ trees ^ of ^ the  forrast  .
Because the end mark is considered essential punctuation, a carat is placed between the word and the end mark to make a correct writing sequences.
Since the first word is correct, it is marked as a correct writing sequence
Misspelled words are not counted.
Grammatical or syntactical errors are not counted. A down carat is place between a incorrect writing sequences. (Note: incorrect writing sequence within the same sentence).
CWS = 10

33. CWS Scoring Practice Steps to Score CBM
Read entire sample before scoring, place vertical lines where sentence ends and circle misspelled words
2. CWS- Two adjoined words are:
Spelled correctly
Semantically correct
Syntactically correct
Correctly punctuation
One day, we were playing outside the school and … I Shrunk a Person olmost Steped on me But I Ran to fast ten David nodest me. I seid Can You help me with Everthing he sied Yes! Ov course I sead Ya!
Let’s practice scoring CWS. Refer teachers to practice sheet. Please note: Bjorn is a Nordic male given name
Score the following CBM probe by calculating the number of correct word sequences (CWS).
(Allow participants 5 minutes to score probe.) 33

34. How to Administer and Score Written Expression CBMv
One day, we were playing outside the school and … ˆ I
Shrunk Ú a
Person
olmost
Steped on me
But
Ran to
fast
ten
David
nodest .
seid
Can
You
help
with
Everthing he
sied
Yes ! Ov
course
sead Ya    
The correct writing sequence should be “too fast”
Are there any questions? (Allow time for questions.)  
CWS = 7 34

35. CIWS Scoring Procedure
Correct Minus Incorrect Word Sequence (CIWS) The number of incorrect word sequences subtracted from the number of correct sequences.
^A ^cat  chasd  the ^mouse^ into ^ the  hoose .
CWS = 5
IWS = 4
CIWS = 1

36. ESTIMATED TARGETS BASED ON SELECTED SCORES FROM MULTIPLE RESOURCES (Best Practices in School Psychology V; The ABCs of CBM) These scores represent averages of the selected scores. Use with caution. As we collect data, we will develop local norms
Grade
Time of year
TWW
WSC
CWS 1
Fall
7-8 5
Spring
14-20 10 2
12-24 20
25-30 27 3
23-36 32 14
34-36 33 26 4
33-41 38 23
41-46 44 37
37-51 48 34
42-57 55 41 6
41-47 42
53-58 56 51 7
48-51 49 53 58 64 8
59-74 70
58-67 66 67

37. Example of a Baseline Administer 3 separate CBM writing probes
(Median Data-point)
Administer 3 separate CBM writing probes
Plot 3 baseline data points on the graph
Identifying the median score
If given 3 baseline date points, first rank the values to find the median.
The values are 15, 22, 10. When ranked ordered the values are 10, 15, 22. In this example the first value is the median score.
The values are 15, 22, 10

38. How to Set and Graph Goals
Once baseline data have been collected (best practice is to administer three probes and use the median score), the teacher decides on an end-of-year performance goal for each student.
Three options for making performance goals:
Published norms (AIMSweb
Intra-individual framework
*Tip*
Set reasonable/attainable yet ambitious goals
Step 6 in the process of monitoring student progress with written expression CBM is to set ambitious goals using graphed data.
Once a few CBM scores have been graphed, the teacher should decide on an end-of-year performance goal for the student. There are three options for determining these goals. Two options, end-of-year benchmarking and national norms, are utilized after at least three CBM scores have been graphed. The third option, the intra-individual framework, is utilized after at least 8 CBM scores have been graphed.

39. Setting Goal Student baseline performance Desired rate of progress
Example: 24 CWS (median of 25, 24, 23)
Desired rate of progress
Example: 1.5 CWS per week
Amount of the time the student will be monitored
Example: 35 week
Take the growth per week (1.5) X the number of weeks (35) = 1.5 X 35 = 52.5
Add total to baseline (24 CWS)
24 CWS = 76.5 goal in 35 weeks

40. Example of a Graphed Goal
Performance Goal
Baseline
(Median Data-point)
If given 3 baseline date points, first rank the values to find the median.
The values are 15, 22, 10. When ranked ordered the values are 10, 15, 22. In this example the first value is the median score.
Goal Iine is something to compare data against

41. Using an Intra-Individual Framework to Set Goals
Weekly rate of improvement is calculated using at least eight data points.
Subtract the lowest from the highest score
EX. 12, 16, 15, 19, 16, 21, 26, 24 (26-12=14)
Divide the difference by the number of weeks the data was collected 14 ÷ 8 = 1.75
This baseline rate of growth is multiplied by 1.5 (1.75 x 1.5= 2.625)
This number is multiplied by the number of weeks left until the end of the year x16 weeks = 42
This number is then added to the median score of the first eight data points used to calculate the baseline growth rate
12, 15, 16, 16, 19, 21, 24, 26 = 17.5 median score
= 59.5
This is our end of year performance goal ( 59.5 round up to 60)

42. Example: Using an Intra-Individual Framework to Set Goals
First eight scores: 3, 2, 5, 6, 5, 5, 7, 4.
Difference between the lowest and highest score = 5
Divide difference by number of weeks of data points 5 ÷ 8 = .625
Multiply by 1.5 (1.5 x = )
Multiply by weeks left: × 14 =
Product is added to the median: =
The end-of-year performance goal is 18.

43. Summary
CBM-W is designed to be a simple, efficient approach to monitoring student progress in writing
CBM-W provides useful information about whether instruction is benefiting student or whether a change is necessary

44. References
Assessing Writing Using Curriculum Based Measurement Webinar presented by Dr. Erica Lembke & Dr. Kristen McMaster 2013.
Using CBM for Progress Monitoring in Written Expression and Spelling by Todd Busch, Tracey Hall, & Erica Lembke. 2007