1. [Plan for 60-90 minutes. Fantastic presentation for explaining how to read charts.]

2. Overview
Overview, history, and purpose of AIMSweb and Curriculum-Based Measurement
AIMSweb and RTI
Administration, Scoring, and Interpretation
Oral Reading Fluency
MAZE
Materials
Data System

3. Overview of Curriculum-Based Measurement (CBM) and AIMSweb®
Mark R. Shinn, Ph.D.
Michelle M. Shinn, Ph.D.
Lisa A. Langell, M.A., S.Psy.S.

4. Evaluations to Inform Teaching— Summative & Formative Assessment
Summative Assessment: Culmination measure. Mastery assessment. Assessment after instruction. Pass/fail type assessments which summarize the knowledge students learn. Typical summative assessments include:
End of chapter tests
High-stakes tests (e.g., State assessments)
GRE, ACT, SAT, GMAT, etc. tests
Driver’s license test
Final Exams.
Formative Assessment: Process of assessing student achievement frequently during instruction to determine whether an instructional program is effective for individual students.
Informs:
When students are progressing, continue using your instructional programs.
When tests show that students are not progressing, you can change
your instructional programs in meaningful ways.

5. Summative & Formative Assessment
Summative Assessment: Characterized as assessment of learning.
Formative Assessment: Characterized as assessment for learning. (Citation:
Summative assessment tells you what happened Formative assessment tells you what’s happening.

6. Evaluations to Inform Teaching— Diagnostic Assessment
Diagnostic Assessments: Measures that indicate specific skill strengths and those areas needing improvement. Results may indicate skill areas needing intervention/instruction. Programming may then address students’ needs. Examples:
Criterion-referenced assessments
Cognitive assessments
Rating scales
Norm-referenced, standardized assessments
Tests may be based on the assessment of cognitive skills, academic skills, behavior, health, social-emotional wellbeing, etc.

7. So, Where Does Curriculum-Based Measurement (CBM) fit?
Summative?
Formative?
Diagnostic?

8. What is Curriculum-Based Measurement (CBM)?
CBM is a form of Curriculum-Based Assessment (CBA).
CBM is the method of monitoring student progress through direct, continuous assessment of basic skills.
CBM is used to assess skills such reading, spelling, mathematics, and written language.
CBM probes require about 1 to 4 minutes to complete, depending on the skill being measured.
Student performance is scored for speed and accuracy to determine proficiency.
Because CBM probes are quick to administer and simple to score, they can be given frequently to provide continuous progress data.
The results are charted to provide for timely evaluation based on hard data.

9. Origins of CBM
CBM was initially developed more than 20 years ago by Stanley Deno and others at the University of Minnesota Institute for Research on Learning Disabilities to develop a reliable and valid measurement system for evaluating basic skills growth
CBM is supported by 30 years of school-based research
CBM is endorsed by the United States Department of Education as a method for assessing student progress.
Starting in the area of reading, researchers have expanded to investigate additional academic areas over the years. This includes in-depth research and ultimately the publication of additional measures in literacy, mathematics, and written language
Supporting documentation can be found in 100’s of articles, chapters, and books available within the professional literature describing the use of CBM to make a variety of important educational decisions

10. Advantages of CBM
Direct measure of student performance Deno, S.L. (1985). Curriculum-based measurement: the emerging alternative. Exceptional Children. 52(3):
Correlates strongly with “best practices” for instruction and assessment
Correlates strongly with research-supported methods for assessment and intervention
Focus is on repeated measures of performance (This cannot be done with most norm-referenced and standardized tests due to practice effect or limited forms.)

11. Common Characteristics of General Outcome Measures
CBM involves the same kind of evaluation technology as other professions.
Powerful measures that are:
Simple
Accurate
Efficient indicators of performance that guide and inform a variety of decisions
Generalizable thermometer that allows for reliable, valid, cross comparisons of data

12. General Outcome Measures (GOMs) from Other Fields
Medicine measures height, weight, temperature, and/or blood pressure.
Department of Labor measures the Consumer Price Index.
Wall Street measures the Dow-Jones Industrial Average.
Companies report earnings per share.
McDonald’s® measures how many hamburgers they sell.

13. CBM is Used for Scientific Reasons Based on Evidence
Reliable and valid indicator of student achievement
Simple, efficient, and of short duration to facilitate frequent administration by teachers
Provides assessment information that helps teachers plan better instruction
Sensitive to improvement of students’ achievement over time
Easily understood by teachers and parents
Improves achievement when used to monitor progress

14. Three-Tiered Assessment Model
Tier 3: PROGRESS MONITOR Intensive monitoring towards specific goals for students at significant risk for failure 1 2 3
Tier 2: STRATEGIC MONITOR Monthly monitoring for students who are mild to moderate risk for failure
Tier 1:
BENCHMARK Universal Screening for all students

15. Measures Currently Available via AIMSweb®:
Early Literacy [K-1 benchmark, Progress Monitor (PM) any age]
Letter Naming Fluency
Letter sound fluency
Phonemic Segmentation Fluency
Nonsense Word Fluency
Early Numeracy (K-1 benchmark, PM any age)
Oral Counting
Number identification
Quantity discrimination
Missing number
Oral Reading (K-8, PM any age)
MAZE (Reading comprehension); (1-8, PM any age)
Math Computation (1-6, PM any age)
Math Facts (PM any age)
Spelling (1-8, PM any age)
Written Expression (1-8, PM any age)
All students in an academic curriculum are “benchmarked” three times per year across any/all of these assessment areas.

16. Mark R. Shinn, Ph.D. Michelle M. Shinn, Ph.D.
Administration and Scoring of READING-CURRICULUM-BASED MEASUREMENT (R-CBM) for Use in General Outcome Measurement
Mark R. Shinn, Ph.D.
Michelle M. Shinn, Ph.D.

17. Reading - Curriculum-Based Measurement (R-CBM)
Students read aloud for 1 minute from Edformation’s Standard Reading Assessment Passages of meaningful, connected text.
Number of words read correct and number of errors are counted.
Reported as WRC/errors

18. Benefits of Using Edformation’s Standard Reading Assessment Passages
Are written to represent general curriculum or be “curriculum independent.”
Allow decision making about reading growth, regardless of between- school, between-school-district, between-teacher differences in reading curriculum.
Are graded to be of equal difficulty.
Have numerous alternate forms for testing over time without practice effects.

19. Administration and Scoring of R-CBM
What Examiners Need To Do…
Before Testing students
While Testing students
After Testing students

20. Things you Need Before Testing
1. Standard Reading Assessment Passage Student Copy:
No numbers
Between words (exception: 1st grade)
An informative first sentence
Same font style and size
Text without pictures
Obtain from your LAM

21. Things you Need Before Testing
2. Standard Reading Assessment Passage Examiner Copy:
Pre-numbered so they can be scored quickly and immediately.
Obtain from your LAM.
The numbered copy is going to be in front of you and allows you to score quickly and easily.

22. 3. Tier 1 (Benchmark) R-CBM Probes:
AIMSweb Manager provides staff with copies of three grade-level probes (teacher and student copies).
FALL (Sept.): Staff administer three, grade-level probes to each student. Report median score.
WINTER (Jan.): Repeat administration of same three probes to each student. Report median score.
SPRING (May): Repeat administration of same three probes to each student. Report median score.
FALL (Sept): Grade 3
P01: It rained all day.
P02: Billy was sitting.
P03: Mama frog carried. E X A M P L
WINTER (Jan): Grade 3
P01: It rained all day.
P02: Billy was sitting.
P03: Mama frog carried.
SPRING (May): Grade 3
P01: It rained all day.
P02: Billy was sitting.
P03: Mama frog carried.

23. 4. Additional Assessment Aids Needed Before Testing
A list of students to be assessed
Stop Watch (required—digital preferred)
Clipboard
Pencil Transparencies or paper copies of examiner passages
Dry Marker or Pencil
Wipe Cloth (for transparencies only)

24. Setting up Assessment Environment
Assessment environments are flexible and could include…
A set-aside place in the classroom
Reading station in the hall way
Reading stations in the media center, cafeteria, gym, or empty classrooms
Computer lab, weight room, principal’s office, storage closet, or wherever there’s a quiet space

25. Things You Need to do While Testing
Follow the standardized directions:
R-CBM is a standardized test
Administer the assessment with consistency
Remember it’s about testing, not teaching
Don’t teach or correct
Don’t practice reading the passages
Remember best, not fastest reading
Sit across from, not beside student

26. R-CBM Standard Directions for 1 Minute Administration
Place the unnumbered copy in front of the student.
Place the numbered copy in front of you, but shielded so the student cannot see what you record.
Say:
When I say ‘Begin,’ start reading aloud at the top of this page. Read across the page (DEMONSTRATE BY POINTING). Try to read each word. If you come to a word you don’t know, I will tell it to you. Be sure to do your best reading. Are there any questions? (PAUSE)
Say “Begin” and start your stopwatch when the student says the first word. If the student fails to say the first word of the passage after 3 seconds, tell them the word, mark it as incorrect, then start your stopwatch.
Follow along on your copy. Put a slash ( / ) through words read incorrectly.
At the end of 1 minute, place a bracket ( ] ) after the last word and say, “Stop.”
Score and summarize by writing WRC/Errors

27. “Familiar” Shortened Directions
When students are assessed frequently and know the directions.
Say:
When I say ‘Begin,’ start reading aloud at the top of this page.

28. Things to do After Testing
Score immediately!
Determine WRC.
Put a slash (/) through incorrect words.
If doing multiple samples, organize your impressions of qualitative features.

29. What is a Word Read Correctly?
Correctly pronounced words within context.
Self-corrected incorrect words within 3 seconds.

30. What is an Error? Mispronunciation of the word Substitutions Omissions
3-Second pauses or struggles (examiner provides correct word)

31. What is not Incorrect? (Neither a WRC or an Error)
Repetitions
Dialect differences
Insertions (consider them qualitative errors)

32. Example
Juan finished reading after 1 minute at the 145th word, so he read 145 words total.
Juan also made 3 errors.
Therefore his WRC was 142 with 3 errors.
Reported as: 142/3.

33. R-CBM Activity: Tallying Scores & Calculating Median

34. Data: Get the MEDIAN score for student’s 3 passages: CORRECT WORDS
67 / / / 9
1 min min min.
Why use Median vs. Average?
Averages are susceptible to outliers when dealing with small number sets. Median Score is a statistically more reliable number than average for R-CBM.

35. MEDIAN SCORE: 74/8 (Report in AIMSweb®)
Data: Get the MEDIAN score for student’s 3 passages: ERRORS
67 / / / 9
MEDIAN SCORE: 74/8
(Report in AIMSweb®)
NOTE: R-CBM is the only measure for which the median score is calculated.
So the final score we report is 74 words read correctly with 8 errors.
1 min min min.
Why use Median vs. Average?
Averages are susceptible to outliers when dealing with small number sets. Median Score is a statistically more reliable number than average for R-CBM.

36. BENCHMARK DATA ACTIVITY
Go to P. 2 of your trainee manual
Calculate the Median Score for Hannah Berg WRC = Words Read Correct ERRORS = Number of Errors
WRC/Errors
Example: 64/2

37. (All identifying information and scores are fictitious.)

38. (All identifying information and scores are fictitious.)
(All identifying information and scores are fictitious.)

39. Practice Exercise 1: Let’s Score – P. 3 in trainee manual

40. Practice Exercise 1: Let’s Score
This student read 73 WRC/7 Errors

41. Practice Exercise 2: Let’s Score – P. 3 in trainee manual

42. Practice Exercise 4: Let’s Score
This student read 86 WRC/5 Errors

43. Paired Practice – Oral Reading Fluency – P. 4 Trainee Manual
P. 4 in Trainee Booklet
First time: partner #1 answers, partner #2 scores
Second time: partner #2 answers, partner #1 scores

44. Administration and Scoring of READING-MAZE (R-MAZE) for Use in General Outcome Measurement
Power Point Authored by
Jillyan Kennedy
Based on Administration and Scoring of Reading R-MAZE for Use
with AIMSweb Training Workbook By
Michelle M. Shinn, Ph.D.
Mark R. Shinn, Ph.D

45. R-MAZE is Used for Scientific Reasons Based on Evidence
It is a reliable and valid indicator of student achievement.
It is simple, efficient, and of short duration to facilitate frequent administration by teachers.
It provides assessment information that helps teachers plan better instruction.
It is sensitive to the improvement of students’ achievement over time.
It is easily understood by teachers and parents.
Improves achievement when used to monitor progress.
MAZE meets the same criteria for a quality screener that the early literacy and RCBM measures.

46. Curriculum Based Measurement Reading R-MAZE
CBM R-MAZE is designed to provide educators a more complete picture of students’ reading skills, especially when comprehension problems are suspected.
Area
Timing
Test Arrangements
What is Scored?
CBM R-MAZE Reading
3 minutes
Individual, Small Group, or Classroom Group
# of Correct Answers

47. Curriculum Based Measurement Reading R-MAZE (Continued)
R-MAZE is a multiple-choice cloze task that students complete while reading silently.
The students are presented with word passages.
The first sentence is left intact.
After the first sentence, every 7th word is replaced with three word choices inside a parenthesis.
The three choices consist of:
Near Distracter
Exact Match
Far Distracter

48. Sample Grade 4 R-MAZE Passage

49. Examples of R-MAZE

50. Administration and Scoring of CBM R-MAZE
What examiners need to do . . .
Before testing students
While testing students
After testing students

51. Items Students Need Before Testing
What the students need for testing:
CBM R-MAZE practice test
Appropriate CBM R-MAZE passages
Pencils

52. Items Administrators Need Before Testing
What the tester uses for testing:
Stopwatch
Appropriate CBM R-MAZE answer key
Appropriate standardized directions
List of students to be tested.
On your end, make sure you have your timer, standardized directions, and the answer key.

53. Things You Need to do While Testing
Follow the standardized directions:
Attach a cover sheet that includes the practice test so that students do not begin the test right away.
Do a simple practice test with younger students.
Monitor to ensure students are circling answers instead of writing them.
Be prepared to “Prorate” for students who may finish early.
Try to avoid answering student questions.
Adhere to the end of timing.

54. CBM R-MAZE Standard Directions
Pass R-MAZE tasks out to students. Have students write their names on the cover sheet, so they do not start early. Make sure they do not turn the page until you tell them to.
Say this to the student (s):
When I say ‘Begin’ I want you to silently read a story. You will have 3 minutes to read the story and complete the task. Listen carefully to the directions. Some of the words in the story are replaced with a group of 3 words. Your job is to circle the 1 word that makes the most sense in the story. Only 1 word is correct.
Decide if a practice test is needed. Say . . .
Let’s practice one together. Look at your first page. Read the first sentence silently while I read it out loud: ‘The dog, apple, broke, ran after the cat.’ The three choices are apple, broke, ran. ‘The dog apple after the cat.’ That sentence does not make sense. ‘The dog broke after the cat.’ That sentence does not make sense. ‘The dog ran after the cat.’ That sentence does make sense, so circle the word ran. (Make sure the students circle the word ran.)

55. CBM R-MAZE Standard Directions (Continued)
Let’s go to the next sentence. Read it silently while I read it out
loud. ‘The cat ran fast, green, for up the hill. The three choices
are fast, green, for up the hill. Which word is the correct word
for the sentence? (The students answer fast)
Yes, ‘The cat ran fast up the hill’ is correct, so circle the correct word fast. (Make sure students circle fast)
Silently read the next sentence and raise your hand when you think you know the answer. (Make sure students know the correct
word. Read the sentence with the correct answer)
That’s right. ‘The dog barked at the cat’ is correct. Now what do
you do when you choose the correct word? (Students answer ‘Circle it’. Make sure the students understand the task)
That’s correct, you circle it. I think you’re ready to work on a
story on your own.

56. CBM R-MAZE Standard Directions (Continued)
Start the testing by saying . . .
When I say ‘Begin’ turn to the first story and start reading silently. When you come to a group of three words, circle the 1 word that makes the most sense. Work as quickly as you can without making mistakes. If you finish a/ the page/first side, turn the page and keep working until I say ‘Stop’ or you are all done. Do you have any questions?
Then say, ‘Begin.’ Start your stopwatch.
Monitor students to make sure they understand that they are to circle only 1 word.
If a student finished before the time limit, collect the student’s R-MAZE task and record the time on the student’s test booklet.
At the end of 3 minutes say: Stop. Put your pencils down. Please close your booklet.
Collect the R-MAZE tasks.

57. CBM R-MAZE Familiar Directions
After the students have put their names on the cover sheer, start the testing by saying . . .
When I say ‘Begin’ turn to the first story and start reading silently. When you come to a group of three words, circle the 1 word that makes the most sense. Work as quickly as you can without making mistakes. If you finish a/ the page/first side, turn the page and keep working until I say ‘Stop’ or you are all done. Do you have any questions?
Then say, ‘Begin.’ Start your stopwatch.
Monitor students to make sure they understand that they are to circle only 1 word.
If a student finished before the time limit, collect the student’s R-MAZE task and record the time on the student’s test booklet.
At the end of 3 minutes say: Stop. Put your pencils down. Please close your booklet.
Collect the R-MAZE tasks.

58. Things to Do After Testing
Score immediately to ensure accurate results!
Determine the number of words (items) correct.
Use the answer key and put a slash (/) through incorrect words.
After they’re done, slash all incorrect words and count up the number of correct words. That’s their final score.

59. An answer is considered an error if the student:
CBM R-MAZE Scoring
What is correct?
The students circles the word that matches the correct word on the scoring template.
What is incorrect?
An answer is considered an error if the student:
Circles an incorrect word
Omits word selections other than those the student
was unable to complete before the 3 minutes
expired

60. Making Scoring Efficient
Count the total number of items up to the last circled word.
Compare the student answers to the correct answers on the scoring template. Mark a slash [/] through incorrect responses.
Subtract the number of incorrect answers from the total number of items attempted.
Record the total number of correct answers on the cover sheet followed by the total number of errors (e.g., 35/2).

61. Paired Practice : Let’s Score – Trainee Manual P. 5
P. 5 in Trainee Booklet
First time: partner #1 answers, partner #2 gives full directions (P. 18) and administers passage on P. 22
Second time: partner #2 answers, partner #1 gives abbreviated directions (P. 20)

62. Accessing Your Materials
Two options:
- Print materials online
- Pull materials from folder at school and make copies

63. Accessing Materials Online

64. So

67. Accessing the Materials Folder
Your school will have one Benchmark package and one Progress Monitoring Package
Benchmarking Package contains:
Student Materials Folder for each grade level
Contains the materials the student will have in front of them when testing for each grade level (make one copy of each MAZE passage for each student because they will be writing on them)
Teacher Materials Folder for each grade level
Contains scoring booklet for each grade level (make one copy per student in your class)
Contains administration and scoring manual for each measure
Contains MAZE answer key for benchmark passages (Grades 3-8)
Progress Monitoring Package contains:
One copy of each progress monitoring probe and answer key at each grade level

68. Distribution of Materials and Collection of Data
Make a plan of action:
Who will distribute materials (i.e., scoring booklets, student probes) at your school during benchmarking?
Will each teacher print/copy their own materials? Or will one person distribute the materials to all the teachers?
When will your school begin benchmarking (a two-week window recommended as the maximum time for each benchmarking period)?
Where will testing take place?
What logistical barriers to testing exist and how will those be handled?

69. (All identifying information and scores are fictitious.)
Easy Score Entry
Once data are collected, it is easily entered into AIMSweb®’s web-based software.
Simply type in the scores! (pictured at left)
Dozens of reports are then instantly available.
(Estimated time: 3-5 minutes.)
(All identifying information and scores are fictitious.)

70. (All identifying information and scores are fictitious.)
AIMSweb as a Program Evaluation Tool: Benchmark 3x/year for Universal Screening—All Kids
(All identifying information and scores are fictitious.)

71. Michael Martin: A student with Benchmark Data that indicates he is performing significantly behind peers at his school.

72. Fall Benchmark Data for Michael Martin
Martin, Michael: Grade 5
(All identifying information and scores are fictitious.)

73. Box & Whiskers Graphs (box plots): A 3-Step Explanation1 3 2
Average Range (middle 50%)
AIMSweb commonly uses box plots to report data.
AIMSweb’s Box plots are somewhat similar in shape and representation as to a vertical bell curve.
Michael Martin
Above 90th percentile *
Above Average Range
90th percentile *
Target Line *
75th percentile
Median (50th percentile)
25th percentile
Average range of population included in sample. * *
Below Average Range
10th percentile *
Below 10th percentile *
*In relation to user-defined comparison group

74. Grade 5: Michael’s School Grade 5: Michael’s District
Discussion: Consider Michael’s R-CBM performance in relationship to different AIMSweb® Comparison Groups
Grade 5: Michael’s School
Grade 5: Michael’s District
Grade 5: National Aggregate Norms
(All identifying information and scores are fictitious.)

75. Grade 5: Michael’s School Grade 5: Michael’s District
Michael appears to be performing below expectations when compared to all three comparison groups.
Consider modifying instructional program(s)
Consider increasing frequency of assessment to assess efficacy of alternate program(s)
Review data regularly to assess progress
Grade 5: Michael’s School
Grade 5: Michael’s District
Grade 5: National Aggregate Norms
(All identifying information and scores are fictitious.)

77. AIMSweb National Aggregate Norm Table

78. AIMSweb Aggregate Norm Table

79. An Introductory Look at Additional Benchmark Data

80. Individual Report: 3rd Grade Student
QUESTIONS:
What does report suggest about Viviana’s progress?
What does report suggest about the school’s progress for its 3rd grade students?
What if you saw this pattern in only one school’s 3rd grade within your district?
What if you saw this pattern across most or all 3rd grade groups in your district?

81. Data to Know that Things Went Well
QUESTIONS:
What does report suggest about Heather A’s progress?
SPRING: Compared to Grade 3 peers at her school, is Heather performing in the: Well Above Average Range? Above Average Range? Average Range? Below Average Range? Well-Below Average Range?

82. Have Data to Know When Things Need Changing
QUESTIONS:
What are possible reasons why U. Boardman might not be making progress?
What might happen if nothing is done to address U. Boardman’s needs?
Without this type of visual data, collected at each Benchmark period, do you believe U. Boardman’s stagnation would have been quickly noticed otherwise?

83. Data to Know that Changes Made a Difference

84. Tier 2: Strategic Monitor
Strategic Monitor: (1x/month)
Provides option to increase assessment frequency from three times per year to once per month for select students.
Example of Strategic Monitor report containing monthly data collected over a full school year
(All identifying information and scores are fictitious.)

85. Strategic Monitoring: October (Lindsey Hunter)
(All identifying information and scores are fictitious.)

86. TIER 3: Progress Monitor
Intensive monitoring of individualized goals for students at-risk of failure 3

87. (All identifying information and scores are fictitious.)
Progress Monitor (Tier 3): Intensive assessment with adjustable frequency that matches need 3
(All identifying information and scores are fictitious.)

88. (All identifying information and scores are fictitious.)
Benchmark (Tier 1): 3x per year
Strategic Monitoring (Tier 2): 1x per month for select students at risk for educational difficulties Progress Monitor (Tier 3): Intensive assessment with adjustable frequency that matches need
(All identifying information and scores are fictitious.)

89. Compare a School to a Composite

90. Many More Reporting Options Available

91. The End