1. Responsiveness to Instruction (RtI)
The Problem-Solving Model &
Analyzing the Core
North Carolina Department of Public Instruction
2013 1 1

2. 1 2 7 3 6 5 4 Data Step 1 Define the Problem Step 2 Step 7 Develop an
Develop a behavioral
(observable) definition
of problem 2 7
Step 7
Analysis of the
Intervention Plan
make a team decision on the effectiveness of the
intervention
Step 2
Develop an
Assessment Plan
Generate a hypothesis and assessment questions
related to the problem 6 3
Data
Step 6
Implement the Intervention Plan
Provide strategies, materials, and resources: include
progress monitoring
Step 3
Analysis of the Assessment Plan
Determine if problem is correctly defined 5 4
Let’s look at this seven-step cyclical process. Take out your colored handout and let’s talk about each step in this process.
Step 5
Develop an Intervention Plan
Base interventions on best practices and research-proven strategies
Step 4
Generate a Goal Statement
Specific Description of the changes expected in student
behavior 2 2

3. Core Analysis
Universal Screening
Formative
Summative 3

4. Introduction to Assessment
Assessment within RtI is equally as important as the intervention provided
No one intervention works for all students so it must be “tested” for effectiveness
Assessment is part of problem-solving
Assessment allows for problem solving. Assessment is really testing the hypotheses you make during problem-solving. 4 4

5. Introduction to Assessment
Assessment within RtI should be:
Easily understood by teacher, parents, students
Provides early intervention
Solution driven: Not aimed at diagnosing a problem but the assessments conducted should drive solutions!
Provide a road map towards what strategies are effective for students 5

6. Tier IV Tier III Tier II Tier I
Assessment
Tier I
Student Needs 6 6

7. Analyzing the Core Core instruction (your teachers’ “P” & “D”)
should meet the needs of
80% of all your students.
ALL 7

8. Core Analysis
Universal Screening
Formative
Summative 8

9. Universal Screening
Quick, low cost, repeatable examination of grade appropriate and basic skills of all students
Purpose(s):
Assess your Core’s effectiveness
Who needs more intervention/enrichment?
“Temperature check”
Universal Screening
- helps to determine if the core is working for all students
- helps identify students who are struggling 9 9

10. Universal Screening Conducted three times a year: Fall, Winter, Spring
Allows problem-solving of whole school/group/grade level skill gaps
Triangulate school data
Traditionally, all students in an RtI model are screened three times per year- fall, winter and spring. This allows for early identification of students needing more help and also comparison of your school to the your district, state and nation. Also allows you to problem solve the effectiveness of your core for your whole school, grade levels, subgroups. 10 10

11. Why Conduct Universal Screening?
Determine how well your core instructional programs are working for all students
Identify specific skill deficits/strengths of all students
Add to summative assessments (EOG/benchmarks) to give specific enough data
Provide timely data to make decisions 11 11

12. Oral Reading Fluency Oral reading analyzing accuracy and speed
Measures words read correctly per minute
Highly correlated with overall reading achievement .91
Correlation between weight and blood pressure- 0.40
Correlation between glucose level and weight- 0.46
Correlation between SAT and college grades- 0.50
No it does not measure every facet of reading but it quickly measures the important things. 12
(Fuchs, Fuchs, and Hosp, 2001) 12

13. Examples of Summative Assessment:
Mastery Measure/Culmination Measure
Asks the question “Did they learn it?”
Useful for summary information
Evaluates if learning has taken place
Examples of Summative Assessment:
EOG testing, benchmark testing, GRE testing, SAT testing, Driver’s License tests, EOC tests, Unit Tests/Quizzes, Report Card Grades
Summative assessments are traditionally your mastery measures. It asks if they learned what you taught. 13 13

14. Formative Assessment
Collected over time, rather than just at the end of a unit, semester, year
Not a mastery measure
Examples of Formative Assessment:
Curriculum-Based Measurement, Common Assessments, Descriptive Feedback
Formative assessment allows us to answer if the child is learning. Collected during instruction not after the instruction has taken place. Allows us to make small adjustments. Imagine it like a GPS when you are driving. You don’t plug in the address then never listen to the directions again. You listen to the directions, look at the map and make adjustments in your route to insure you reach your destination. 14 14

15. Formative Assessment
Although summative assessments have a place….we need a way to measure performance over time with frequency
Formative assessment is a key to good Instruction!
Progress monitoring is formative assessment. 15 15

16. Analyzing the Core Core instruction (your teachers’ “P” & “D”)
should meet the needs of
80% of all your students.
ALL 16

17. Analyzing Your Core
Are at least 80% of your students proficient in each subgroup?
What is working? Why? How do you know?
What’s not working? Why? How do you know?
Do teachers have needed skills & content knowledge?
Provide 15 minutes for team reflection
Your Core Program includes many components, with the adopted curriculum as a piece of that core. 17 17

18. What is your Data showing?
Or? 18

19. 19

20. 20

21. Analyzing Your Core Program
Is your curriculum aligned with standards and assessment?
Are teachers using research-based strategies?
Is your schedule working? 21

23. Develop an Assessment Plan
Environment
Instruction
Strategies
Pacing
Schedule
Size of Groups
Curriculum
Learner
Materials
Order 23

24. Develop an Assessment Plan
Instruction
Curriculum
Environment
Leaner
97% of all first grade students did not meet expectations in one area of phonemic awareness because the curriculum delivered between fall and winter was lacking content in this area. 24

25. 1 2 7 3 6 5 4 Data Step 1 Define the Problem Step 2 Develop an
Develop a behavioral
(observable) definition
of problem 2 7
Step 2
Develop an
Assessment Plan
Generate a hypothesis and assessment questions
related to the problem
Step 7
Analysis of the
Intervention Plan
make a team decision on the effectiveness of the
intervention 6 3
Data
Step 6
Implement the Intervention Plan
Provide strategies, materials, and resources: include
progress monitoring
Step 3
Analysis of the Assessment Plan
Determine if problem is correctly defined 5 4
Let’s look at this seven-step cyclical process. Take out your colored handout and let’s talk about each step in this process.
Step 5
Develop an Intervention Plan
Base interventions on best practices and research-proven strategies
Step 4
Generate a Goal Statement
Specific Description of the changes expected in student
behavior 25 25

26. Scheduling: Considerations
Create master schedule based on student needs
Do students receive core instruction?
Typically we rely on students to make the connection to core
How do we connect the varying programs/interventions for children? 26

27. Curriculum Based Measurement (CBM)
Core Analysis: Learner
Curriculum Based Measurement (CBM)
Universal Screening
Formative
Summative 27

28. Curriculum-Based Measurement (CBM)
Assessment approach emphasizing repeated direct measurement of student performance
High levels of validity and reliability
Multiple forms at the same grade level allow for comparison across time
Over 25 years of educational research indicating it promotes positive student outcomes
Most common form of assessment used in an RtI model for progress-monitoring.
Citation: 28 28

29. Curriculum-Based Measurement
"any set of measurement procedures that use direct observation and recording of a student’s performance in a local curriculum as a basis for gathering information to make instructional decisions” Deno (1987)
Method of monitoring student educational progress through direct assessment of academic skills 29 29

30. Curriculum-Based Measurement
Characteristics:
Simple Accurate Efficient Generalizable
Reliable and Valid Predictive Sensitive Flexible
CBM is a quick, systematic means of collecting data that can be used to screen, further investigate a strengths/weaknesses, and determine the effectiveness of instructional strategies.
Simple – easy to obtain data (less time, good data)
Accurate – provide specific information
Efficient – tasks only take a few minutes to administer
Generalize – can compare / contrast student performance across school, district, country
Reliable and valid
Predictive – student performance now can predict later performance
Sensitive
Flexible
Go back to assessment list and see which ones have all these characteristics
Imagine you are going to fly somewhere for vacation. You have your choice of many different airlines, fares, schedules, seating options, and routes. The last flight you took on Traditional Air was expensive. In addition, the schedules didn’t accommodate your schedule, the seats were uncomfortable, and the flight wasn’t direct, so it took forever. But now there is an alternative carrier: Air CBM. Air CBM goes directly where you want to go in less time for half the price! In addition, it flies when you want and the seats are adjustable! Which would you choose?
Would you decide to fly both airlines and make the same trip twice for one vacation? (CBM is not an “add on,” its an alternative!)
from the ABCs of CBM – Hosp, Hosp, and Howell
Curriculum-Based Measurement (CBM) was developed more than 20 years ago by Stanley Deno at the University of Minnesota through a federal contract to develop a reliable and valid measurement system for evaluating basic skills growth.
CBM is supported by more than 25 years of school-based research by the US Department of Education.
Supporting documentation can be found in 100s of articles, book chapters, and books in the professional literature describing the use of CBM to make a variety of important educational decisions.
CBM is a form of classroom assessment for ...
describing academic competence in reading, spelling, or mathematics
tracking academic development
improving student achievement 30 30

31. Curriculum Based Measurement
Purposes
Types
Survey Level Assessment
Determine student’s instructional level
Specific Level Assessment
General Outcome Measures
Skill based measures
Mastery measures 31

32. Curriculum Based Measurement: Types
General Outcome Measures
Skills –Based Measures
Mastery Measures
Screening
Diagnostic Evaluation
Survey-level assessment
Specific-level assessment
Progress Monitoring
To target content areas of concern
Survey-level assessment – screening and diagnostic functions
Specific-level assessment – determine what and how the student needs to be taught
Each type of CBM plays a role in an RtI model – just as summative, benchmark, and formative assessment play a role in assessment
CBMs as: General Outcome Measures, Skills-based measures, Mastery measures
General Outcome Measures:
Sample performance across several goals at the same time, using tasks that can only be accomplished by applying a number of contributing skills (think oral reading fluency)
Disadvantage of GOM – they are general (if oral reading fluency is inadequate, will need more specific information about specific skills; GOMs are difficult to develop in math beyond early grades)
Non-educational example – McDonalds tracks how many hamburgers they sell
Skills-based measure:
Designed to accomplish many features of GOMs -
Advantages – can be used to screen, progress monitor, and do survey-level when overarching tasks aren’t available (think math computation)
Disadvantage – at any given time only a few items on the test will be directly related to what the student is currently learning (second grade math computation would include: single digit addition and subtraction, two-digit addition and subtraction, two-digit addition with regrouping)
Non-educational example – Number of hamburgers won’t let McDonalds know the success of the the breakfast menu
Mastery measures:
Contain only discrete and easily identified set of items (punctuation, multiplying fractions, sounds of letters)
Used when need to focus on a particular set of skills, trying to troubleshoot a problem doing specific-level testing, monitor learning when a skill is taught in isolation
Disadvantage: very narrow focus, cannot monitor progress on long-term goals
Non-educational example – Number of hamburgers won’t let McDonalds know if the new fish sandwich is a hit
EACH type of CBM has a role in the RtI model 32 32

33. CBM as a General Outcome Measure
This is a general thermometer of academic health It is complimentary to any curriculum- not curriculum-specific
Examples of GOM measures outside of education
Height
Weight
Blood pressure
Stock Market
McDonald’s measuring number of hamburgers sold
CBM is not tied to any one curriculum. 33 33

34. Curriculum Based Measurement: Examples
Early Literacy
Oral Reading Fluency
Comprehension-MAZE
Multiple Choice Reading Comprehension
Written Expression
Early Numeracy
Math Computation
Math Applications 34

35. Finding Curriculum Based Measurements
Interventional Central allows for creating your own CBM
Needs to meet this criteria:
Reliable
Valid
Norms available
Alternate (equivalent) forms
Sensitive to improvement over time
Standardized administration 35 35

36. Oral Reading Fluency Oral reading analyzing accuracy and speed
Measures words read correctly per minute
Highly correlated with overall reading achievement .91
Correlation between weight and blood pressure- 0.40
Correlation between glucose level and weight- 0.46
Correlation between SAT and college grades- 0.50
No it does not measure every facet of reading but it quickly measures the important things. 36
(Fuchs, Fuchs, and Hosp, 2001) 36

37. Oral Reading Fluency Students read aloud for one minute
Words read correctly per minute are computed
What is correct?
Self-corrects (within 3 seconds), correctly read words, incorrectly read words that are dialectical in nature 37 37

38. Oral Reading Fluency Read connected text accurately and fluently
Passages available from first to ninth grade levels
Oral Reading Fluency, Passage Reading Fluency, Reading CBM
Student reads passage for one minute.
The number of words read correctly is recorded.
Final score is words correct per minute (rate)
Can also obtain an accuracy score (percentage of words read correctly)
Even though ORF only measures a part of reading fluency (speed) it is a good predictor of overall reading skills. We know that as students become more fluent readers, comprehension increases.
When using oral reading fluency in a universal screening model, three passages are administered and the median score is used.
When using oral reading fluency for progress monitoring, only one passage is administered for each progress monitoring point.
Activity – have participants discuss what type of CBM oral reading fluency is (general outcome, skills-based, mastery measure) 38 38

39. Fluency Rubrics Smoothness Pacing Confidence Accuracy Expression
Gives more robust assessment of fluency 39

40. Reading Comprehension: MAZE
Student reads passage silently for 3 minutes
Every 7th word is replaced with three choices
Student circles correct choice
Can be group administered
MAZE, CLOZE, Reading Comprehension
Student reads passage silently for three minutes. When the student comes to a missing word (every 7th word), the student circles the correct word.
Final score is items correct per 3 minutes. Can also obtain an accuracy score (percentage of items correct out of items attempted).
Activity – have participants discuss what type of CBM Maze is (general outcome, skills-based, mastery measure) 40 40

41. Reading Comprehension: Multiple Choice
For screening for grades 6-8
Students can take online or pencil/paper
No set time limit (estimate 45 minutes)- questionable reliability/validity at this point
(Natl. Center on RtI) 41

42. Math Computation Administered for two to eight minutes
Single Skill or Multi Skill
*Timed Tests are not an intervention nor a good instructional tool. Can be used for assessing one type of “fluency”
Math Calculation –
Generally administered for two minute from grades 1 – 3 and four minutes grades 4 and up
Can use probes that assess either single skills (i.e., addition, single digits, solutions to ten) or multiple skills (i.e., addition, subtraction, multiplication)
Different grade levels will take different skills - see chart for suggested skills based on NCSCOS
Math CBM is always scored for digits correct – an answer of 32 is 2 points. This makes the measures more sensitive.
Activity – have participants discuss what type of CBM calculation is (general outcome, skills-based, mastery measure) 42 42

43. Math Applications Concepts and Problem Solving
Measures the application of math concepts
Administration times vary- typically 8-10 minutes
Math Concepts and Problem Solving
- Depending on the measure, administration times will vary. Most measures recommend 8 minutes.
- Can be administered as a group
- Item content will vary based on grade level expectations
Activity – have participants discuss what type of concepts and application is (general outcome, skills-based, mastery measure) 43 43

44. Written Expression Spelling Writing (typically 4 minutes total)
Correct Spelling Sequences
Writing (typically 4 minutes total)
Correct Writing Sequences or Correct Words Written scoring
Spelling:
Although they can be administered individually, CBM spelling probes are typically given to groups of students. The examiner reads aloud a list of 12 to 17 spelling words, reading successive words after a predetermined number of seconds. Students attempt to spell the words on their answer sheets in the time allotted.
Scoring - The scoring of CBM spelling probes is similar to that of other CBM measures in that it is designed to give credit to the student for even partial competencies. Instead of giving credit for words only when all letters are correct, CBM views spelling words as being made up of smaller units called letter-sequences. Correct letter-sequences are pairs of letters in a word that are placed in the proper sequence. Therefore, if a student is able to put at least some letters in the proper sequence, that child will be given partial credit for a word. The CBM method of scoring words is also quite sensitive to short-term student gains in spelling skills.
Correct Writing Sequences:
CBM Writing probes are simple to administer but offer a variety of scoring options. As with math and spelling, writing probes may be given individually or to groups of students. The examiner prepares a lined composition sheet with a story starter sentence or partial sentence at the top. The student thinks for 1 minute about a possible story to be written from the story-starter, then spends 3 minutes writing the story. The examiner collects the writing sample for scoring. Depending on the preferences of the teacher, the writing probe can be scored in several ways.
When scoring correct writing sequences, the examiner goes beyond the confines of the isolated word to consider units of writing and their relation to one another. Using this approach, the examiner starts at the beginning of the writing sample and looks at each successive pair of writing units (writing sequence). Words are considered separate writing units, as are essential marks of punctuation. To receive credit, writing sequences must be correctly spelled and be grammatically correct. The words in each writing sequence must also make sense within the context of the sentence. In effect, the student's writing is judged according to the standards of informal standard American English. A caret (^) is used to mark the presence of a correct writing sequence.
Activity – have participants discuss what type of CBMs each writing measure is (general outcome, skills-based, mastery measure) 44 44

45. Types of Assessment: Early Literacy
Letter Naming
Phoneme Segmentation
Decoding
Letter Sounds
Use CBM examples handout to show letter naming fluency
Letter Naming Fluency:
Student says the names of letters for one minute.
The number of letter names identified correctly is recorded.
Final score is words correct per minute (rate)
Can also obtain an accuracy score (percentage of words read correctly)
LNF measures automaticity of letter naming and is a reliable indicator of future reading success.
Difference between this task and just identifying letter names – measures not only whether the student knows the information but how well the student knows the information. The better the student knows the information, the easier they can access the information to apply in academic settings, such as reading.
Letter Naming is the only measure that isn’t necessary for reading – but it is a very reliable indicator of risk for reading failure.
Phoneme Segmentation: the examiner says a word and the student breaks the word into individual sounds. The score obtained is sounds correct per minute. It measures a student’s ability to hear individual sounds in a word.
Decoding: many curriculum based measurement systems use nonsense words to measure early decoding skills – which is a way to ensure we are measuring decoding and not sight reading.
Activity – have participants discuss what type of CBMs each early literacy measure is (general outcome, skills-based, mastery measure) 45 45

46. Types of Assessment: Early Numeracy
Oral Counting
Number Identification
Quantity Discrimination
Missing Number
Number Identification:
Name single digit and double digit numbers
One minute timed task
Usually administered to students in kindergarten and first grade
Results are numbers correct in one minute
Quantity Discrimination:
Identify greater number
Student can either point or say the name of the number, so it is a good measure for students who aren’t yet fluent in English
Measure of mental number line
Missing Number:
Identify missing number
Student can write the missing number if they aren’t yet fluent in English
Activity – have participants discuss what type of CBMs each early numeracy measure is (general outcome, skills-based, mastery measure) 46 46

47. Types of Assessment: Curriculum Based Measurement
Reflection:
Why would you want to use CBM?
What are the benefits of CBM in the classroom?
What are the benefits of CBM for a school?
Turn and Talk!!! 47 47