1. National Center on Response to Intervention OSPI & National Center on Response to Intervention November 1, 2011 Implementer Series Module 3: Multi-level Prevention System

2. National Center on Response to Intervention Session Agenda  Welcome and Introductions  Review  What Is Multi-level Instruction?  Selecting Evidence-Based Practices  Wrap-Up: Review, Questions, and Resources 2

3. National Center on Response to Intervention Upon Completion Participants Will Be Able To:  Use screening and progress monitoring data to make decisions at all levels of the multi- level prevention system, including movement between levels.  Develop a multi-level prevention system.  Select evidence-based interventions and practices. 3

4. National Center on Response to Intervention REVIEW: SCREENING AND PROGRESS MONITORING 4

5. National Center on Response to Intervention 5  Response to intervention (RTI) integrates assessment and intervention within a schoolwide, multi ‑ level prevention system to maximize student achievement and reduce behavior problems. Defining RTI (NCRTI)

6. National Center on Response to Intervention 6  With RTI, schools identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions based on a student’s responsiveness; and  RTI may be used as part of the determination process for identifying students with specific learning disabilities or other disabilities. Defining RTI (NCRTI)

7. National Center on Response to Intervention RTI as a Preventive Framework  RTI is a multi-level instructional framework aimed at improving outcomes for ALL students.  RTI is preventive and provides immediate support to students who are at risk for poor learning outcomes.  RTI may be a component of a comprehensive evaluation for students with learning disabilities. 7

8. National Center on Response to Intervention Essential Components of RTI  Screening  Progress Monitoring  Schoolwide, Multi-level Prevention System Primary Level Secondary Level Tertiary Level  Data-Based Decision Making for Instruction Evaluating Effectiveness Movement within the multi-level system Disability identification (in accordance with state law) 8

9. National Center on Response to Intervention Essential Components of RTI 9

10. National Center on Response to Intervention Screening  PURPOSE: Identify students who are at risk for poor learning outcomes  FOCUS: ALL students  TOOLS: Brief assessments that are valid, reliable, and demonstrate diagnostic accuracy for predicting learning or behavioral problems  TIMEFRAME: Administered more than one time per year (e.g., fall, winter, spring) 10

11. National Center on Response to Intervention Progress Monitoring  PURPOSE: Monitor students’ response to primary, secondary, or tertiary instruction in order to estimate rates of improvement, identify students who are not demonstrating adequate progress, and compare the efficacy of different forms of instruction  FOCUS: Students identified through screening as at risk for poor learning outcomes  TOOLS: Brief assessments that are valid, reliable, and evidence based  TIMEFRAME: Students are assessed at regular intervals (e.g., weekly, biweekly, or monthly) 11

12. National Center on Response to Intervention Data-Based Decision Making  Analyze data at all levels of RTI implementation (e.g., state, district, school, grade level) and all levels of prevention (e.g., primary, secondary, or tertiary).  Establish routines and procedures for making decisions.  Set explicit decision rules for assessing student progress (e.g., state and district benchmarks, level and/or rate).  Use data to compare and contrast the adequacy of the core curriculum and the effectiveness of different instructional and behavioral strategies. 12

13. National Center on Response to Intervention Review Activity  What is the difference between a mastery measure and a general outcome measure?  T or F: All progress monitoring tools are created equal.  Where can I find evidence of the reliability and the validity of progress monitoring tools?  Name three uses of progress monitoring data.  What is a trend line?  What are three ways to establish progress monitoring goals?  Describe two ways to analyze progress monitoring data. 13

14. National Center on Response to Intervention WHAT IS A MULTI-LEVEL PREVENTION SYSTEM? 14

15. National Center on Response to Intervention Essential Components of RTI 15

16. National Center on Response to Intervention Levels, Tiers, and Interventions FRAMEWORK: 3 levels of intensity Primary Secondary (Targeted/Strategic) Tertiary (Intensive) Secondary Level of Prevention (~15% of students) Tertiary Level of Prevention (~ 5 % of students) Primary Level of Prevention (~80% of students) 16

17. National Center on Response to Intervention Levels, Tiers, and Interventions MODEL: Minimum of 3 tiers representing each level of intensity Tier II (secondary) Tier I (primary) 17 Tier III (tertiary)

18. National Center on Response to Intervention Levels, Tiers, and Interventions Interventions are provided at each level and within each tier. Secondary Level of Prevention Tertiary Level of Prevention Primary Level of Prevention 18

19. National Center on Response to Intervention NCRTI recommends different evidence standards across intervention levels. Research-based curricula Recommended for primary prevention across subjects. Components have been researched and found to be generally effective. Curriculum has not been rigorously evaluated. Evidence-based intervention Recommended for secondary and tertiary prevention Evaluated using rigorous research design Evidence of positive effects for students who received the intervention 19

20. National Center on Response to Intervention PRIMARY PREVENTION LEVEL 20

21. National Center on Response to Intervention Primary Prevention Level  FOCUS: ALL students  INSTRUCTION: District curriculum and instructional practices that are research based; aligned with state or district standards; and incorporate differentiated instruction  SETTING: General education environment  ASSESSMENTS: Screening, continuous progress monitoring, and outcome measures or summative assessments 21

22. National Center on Response to Intervention Primary Prevention Focus  ALL students  Includes students with disabilities, learning differences, or language barriers  Increase access through Differentiated instruction. Practices that are linguistically and culturally responsive. Accommodations. Modifications. 22

23. National Center on Response to Intervention Primary Level Instruction  Research-based curriculum materials for students (including sub groups)  Implementation fidelity  Articulation of teaching and learning within and across grades  Differentiation of instruction based on data  Ongoing professional development See NCRTI Integrity Rubric 23

24. National Center on Response to Intervention What Is Core Curriculum in RTI?  Grade level standards  Usually mandatory for all students of a school or a school system  Often instituted at the elementary and secondary levels by local school boards, departments of education, or other administrative agencies charged with overseeing education  May or may not include instructional materials 24

25. National Center on Response to Intervention What Are Differentiated Learning Activities?  Offers students in the same class different teaching and learning strategies based on Student assessment data and knowledge of student readiness Learning preferences, Language and culture 25

26. National Center on Response to Intervention What Are Differentiated Learning Activities?  Involves Mixed instructional groupings, Team teaching, Peer tutoring, Learning centers, and Accommodations to ensure that all students have access to the instructional program  Is NOT the same as providing more intensive interventions to students with low achievement or learning disabilities 26

27. National Center on Response to Intervention Primary Prevention Setting  General education environment  Various grouping strategies Whole class Cooperative learning groups 27

28. National Center on Response to Intervention Primary Prevention Assessment  Universal screening to determine students’ current level of performance  Continuous progress monitoring to confirm risk status and monitor progress of at-risk students  Outcome measures or summative assessments for accountability 28

29. National Center on Response to Intervention Progress Monitoring and Screening Data Within Primary Prevention  Screening data Identify students who need additional assessment or instruction. Evaluate the effectiveness of primary prevention for all students.  Progress monitoring data Confirm and disconfirm risk. 29

30. National Center on Response to Intervention Screening: Identify Students Who Need Additional Assessment and Instruction 10 60 50 30 40 20 70 Fall Above average Average Below average Student Benchmark Scores for Grade 2 Screening Measure Score 30

31. National Center on Response to Intervention Screening: Evaluate Effectiveness of Primary Prevention 20 120 100 60 80 40 140 FallSpringWinter Target score General population. Title I Special education Score 31

32. National Center on Response to Intervention Progress Monitoring: Confirming Risk Status X X 32

33. National Center on Response to Intervention XX Progress Monitoring: Confirming Risk Status 33

34. National Center on Response to Intervention Team Activity: Primary Prevention  Complete Primary prevention (column 2) in Appendix C “Developing an RTI Model”: Tier(s) Focus Instruction Setting Assessment Data-based decision making Other 34

35. National Center on Response to Intervention SECONDARY PREVENTION LEVEL 35

36. National Center on Response to Intervention Secondary Prevention Level  FOCUS: Students identified through screening as at risk for poor learning outcomes  INSTRUCTION: Targeted, supplemental instruction delivered to small groups  SETTING: General education classroom or other general education location within the school  ASSESSMENTS: Progress monitoring, diagnostic 36

37. National Center on Response to Intervention Secondary Prevention Focus  Students identified through screening as at risk for poor learning outcomes  Typically 15% of entire population 37

38. National Center on Response to Intervention Secondary Level Instruction  Evidence based  Aligns with and supports core instruction  Implementation fidelity based on developer guidelines.  Delivered by well-trained staff in optimal group sizes  Decisions are based on valid and reliable data and criteria are implemented accurately.  Supplements core instruction See NCRTI Integrity Rubric 38

39. National Center on Response to Intervention Secondary Prevention Setting  General education classroom or similar setting  Adult-led instruction  Small group rather than whole class 39

40. National Center on Response to Intervention Secondary Prevention Assessment  Decisions about responsiveness to intervention Are based on reliable and valid progress monitoring data. Reflect judgment based on the slope of improvement or final status at the end of the intervention period.  Decision-making rules are applied accurately and consistently 40

41. National Center on Response to Intervention Secondary Prevention Assessment  Progress monitoring Monitor student response to secondary instruction. Evaluate the efficacy of the secondary system. Conduct at least monthly.  Diagnostic assessment Match students needs to interventions. 41

42. National Center on Response to Intervention Secondary Prevention Goal Setting  End-of-year benchmarking  National norms for weekly rate of improvement (slope) 42

43. National Center on Response to Intervention Progress Monitoring Data Within Secondary Prevention  Progress monitoring data Determine response to secondary interventions using  The four-point rule.  Trend-line analysis. Compare efficacy of secondary interventions. 43

44. National Center on Response to Intervention Progress Monitoring: Determining Response Using the Four-Point Rule 44 Goal line

45. National Center on Response to Intervention Progress Monitoring: Determining Response Using Trend Line Analysis X Goal line Trend line 45

46. National Center on Response to Intervention Growth by Intervention Type Progress Monitoring: Compare Efficacy of Secondary Interventions 46 Words Read Correctly

47. National Center on Response to Intervention Progress Monitoring: Evaluate Efficacy of Secondary System  Data should indicate the following: 70%-80% of students in secondary interventions demonstrating adequate progress Implementation fidelity for interventions and data- based decision rules 47

48. National Center on Response to Intervention Team Activity: Secondary Prevention  Complete secondary prevention (column 3) in Appendix C “Developing an RTI Model”: Tier(s) Focus Instruction Setting Assessment Data-based decision making Other 48

49. National Center on Response to Intervention TERTIARY PREVENTION LEVEL 49

50. National Center on Response to Intervention Tertiary Prevention Level  FOCUS: Students who have not responded to primary or secondary level prevention  INSTRUCTION: Intensive, supplemental instruction delivered to small groups or individually  SETTING: General education classroom or other general education location within the school  ASSESSMENTS: Progress monitoring, diagnostic 50

51. National Center on Response to Intervention Tertiary Prevention Focus  Students who have not responded to primary or secondary level prevention  Typically 3%-5% of the entire population 51

52. National Center on Response to Intervention Tertiary Level Instruction  Evidence based or based on validated progress monitoring methods for individualizing instruction  More intense than secondary  Implementation fidelity  Delivered by well-trained staff in optimal group sizes  Decisions are based on valid and reliable data, and criteria are implemented accurately.  Address general education curriculum in appropriate manner for students. See NCRTI Integrity Rubric 52

53. National Center on Response to Intervention Tertiary Prevention Setting  General education setting or other appropriate setting  Decisions are made on a case-by-case basis.  Optimal group size is chosen for ages and needs of students. 53

54. National Center on Response to Intervention Tertiary Prevention Assessment  Decisions about responsiveness to intervention Are based on reliable and valid progress monitoring data. Reflect judgment based on the slope of improvement or final status at the end of the intervention period.  Decision-making rules are applied accurately. 54

55. National Center on Response to Intervention Tertiary Prevention Assessment  Progress monitoring Frequent progress monitoring (at least bi-weekly) is recommended. Regularly monitor based on established learning trajectories indicated by the goal line.  Diagnostic Match instruction to needs. Inform individualized instructional planning. 55

56. National Center on Response to Intervention Tertiary Prevention: Goal Setting Three goal-setting options:  End-of-year benchmarking  National norms for weekly rate of improvement (slope)  Intra-individual 56

57. National Center on Response to Intervention Progress Monitoring Data Within Tertiary Prevention  Progress monitoring data Determine response to secondary interventions using  The four-point rule.  Trend line analysis.  Trend line analysis and slope. Compare efficacy of tertiary interventions. 57

58. National Center on Response to Intervention Progress Monitoring: Determining Response Using Four-Point Rule 58 Goal line

59. National Center on Response to Intervention Progress Monitoring: Determining Response Using Trend Line Analysis 59 XX Goal line Trend line

60. National Center on Response to Intervention Progress Monitoring: Determining Response Using Trend Line Analysis and Slope 60 Goal line Trend line

61. National Center on Response to Intervention Progress Monitoring: Evaluate Efficacy of Tertiary System  Data should indicate the following: Majority of students in tertiary prevention are demonstrating adequate progress (e.g., eventually move from tertiary to secondary) Implementation fidelity for interventions and data decision rules 61

62. National Center on Response to Intervention Team Activity: Tertiary Prevention  Complete tertiary prevention (column 4) Appendix C “Developing an RTI Model”: Tier(s) Focus Instruction Setting Assessment Data-based decision making Other 62

63. National Center on Response to Intervention Changing the Intensity and Nature of Instruction  Intervention  Duration  Frequency  Interventionist  Group size 63

64. National Center on Response to Intervention IDEA AND MULTI-LEVEL PREVENTION SYSTEM 64

65. National Center on Response to Intervention What About Special Education? Two groups to consider:  Students with disabilities who are currently receiving special education  Students being referred for special education eligibility consideration 65

66. National Center on Response to Intervention What About Students with Disabilities? 66 ~15% ~5% Tier III (tertiary) Specialized individualized systems for students with intensive needs Tier II (secondary) Supplemental group systems for students with at- risk response to primary level Tier I (primary) Schoolwide instruction for all Students, including differentiated instruction ~80%

67. National Center on Response to Intervention Disability Identification To ensure that underachievement in a child suspected of having a specific learning disability is not due to a lack of appropriate instruction in reading or math, the group must consider, as part of the evaluation, what is described in 34 CFR 300.304 through 300.306:  Data demonstrate that prior to, or as a part of, the referral process, the child was provided appropriate instruction in regular education settings, delivered by qualified personnel.  Data-based documentation of repeated assessments of achievement at reasonable intervals reflect formal assessment of student progress during instruction, which was provided to the child’s parents. 67 ( www.idea.ed.gov )

68. National Center on Response to Intervention 68 When Do We Refer to Special Education? ~15% ~5% Tier III (tertiary) Tier II (secondary) Tier I (primary) ~80% of students Example 1: After nonresponsiveness to two secondary interventions

69. National Center on Response to Intervention 69 When Do We Refer to Special Education? ~15% ~5% Tier III (tertiary) Tier II (secondary) Tier I (primary) ~80% of Students Example 2: After nonresponsiveness to one secondary and one tertiary intervention

70. National Center on Response to Intervention Recommendations  Collaborate with special education to develop an inclusive multi-level instruction model  Create written guidance on the following: How do special education services “fit” in the model? What are the roles and responsibilities of special education staff? When should students be referred for eligibility consideration? 70

71. National Center on Response to Intervention Prereferral Model Versus RTI 71

72. National Center on Response to Intervention SELECTING EVIDENCE-BASED PRACTICES 72

73. National Center on Response to Intervention Selecting Evidence Based Practices: Definitions 73  Evidence-based intervention Data demonstrates (or empirically validates) efficacy through scientific, rigorous research designs.  Research-based curricula May incorporate design features that have been researched generally. The curriculum or program as a whole has not been studied using a rigorous research design. See Training Manual Page 9

74. National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. See Training Manual Page 9 74

75. National Center on Response to Intervention 1) Identifying Needs and Priorities 1.Gather a team. 2.Conduct a “needs assessment.” Gather information from multiple sources. Compile and summarize data. 3.Determine priorities. See Training Manual Page 9 and Instructional Intervention Tools Chart User’s Guide 75

76. National Center on Response to Intervention 1) Identifying Needs and Priorities  Evaluate existing data to determine baseline performance on indicators of interest. Academic achievement Discipline referrals Attendance/truancy data 76

77. National Center on Response to Intervention 1) Identifying Needs: Interventions  For what skills do we need a secondary intervention instructional program? Is there a specific academic outcome or measure we are interested in providing supplemental instruction for?  For what grades do we need an instructional program?  Will this program be used with all students who are not progressing in the core curriculum or only with specific sub groups of students? Tier I? Tier II? Tier III?  Which sub groups? English language learners (ELLs)? Special education? 77 See Instructional Intervention Tools Chart User’s Guide

78. National Center on Response to Intervention 1) Identifying Priorities: Interventions  Is it a program that can be purchased for a reasonable cost?  Is it a program with a reasonable implementation time?  Is it a program that does not require specialized expertise or lengthy training to administer?  Is it a program that offers ready access to training and technical support for staff?  Is it a program that has documented evidence of efficacy through the most rigorous research?  Is it a program whose effectiveness has been studied and demonstrated in our district or state? 78 See Instructional Intervention Tools Chart User’s Guide

79. National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 79

80. National Center on Response to Intervention 2) Selecting Evidence-Based Practices  Screening and progress monitoring  Data-based decision making  Instruction/interventions 80

81. National Center on Response to Intervention 2) Selecting Evidence-Based Practices: Screening and Progress Monitoring  NCRTI: Screening Tools Chart  NCRTI: Progress Monitoring Tools Chart  IES practice guide: RTI for mathematics and reading  IRIS Center webinars and RTI/assessment training modules 81

82. National Center on Response to Intervention 2) Selecting Evidence-Based Practices: Data-Based Decision Making  IES Practice Guide: Using Student Achievement Data  U.S. Department of Education: Doing What Works and the American Recovery and Reinvestment Act  IRIS Center Case Study Unit RTI: Data-based Decision Making 82

83. National Center on Response to Intervention 2) Selecting Evidence-Based Practices: Instruction/Intervention  What Works Clearinghouse  Best Evidence Encyclopedia  NCRTI Instructional Intervention Tool Chart  Florida Center for Reading Research  IES Practice Guide: RTI for Mathematics and Reading  IRIS Center Learning Strategies Modules 83

84. National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 84

85. National Center on Response to Intervention 3) Evaluating Evidence  Where can I find evidence?  What type of evidence exists?  What is the quality of the evidence?  What were the desired outcomes?  What are the effects of the intervention?  Is the sample population similar? 85 See Training Manual Page 10

86. National Center on Response to Intervention 3a) Evaluating Evidence: Where?  Curriculum websites (use with caution)  Peer-reviewed journals (various) ERIC (www.eric.ed.gov/)www.eric.ed.gov/ Google Scholar (www.scholar.google.com/)www.scholar.google.com/ Education Abstracts (Education Full Text) Psychological Abstracts See Training Manual Page 10 86

87. National Center on Response to Intervention 3a) Evaluating Evidence: Where?  What Works Clearinghouse  Best Evidence Encyclopedia  NCRTI Instructional Intervention Tools Chart  Florida Center for Reading Research 87 See Training Manual Appendix B

88. National Center on Response to Intervention 3b) Evaluating Evidence: Type?  What type of evidence is available? Research study Summary of existing research Technical report Peer/publisher claims Other methods? See Training Manual Page 11 88

89. National Center on Response to Intervention 3b) Evaluating Evidence: Type? 89  Randomized control trial*  Quasi-experiment  Single-case design  Quantitative research synthesis * Most rigorous form of research See Training Manual Page 11

90. National Center on Response to Intervention 3c) Evaluating Evidence: Quality?  How was the program implemented? Is it realistic under normal circumstances? Who implemented the intervention? Was the intervention described? Was there a manual or a script? How often did the intervention occur? Was fidelity evaluated in the study? If so, how? What phase of research? (exploratory, pilot, efficacy, scale-up) See Training Manual Page 11 90

91. National Center on Response to Intervention 3d) Evaluating Evidence: Desired Outcomes?  Were the outcomes assessed relevant to your outcomes? What outcome measures were used to evaluate the intervention? Do the outcome measures seem reasonable? Are they relevant to your concerns? Are they reliable and valid? Is it feasible to implement with your population? See Training Manual Page 12 91

92. National Center on Response to Intervention 3e) Evaluating Evidence: Effects?  Are the effects large enough to be meaningful? Significance of differences Interpretation of effect sizes 92 Practitioners can use this information to compare programs and identify those most likely to meet their specific needs. See Training Manual Page 12

93. National Center on Response to Intervention 3f) Evaluating Evidence: Population?  For which population does the evidence show an effect? Is the sample described? Can you tell who was studied? Is the participant similar to or representative of your student population? Are there different effects for different population groups? See Training Manual Page 12 93

94. National Center on Response to Intervention Demonstration  What Works Clearinghouse  Best Evidence Encyclopedia  NCRTI Instructional Intervention Tools Chart 94

95. National Center on Response to Intervention Click here to select a topic area What Works Clearinghouse

96. National Center on Response to Intervention You can use search terms to search for interventions here You can also click on the Find what works link for a more extensive search feature What Works Clearinghouse 96

97. National Center on Response to Intervention 97 What Works Clearinghouse What outcome are you focused on? What grade level are you looking for? Is there a specific focus or population?

98. National Center on Response to Intervention What Works Clearinghouse 98 Click here to create a report What level of effectiveness are you looking for? What is the extent of evidence you are looking for? How will the intervention be delivered? Are you looking for a curriculum, a supplemental intervention, or a practice?

99. National Center on Response to Intervention 99 Use the filter to select your interest Look at the results provided for the student outcome, the improvement index, the effectiveness rating, and the extent of evidence. Click the link to learn more about the intervention and research

100. National Center on Response to Intervention Evaluating Evidence 100 Click here to learn more about the interventions

101. National Center on Response to Intervention Evaluating Evidence 101

102. National Center on Response to Intervention Evaluating Evidence Accelerated Reader™ (Reviewed for Beginning Reading) 102

103. National Center on Response to Intervention 103 Evaluating Evidence Accelerated Reader™ (Reviewed for Beginning Reading)

104. National Center on Response to Intervention Evaluating Evidence 104 Accelerated Reader™ (Reviewed for Beginning Reading)

105. National Center on Response to Intervention Evaluating Evidence 105

106. National Center on Response to Intervention Program Information 106

107. National Center on Response to Intervention Team Activity: What Works Clearinghouse 1.What effective supplemental (Tier II/III), small group, and reading comprehension programs are available for eighth- grade students? 2.How many programs show a positive improvement index for mathematics achievement? 3.Identify a program used in your district/school What is the improvement index, the evidence rating, and the extent of evidence? What additional information is available about the evidence in the program report? 107

108. National Center on Response to Intervention Best Evidence Encyclopedia Click to select the topic: subject matter grade level focus area (e.g., ELLs, struggling readers)

109. National Center on Response to Intervention Finding an Evidence-Based Program Find programs that show high or moderate levels of effectiveness Find programs that show limited evidence of effectiveness Find programs with insufficient evidence or no qualifying studies See the key findings, the summary and, the methods

110. National Center on Response to Intervention Best Evidence Encyclopedia 110

111. National Center on Response to Intervention 111 Review criteria Studies are rated based on the effect size, the type of study, and the size of study

112. National Center on Response to Intervention 112 Selecting an Evidence- Based Program

113. National Center on Response to Intervention Evaluating Evidence 113

114. National Center on Response to Intervention Determine the Type of Program 114 The type of program is defined as

115. National Center on Response to Intervention Learn About the Program 115 The program name, a brief description, and contact information for more information are provided.

116. National Center on Response to Intervention Team Activity: Best Evidence Encyclopedia 1.What programs show strong or moderate levels of effectiveness for eighth-grade reading? 2.Were any small group tutorials (SGT in Column 3) shown to have a strong level of effectiveness for struggling readers? 3.Identify a program used in your district/school: What level of evidence is reported for the program? 116

117. National Center on Response to Intervention 117 www.rti4success.org NCRTI Instructional Intervention Tools Chart

118. National Center on Response to Intervention NCRTI Instructional Interventions Tools Chart 118  The tools chart lists instructional programs that can be used as secondary interventions within an RTI context.  The technical review committee (TRC) does not rate instructional programs; instead the TRC rates studies of program efficacy.

119. National Center on Response to Intervention Instructional Intervention Tools Chart NCRTI definition of instruction:  Additional or alternative intervention programs to the core curriculum conducted in small groups or individually with evidence of efficacy for improving academic outcomes for students whose performance is unsatisfactory in the core program. 119

120. National Center on Response to Intervention Instructional Intervention Tools Chart  Purpose: help consumers identify secondary programs that Have been evaluated through rigorous design. Have shown positive, meaningful treatment effects. 120

121. National Center on Response to Intervention Search by Content: Reading, Math, or Writing Search by Grade: Elementary or Secondary

122. National Center on Response to Intervention Comparing Tools Narrow search selection to programs that fit your needs

123. National Center on Response to Intervention 123 Comparing Tools Compare them side by side

124. National Center on Response to Intervention Instructional Intervention Tools Chart Click on the name of the program to see the implementation requirements.

125. National Center on Response to Intervention Implementation Requirements

126. National Center on Response to Intervention Technical Rigor of the Study 126 Study Quality

127. National Center on Response to Intervention Participants 127

128. National Center on Response to Intervention Design 128

129. National Center on Response to Intervention Fidelity of Implementation 129

130. National Center on Response to Intervention Measures 130

131. National Center on Response to Intervention Technical Rigor of the Study 131 Effect Size Adjusted posttest: corrects for differences in groups on pretest

132. National Center on Response to Intervention Technical Rigor of the Study 132 Effect Size Unadjusted posttest: does not account for differences in groups on pretest

133. National Center on Response to Intervention Effect Size: Sample 1 133 Developer was unable to provide necessary data for NCRTI to calculate effect sizes.

134. National Center on Response to Intervention Effect Size: Sample 2 134 Mean for proximal and distal measures

135. National Center on Response to Intervention Effect Sizes 135

136. National Center on Response to Intervention Disaggregated Data  The column reports effect size data that have been disaggregated for sub groups, if available. Students with disabilities ELLs Students from diverse racial-ethnic groups 136

137. National Center on Response to Intervention Team Activity: Instructional Intervention Tools Chart 1.What secondary instruction programs are available for elementary mathematics students? 2.Which study has the highest average adjusted effect size? 3.What are the average unadjusted effect sizes for SSRD Writing Strategies? 4.Identify a program used in your district/school What level of evidence is reported for the program? 137

138. National Center on Response to Intervention Team Activity Appendix D: Selecting Evidence-Based Practices  As a table group, choose one of the following examples  Think of intervention materials you have or may considering adopting to support the student  Using the three websites, investigate the evidence for that program (or programs)  We’ll reconvene in 20 minutes to report decisions 138

139. National Center on Response to Intervention Appendix D, Example 1: Bell Top Elementary School Ms. Jones, a third-grade math teacher, has noticed that Jack, Bobby, and Jane seem to be struggling with word problems despite the strong research-based core curriculum. In looking at their scores on the screening measure, Ms. Jones noticed that they are all below the cut score. After monitoring their progress over a number of weeks she continued to see no progress. Can you identify an intervention that might help Jack, Bobby, and Jane improve? Why did you select that intervention? 139

140. National Center on Response to Intervention Appendix D, Example 2: Lake Ridge Middle School Mr. Morris has identified that Jessica does not have a strong grasp on the basic skills of reading and is therefore struggling in his class. After looking at her screening data and monitoring her progress, he has decided to implement an intervention to help her with basic reading skills. Is there an intervention available that Mr. Morris might use to help Jessica? What questions might he ask in selecting an appropriate intervention? 140

141. National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 141

142. National Center on Response to Intervention 4) Implement Practices  Provide initial recommended training and professional development.  Plan for initial implementation (e.g., scheduling, materials).  Provide ongoing coaching and professional development.  Monitor and evaluate fidelity of implementation. 142

143. National Center on Response to Intervention What is Fidelity of Implementation? 143

144. National Center on Response to Intervention 4) Implement Practices: Fidelity The best way to monitor fidelity is to measure it.  Consider adherence to schedule, protocols, and intervention plans  Self-Report Data  Observation  Logs/Lesson Plans Training Manual Page 11 144

145. National Center on Response to Intervention Evaluating Implementation: Monitoring Fidelity Self-Report Data Questionnaires, surveys, interviews May provide an indicator of teacher knowledge, context of implementation Often unreliable when used alone

146. National Center on Response to Intervention Evaluating Implementation: Monitoring Fidelity Evaluating fidelity through observation Develop checklists of critical implementation components Record and listen to sessions at random Spot checks Peer observations Peer coaching

147. National Center on Response to Intervention Evaluating Implementation: Monitoring Fidelity Logs/lesson plans Student work Allows for evaluation of what was done Content covered Student progress Provides less information about: Delivery Dosage Adherence to scripts (if applicable)

148. National Center on Response to Intervention 6) Evaluating Implementation Using Fidelity Data Distinguish curriculum/intervention vs. quality of implementation when problems occur Identify implementation strengths (people, potential coaches, parts of intervention) Target areas in need of improvement Coaching Professional Development Retraining

149. National Center on Response to Intervention Selecting Evidence-based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 149

150. National Center on Response to Intervention 5) Evaluate Effectiveness  Evaluate general intervention effectiveness.  Evaluate effectiveness for individual students. Ongoing, graphed progress monitoring data Reviewed at least every 4-6 weeks Evidence-based decision-making criteria General outcome measure versus mastery measure 150

151. National Center on Response to Intervention Growth by Intervention Type Compare Efficacy of Interventions 151 Words Read Correctly

152. National Center on Response to Intervention Intervention Effectiveness 152 Intervention A Goal line Trend line Intervention B Indicates Change

153. National Center on Response to Intervention CLOSING 153

154. National Center on Response to Intervention Things to Remember  Good data IN … Good data OUT Know where your data came from and the validity of that data  Focus on the big picture or ALL students Are most students making progress?  ALL instructional and curriculum decisions should be based on DATA.  Keep it SIMPLE and EFFICIENT! 154

155. National Center on Response to Intervention Implementing the RTI Framework  Select and implement research and evidence-based practices and procedures.  Implement essential components with integrity.  Ensure cultural, linguistic, and socioeconomic factors students bring to the classroom are reflected in tool selection and implementation. 155

156. National Center on Response to Intervention Review Activity  Why does the NCRTI use levels instead of tiers?  Where can districts and schools find evidence of an intervention’s effectiveness?  How can schools monitor the fidelity of intervention implementation? 156

157. National Center on Response to Intervention Next Steps  Develop a district/school multi-level prevention framework model (e.g., guidance document).  Develop an RTI district implementation plan. 157

158. National Center on Response to Intervention 158 National Center on Response to Intervention www.rti4success.org RTI Action Network www.rtinetwork.org IDEA Partnership www.ideapartnership.org Need More Information?

159. National Center on Response to Intervention www.rti4success.org Questions? 159

160. National Center on Response to Intervention 160 This document was produced under U.S. Department of Education, Office of Special Education Programs Grant No. H326E07000.4 Grace Zamora Durán and Tina Diamond served as the OSEP project officers. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this publication is intended or should be inferred. This product is public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: www.rti4success.org.www.rti4success.org National Center on Response to Intervention

161. 161 DISCLAIMER The opinions and positions expressed herein are not intended to ensure compliance with any particular law or regulation pertaining to the provision of educational services for eligible students. This presentation and/or materials should be viewed and applied by users according to their specific needs. This presentation and/or materials represent the views of the presenter(s) regarding what constitutes preferred practice based on research available at the time of this publication. The presentation and/or materials should be used as guidance. Any references specific to any particular education product are illustrative, and do not imply endorsement of these products by OSPI, or to the exclusion of other products that are not referenced in the presentation materials. OSPI, Special Education, is not responsible for the content of those educational product(s) referenced in this presentation. Douglas H. Gill, Ed.D., Director, Special Education WA OSPI