6 Graphs in Education Assessing level and rate Self monitoring IEP goals Batsche, Castillo, Dixon, & Forde, 2008Self monitoringFuchs et al., 2006IEP goalsShinn, 2008Professional accountabilitySulzer-Asaroff & Mayer, 1991Treatment integrityMortenson & Witt, 1998Teacher performance feedbackNoell et al., 2005
7 Why do we like graphs so much? Seeing is believing!A picture is worth a thousand words!People rememberSpeeches that included visuals, especially in color, improved immediate recall by 8.5% and delayed recall (3 days) by 10.1% (Vogel, Dickson, & Lehman, 1990)Visual aids are more effective for communicating large amounts of information quicklyTranscend language barriers (Karwowski, 2006)
8 Is seeing enough?If there are only two data points, basic slope calculation is accurate.
9 Why do I need to know ROI?This is also how to calculate an aimline.
10 Why Bother with all the Data? If there are multiple data points, should you calculate the slope between the first and last data points only?
11 ROI in Educational Decisions Data(slope)Interpretation(progress?)DecisionInstructional Need
12 Quick Stretch of the Mind DataBenchmark: 77Diego: 41InterpretationNOT GOOD!Decision???Will Diego catch up?Do you have enough info?
13 Quick Stretch of the Mind DataBenchmark: 77, 92Diego: 41, 62, 63InterpretationSlightly betterDecision???Will Diego catch up?Do you have enough info?
14 Quick Stretch of the Mind DataBenchmark: 77, 92Diego: 41 … 104InterpretationGood workDecisionCured!!!Will Diego maintain that progress?
15 Quick Stretch of the Mind DataBenchmark: 77, 92Diego: 41 … 74InterpretationUmmmDecisionUh oh…What EXACTLY IS Diego’s Progress??
16 Quick Stretch of the Mind Now can you make a decision?
17 Rationale for Importance of ROI Shift from discrepancy to RtI for SLD eligibility determinationUse of ROI to determine lack of RtIQuestions to be Empirically AnsweredWhat parameters of ROI indicate lack of RtI?How does ROI present between SLD and non-SLD?What are reasonable goals using ROI?
18 Rationale for Importance of ROI School psychological practices including more CBM as a result of IDEA 2004 and NCLB 2001CBM is great because:ComparisonsDual discrepancyEfficientFederal regulationsGraphGOMGoal settingInstructional planningMulti-tiered assessmentMatch instructional levelNorms availableParentsProgress monitoringSurvey level assessmentsSensitive to changeScreeningSystems-levelTechnically adequate
19 Rate of Improvement Algebraic terms: Aimline: expected performance slope is the change in the output because of the input, over timea way of looking at growthAimline: expected performanceTrendline: actual performanceDefinitions:Fuchs and Fuchs (1998)Batsche, Castillo, Dixon, and Forde (2008)
20 Rate of Improvement Shapiro (2008) Criteria for setting reasonable, achievable, and ambitious goalsFuchs, Fuchs, Hamlett, Walz, and Germann (1993)weekly rates of growth in curriculum-based measurementsslopes between 1.5 and 2.0 times that of their peers were most likely to remediate skill deficits with current instruction
21 Functions of ROI Comparisons peers/class local norms/district national norms
22 Functions of ROI A decision tool Shinn’s estimate for decision-making in RTI trainings in Pennsylvania (Kovaleski, 2008)Slope of 2.0x or greater (than expected slope, to close the gap)
23 Rate of Improvement and RTI Fuchs and Fuchs (1998)hallmark components of RTIongoing formative assessmentidentifying non-responsive studentstreatment fidelity of core / supplemental instruction“dual discrepancy”a student performing at or greater than one standard deviation from typically performing peers in(a) level (i.e., grade) and(b) rate (slope)
24 Eligibility and ROI School psychologists Adopt use of additional information (CBM!)“Snapshot” dataGood day, bad day dataVersusData continuously collected over timeData that accounts for good days and bad days!Eyeball the graph vs. decision guidelines
25 Determining SLD in an RTI Model Gresham, 2001RTI is viable alternative for identifying learning disabilitiesStuebing, Fletcher, LeDoux, Lyon, Shaywitz, & Shaywitz, 2002Poor reliability and validity of discrepancy model
26 Determining SLD in an RTI Model Gresham, 20013 models of RTIpredictor-criteriondual discrepancyapplied behavior analytic
27 Determining SLD in an RTI Model Gresham, 2001All three models involved:multiple tiers of interventionprogress monitoringeffective instructional strategiessupport for applying the RTI model to identifying students for learning disabilities
28 Eligibility and ROI Deno, Fuchs, Marston, and Shin, 2001 slopes of children identified as frequently non-responsive to robust interventions approximated the slopes of children already identified as having a specific learning disabilitysupportive evidence for dual-discrepancy model
29 Summary of ROI Research Through an RTI model, students receive instruction based on data. Rate of improvement, or slope, has surfaced in the literature as an indicator for making instructional decisions.Although research is expanding in terms of reliability and validity for both RTI and ROI, extant studies allude to strong utility for educational planning.
31 Get out your laptops!Open Microsoft ExcelI loveROI
32 Setting Up Your Spreadsheet Open Microsoft ExcelIn cell B2, type School Week to represent the weeks of schoolIn cell C2, type Benchmark to represent the benchmarks or goals of the skill you are graphingIn cell D2, type WPM to represent Words Per Minute (or an abbreviation the basic skill you are graphing)
33 Labeling School WeeksIn cell B3, type 1 to represent the first week of schoolContinue numbering to 36 in column 2Why 36? There’s typically 36 school weeks per school year.Finish at cell B38
34 Entering Benchmarks/Goals In cell C3, type the number that represents the fall benchmark (months 1-3) of the skill you are graphing (e.g., 77). This score should be next to school week 1.In cell C20, type the number that represents the winter benchmark (months 4-6) of the skill you are graphing (e.g., 92). This score should be next to school week 18.In cell C38, type the number that represents the spring benchmark (months 7-10) of the skill you are graphing (e.g., 110). This score should be next to school week 36.
35 Entering Student Scores In cells D3 through D38 type the number that represents the score the student achieved during that week of the school year.If a student was not assessed during a certain week of the school year, leave that cell blank*.
36 Entering Student Scores *Do NOT enter zero (0) or a score of zero will be calculated into the trend line and interpreted as the student having read zero words correct per minute that week.
37 Creating a Graph Highlight the data in columns C and D. Left click with mouse in cell C2.While holding down mouse, highlight columns C and D from row 2 through row 38.These will be your data points contributing to your graph.The selection should include the blank cells in column C.
38 Creating a GraphLeft click “Insert” in the tool bar (typically in the top row) while columns C and D are still highlighted.Left click “Chart” in the drop-down menu. A “Chart Wizard” window will appear.
39 Creating a GraphIn the Chart Wizard are two tabs. Make sure you are in the left tab, “Standard Types.”Next, you want to left click on the “Line” graph on the left side of your Chart Wizard.
40 Creating a GraphOn the right side is “Chart sub-type.” Choose the graph with the description “Line with markers displayed at each data value.” This option is typically the first graph icon in the second row.Left click “Next” at the bottom of your Chart Wizard.
41 Creating a GraphThe upper left tab says “Data Range.” The bottom half of that tab has your data range and series type. You want to select “Columns.”
42 Creating a GraphThe top right tab is labeled “Series.” Left click on “Series”The top half of the “Series” tab has an icon of your graph. The bottom half shows you the label your data for the legend.Left click “Next” at the bottom of your Chart Wizard.
43 Creating a GraphThe next options include tabs for Titles, Axes, Gridlines, etc.If you would like to title your chart, left click on the “Titles” tab. Enter a title (i.e., Diego’s Rate of Improvement) in the first box.Enter a description in the Category (X) Axis, which is your number of school weeks. Suggestion: School WeekEnter a description in your Value (Y) Axis, which is the number of the skill you are graphing. Suggestion: Words Per Minute.Left click “Next” at the bottom of your Chart Wizard.
44 Numerical Label to Data Points Click on “Data Labels”Click to put a check mark next to “Value”Click on the “Next” ButtonNote: This can clutter your graph but provide useful info.
45 Creating a GraphLeft click the bottom circle to choose “As object in: …” This will put the graph in the worksheet you have open.Left click “Finish” in your Chart Wizard. This will place your graph in your Excel Document.
47 Creating a GraphYou can choose to have the graph created in a new worksheet within your Excel document.Click next to “As new sheet…”Label the worksheet by clicking on the tab at the bottom of the sheet and typing a name (i.e., Diego’s Graph)Left click “Finish” in your Chart Wizard. This will place your graph in a new sheet in your Excel Document.
49 Resizing the GraphYou may resize the graph by clicking in any white space within your graph which will bring up squares at the corners of your graph.You can put your mouse over these squares and drag the graph to a size you prefer.Keep in mind that a large graph may not print easily.
50 Coloring the GraphRight click in the gray area and a drop down menu appearsClick on “Format Plot Area”
51 Coloring the Graph The Format Plot Area menu appears. To eliminate a color border, click next to “None”To eliminate a color plot area, click next to “None” or choose a color.Click “OK”
52 Coloring Data PointsRight click on the data points for which you would like to change the colorClick on “Format Data Series”Choose the foreground, background, and line color
53 Adding TrendlinesRIGHT click on any of the student’s scores/data points within your graph.Left click “Add Trendline” from the menu that appears.
54 Adding Trendlines Under the “Type” tab, click on “Linear.” Under the “Options” tab, click next to “Custom” and type “Diego’s ROI”Click by “Display Equation…”Click “OK”
55 Adding TrendlinesLeft clicking on the equation will highlight a “box” around it, and clicking on that box will allow you to move the equation above the chart legend to see it better.
56 Adding TrendlinesRepeat this process for the other set of data – your benchmarks. Begin by right clicking on any of your three goal data points.For your benchmark data, label the trendline title “Expected Slope”Click next to “Display equation on chart”Click “OK”
57 Adding TrendlinesMove your equation under the first one
59 Cautions in Interpreting Slope (my disclaimer!) The graph depicts the actual scores as data points on the graph.You can change the size of the font for the numeric values by right clicking on any of them and choosing a smaller font size. This may reduce some of the clutter on your graph.The rate of improvement, or trendline, is calculated using a linear regression, a simple equation of least squares. This line takes into account each score entered to portray an average rate of improvement across school weeks.The equation indicates the slope, or rate of improvement. The number, or coefficient, before "x" is the average improvement, which in this case is the average number of words per minute per week gained by the student.
60 Cautions in Interpreting Slope (my disclaimer continued!) The slope can change depending on which week (where) you put the benchmark scores on your chart. Enter benchmark scores based on when your school administers their benchmark assessments for the most accurate depiction of expected student progress.To add additional progress monitoring/benchmark scores once you’ve already created a graph, enter additional scores in Column D in the corresponding school week. Remember to leave cells blank for the weeks that no score was obtained. The graph will incorporate that score into the set of data points and into the trendline.
61 Program Excel to Calculate ROI Type “ROI” in cell B39 (below your last week of school)
62 Program Excel to Calculate ROI To calculate the expected slope as per the benchmarks:Click on cell C39Put your cursor at the top next to the fxType:=SLOPE(C3:C38,B3:B38)Hit Enter/Return
63 Program Excel to Calculate ROI To calculate the student’s slope:Click on cell D39Put your cursor at the top next to the fxType:=SLOPE(D3:D38,B3:B38)Hit Enter/Return
64 within a Problem-Solving Model ROI as a Decision Toolwithin a Problem-Solving Model
65 Steps Gather the data Ground the data Interpret the data Figure out how to fit Best Practice into Public Education
76 What do we do when we do not get the growth we want? When to make a change in instruction and intervention?When to considered SLD?
77 When to make a change in instruction and intervention? Enough data points (6 to 10)?Less than 100% of expected growth.Not on track to make benchmark (needed growth).Not on track to reach individual goal.
78 When to consider SLD? Continued inadequate response despite: Fidelity with Tier I instruction and Tier II intervention.Multiple attempts at intervention.Individualized Problem-Solving approach.
84 Individual Kid Making growth? How much (65% of expected growth). Atypical growth across the year (last 3 data points).Continue? Make a change? Need more data?
85 Step 4: Figure out how to fit Best Practice into Public Education
86 Things to Consider Who cares about Rate of Improvement? Who is At-Risk and needs progress monitoring?Who will collect, score, enter the data?Who will monitor student growth, when, and how often?What changes should be made to instruction & intervention?
87 Who cares about Rate of Improvement? Explaining the concept of RoI.Creating buy-in with ORF, M-CBM, etc.Defending frequent data collection (the “we spend too much time testing and not enough time teaching” argument).
88 Who is At-Risk and needs progress monitoring? Below level on universal screeningEntering 4th Grade ExampleDORF (110)ISIP TRWM (55)4Sight (1235)PSSA (1235)Student A1155812551232Student B854812161126Student C723510561048
89 Who will collect, score, and enter the data? Using MBSP for math, teachers can administer probes to whole class.DORF probes must be administered one-on-one, and creativity pays off (train and use art, music, library, etc. specialists).Schedule for progress monitoring math and reading every-other week.
91 Who will monitor student growth, when, and how often? Best Practices in Data-Analysis Teaming (Kovaleski & Pedersen, 2008)Chambersburg Area School District Elementary Response to Intervention Manual (McCrea et. al., 2008)Derry Township School District Response to Intervention Model (http://www.hershey.k12.pa.us/ /lib/ /_files/Microsoft_Word_-_Response_to_Intervention_Overview_of_Hershey_Elementary_Model.pdf)
92 What changes should be made to instruction & intervention? Ensure treatment fidelity!!!!!!!!Increase instructional time (active and engaged)Decrease group sizeGather additional, diagnostic, informationChange the intervention
95 Check These Out www.interventioncentral.com www.aimsweb.com
96 Check These Out www.fcrr.org Florida Center for Reading Research What Works ClearinghouseRate of ImprovementNational Center on RtI
97 ReferencesBatsche, G. M., Castillo, J. M., Dixon, D. N., & Forde, S. (2008). Best practices in linking assessment to intervention. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V. (pp ). Bethesda, MD: National Association of School Psychologists.Batsche, G. M., Elliot, J., Graden, J. L., Kovaleski, J. F., Prasse, D., et al. (2005). Response to intervention: Policy considerations and implementation. Alexandria, VA: National Association of School Psychologists.Colvin, G., Sugai, G., Good, R. H., III, & Young-Yon, L. (1997). Using active supervision and precorrection to improve transition behaviors in an elementary school. School Psychology Quarterly, 12,Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52,Deno, S. L., & Fuchs, L. S. (1987). Developing curriculum-based measurement systems for databased special education problem solving. Focus on Exceptional Children, 19(8), 1-15.Deno, S. L., Fuchs, L.S., Marston, D., & Shin, J. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30,
98 ReferencesFlinn, C. S. (2008). Graphing rate of improvement for individual students. InSight, 28(3),Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22,Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13,Fuchs, L. S., & Fuchs, D. (1997) Use of curriculum-based measurement in identifying students with learning disabilities. Focus on Exceptional Children, 30, 1–16.Fuchs, L. S., Fuchs, D., Hamlett, C. L., Hope, S. K., Hollenback, K. N. Capizzi, A. M., Craddock, C. F., & Brothers, R. L. (2006). Extending responsiveness-to-intervention to math problem-solving at third grade. Teaching Exceptional Children, 38,Gresham, F. (2001, August). Responsiveness to intervention: An alternative approach to the identification of learning disabilities. In R. Bradley, L. Danielson, & D. P. Hallahan (Eds.). Identification of learning disabilities: Research in practice (pp. 467–519). Mahwah, NJ: Erlbaum.Individuals with Disabilities Education Improvement Act (IDEIA) of 2004, Pub. L. No. 108,446.
99 ReferencesKovaleski, J. F. (2008, September 19). Pennsylvania promotes best practices in school psychology. Paper presented at the Pennsylvania Training and Technical Assistance Network, King of Prussia, PA.Mortenson, B. P., & Witt, J. C. (1998). The use of weekly performance feedback to increase teacher implementations of a prereferral academic intervention. School Psychology Review, 27,No Child Left Behind Act of Pub. L. No , 115 Stat (2002).Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Giatti, S. L., Williams, K. L., et al. (2005). Treatment implementation following behavior consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34(1),Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V (Vol. 2, pp ). Bethesda, MD: National Association of School Psychologists.Shinn, M. R. (Ed.). (1998). Advanced applications of curriculum-based measurement. New York: Guilford Press.
100 ReferencesShinn, M. R. (2008). Best practices in using curriculum-based measurement in a problem-solving model. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V (Vol. 2, pp ). Bethesda, MD: National Association of School Psychologists.Sulzer-Azaroff, B., & Mayer, G. R. (1991). Behavior analysis for lasting change. Fort Worth: Holt, Rinehart, & Winston.Stuebing, K., Fletcher, J., LeDoux, J., Lyon, G. R., Shaywitz, S., & Shaywitz, B. (2002). Validity of IQ-discrepancy classifications of reading disabilities: A meta-analysis. American Educational Research Journal, 39, 469–518.Upah, K. R. F. (2008). Best practices in designing, implementing, and evaluating quality interventions. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V. (pp ). Bethesda, MD: National Association of School Psychologists.Wright, J. (2008). Internet resources for ‘response to intervention’. Retrieved June 24, 2008, from