1. DATA-BASED DECISION MAKING Using Outcome and Fidelity Data with Individual Student Support Plans Session E12 Kelsey R. Morris, EdD—University of Oregon Diane LaMaster—Illinois State Technical Assistance Collaborative

2. Session Overview Progress monitoring student-specific, behavior support plans Selecting and prioritizing target behavior(s) to monitor Implementing a measurement system Evaluating behavioral progress monitoring data to inform intervention decisions Goal Plan and carry out data collection to monitor target behavior Use graphed progress monitoring data to determine when intervention changes are needed

3. Maximizing Your Session Participation Where are you in your implementation of the concepts presented? Exploration & Adoption Installation Initial Implementation Full Implementation What do you hope to learn? What new learning do you take away from the session? What will you do with your new learning?

4. Data-based Decision Making Effective teams use data to document progress and outcomes, guide decisions, and inform stakeholders (Boudett, City, & Murnane, 2006; Burke, 2010; Deno, 2005; Hill 2010; Newton, Algozzine, Algozzine, Horner, & Todd, 2011; Newton, Horner, Algozzine, Todd, & Algozzine, 2009; Pidgeon & Gregory, 2004; Renfro & Grieshaber, 2009) A critical predictor of sustained implementation of SWPBIS (Coffey & Horner, 2012; McIntosh et al., 2013) Fidelity and student outcome data are essential (Fixsen, Blase, Metz, & Van Dyke, 2013) Continues to be a struggle for schools (Dunn, Airola, Lo, & Garrison, 2013; Schildkamp, Ehren, & Lai, 2012; Telzrow, McNamara, & Hollinger, 2000) Advances in computer technology could provide efficient means for data management (Wayman, 2005)

5. Intensity of Supports Essential Question Is the student successful with this level of support? Intensity is a two-way street. Improved student outcomes are the result of continually monitoring and modifying (as needed) instruction, interventions, and supports.

6. Essential Tier III Systems Elements Coordinating Team Tier III systems planning team meets regularly Team has an identified leader Membership represents behavioral expertise, administrative authority, intensive support expertise, knowledge about students, and knowledge about school operations Student Support Team A uniquely constructed team exists for each individual student support plan Student support team is comprised of relevant stakeholders Student support team exists to design, implement, monitor, and adapt the student- specific support plan Data-based Decision Making Outcome and fidelity data are reviewed by a student’s support team at least monthly Data are used to modify the support plan to improve behavior outcomes and improve fidelity of implementation

7. Student Support Plan Hypothesis Statement Operational description of problem behavior Identification of context where problem behavior is most likely Maintaining reinforcers (e.g., behavioral function) in the identified context Comprehensive Support Teaching strategies Strategies for removing rewards for problem behavior Specific rewards for desired behavior Safety elements, as needed Systematic Evaluation Process for assessing fidelity Process for assessing outcomes Action plan for implementation

8. Intensity of Assessment U niversal—primary prevention Monthly Secondary—small group, targeted Weekly or twice monthly Tertiary—individualized, intensive Daily or multiple times per week

9. Progress Monitoring Progress monitoring is the process of systematically planning, collecting, and evaluating data to inform programming decisions. Helps determine intervention effectiveness Helps in the development of effective support plans

10. Planning Progress Monitoring Selecting target behaviors is part of planning for behavioral progress monitoring. Plan for data collection Select target behavior(s) to monitor Choose method for monitoring the behavior(s) Create a plan for collecting data (e.g., schedule, person(s) responsible) Collect data Evaluate data to make decisions

11. Selecting Target Behaviors Identify the target behavior(s) of concern What does it look like? When does it occur? What is the perceived motivation? Define the target behavior(s) A clear definition allows us to collect more reliable data Can you see it? Can you measure it? Do you know what it is and is not? Prioritize the target behavior(s) More feasible data collection More efficient data analysis More effective decision making because the most important behavior is the focus

12. Develop a Measurement Approach Considerations: How often will data be collected? Related to intensity of behavior and timelines for making intervention decisions Where will data be collected? What context(s)? When will data be collected? Who will collect the data? Consider when, where, and how the data will be collected When and how will the data be entered to allow for analysis?

13. Adult Behaviors Cause Student Change OutcomesFidelity

14. Monitoring and Evaluating Progress Involves examining the progress monitoring data to determine if the student is responding to the intervention and supports. Involves managing and organizing data to support summary and analysis. Consider: Do you have a data system that supports graphing? Who will be responsible for entering the data? Who will be responsible for generating graphs/reports? Who will review/analyze the data?

15. SIMEO Systematic Information Management for Educational Outcomes

16. SIMEO II Online data collection and graphing database system used to assist PBIS tier III student/family teams with data- based decision making. Access to this data through a virtual connection 24 hours a day, 7 days a week Password protected Graphing capability

17. SIMEO Tools and Data Student support teams use SIMEO to/for: Engage students, families, & teachers Team development & team ownership Ensure student/family/teacher voice Identifying true needs and prioritizing items Effective interventions Serious use of strengths Natural supports Focus on needs vs. services Progress monitoring and sustainability System support buy in

18. SIMEO Tools at Tier III Student Disposition Tool (SD-T) Complex FBA/BIP and Wraparound Education Information Tool (EL-T) Complex FBA/BIP and Wraparound Home, School, Community Tool (HSC-T) Wraparound and RENEW RENEW High School Youth Status Tool RENEW H.S.

19. Home, School, Community Tool (HSC-T) 33 items to assist teams with capturing strengths and needs of the student/family across home, school and community Questions address life domain areas: Safety/Medical basic needs Social relationships Emotional functioning Behavioral functioning Cultural/Spiritual

20. HSC-T

21. Milton’s HSC-T HOME SCHOOLCOMMUNITY Student’s family needs to know he is safe. Student needs to learn skills to be independent at home and in the community.

22. Milton’s Student Disposition Tool (SD-T) At Risk of Failure in Home, School, Community Placements

23. Jude’s HSC-T Utility: engage the family and identify needs at baseline HOME SCHOOLCOMMUNITY

24. Jude’s HSC-T Progress Monitoring HOME SCHOOLCOMMUNITY

25. Longitudinal Data HOME SCHOOLCOMMUNITY

26. Jude’s Student Disposition Tool (SD-T) BT2T3T4T5T6T7 ODRODR 1613201615 ISSISSS 2240000

27. Jude’s Education Information Tool

28. ISIS-SWIS Individual Student Information System

29. ISIS-SWIS is a decision system for students receiving more intensive, individualized supports for academic, social, or mental health services. Allows for: Uploading and storing documentation Defining data collection measures Summarizing data for decision making

30. Advantages of ISIS-SWIS Efficiency Structured creation and maintenance of student files One home for progress monitoring, goal setting, and decision making Instantaneous access to data Equity Equal access to quality support plan management Enabling of clear roles, responsibilities, and predictability Quality Supports compliance with federal procedures for Tier III support Comprehensive student file for quality decision making Documentation of progress and intervention history Flexibility Files and measures tailored to a student’s needs

31. ISIS-SWIS Main

32. ISIS-SWIS School-wide Reports

33. Student File: Brian Bender

34. Data Entry Aligned with Measures

35. Brian: Assignment Completion

36. Brian: Assignment Completion & Asking For Help

37. Carly: Rate of Disruption & Staff Fidelity

38. ISIS-SWIS Study The Effects of Self-delivered Performance Feedback and Impact Assessment via the Individualized Student Information System (ISIS-SWIS) on Behavior Support Plan Treatment Fidelity and Student Outcomes (Pinkelman, 2014) Study Conditions Researcher as ISIS-SWIS Faciliator Provide ISIS-SWIS training Provide follow-up coaching and support Provide ongoing technical assistance Trained teachers as ISIS-SWIS Coordinators Trained educational assistants as ISIS-SWIS Users Held weekly meetings with teachers and educational assistants

39. Teachers as ISIS-SWIS Coordinators Manages student educational program Manages student file in ISIS-SWIS Outcome Teacher competent in using ISIS-SWIS Student file established and ready for use

40. EA as ISIS-SWIS User Team member who needs access to student file Enter data into ISIS-SWIS Self-monitor fidelity data Fidelity checklist Rating scale (0-5) Student outcome data Percent of points earned Frequency of problem behavior Number of teacher directed tasks

41. Weekly Meetings EA, teacher, and researcher To ensure EA was using ISIS-SWIS regularly Praise for using specific features of ISIS-SWIS regularly Identify features not being used Model, practice, and give feedback for features not being used Agreement for use Review graphs for data-based decision making Is the plan being implemented with fidelity? Is the plan effective at improving student behavior? Do any changes need to be made?

42. Results: Fidelity Baseline ISIS-SWIS Observed Fidelity Percentage BSP Components

43. Results: Fidelity & Academic Engagement Baseline ISIS-SWIS Academic Engagemen t Percentage BSP Components Percent 10 s Intervals AE Observed Fidelity

44. Results: Fidelity & Problem Behavior Baseline ISIS-SWIS Percentage BSP Components Percentage 10s Intervals PB Observed Fidelity

45. Takeaways Tier III progress monitoring should be individualized just as the Tier III supports are individualized. Student outcome and implementation fidelity data are critical Data-based decision making continues to be difficult for schools (Dunn et al., 2013; Newton et al., 2012; Schildkamp et al., 2012; Telzrow et al., 2000). A lack of efficient tools and systems to assist in data collection, organization, and summarization impedes the process.

46. DATA-BASED DECISION MAKING Using Outcome and Fidelity Data with Individual Student Support Plans Session E12 Kelsey R. Morris, EdD—University of Oregon Diane LaMaster—Illinois State Technical Assistance Center