Presentation on theme: "Team Initiated Problem Solving"— Presentation transcript:
1 Team Initiated Problem Solving Rob Horner, Steve Newton, & Anne Todd,University of OregonBob Algozzine & Kate Algozzine,University of North Carolina at CharlotteCoaches Conference Feb 3, 2010Oregon State UniversityThis session provides a model and some strategies for teams to use as they use SWIS data for problem-solving and decision making.The TIPS model (team initiated problem solving) is a model that has been built as a result of a federal IES grant charged with figuring out how to teach and coach teams when using SWIS data. The project is a collaboration between UO (Horner, Newton, Todd) and UNC at Charlotte (Algozzine’s)Something to remember: The word ‘data’ is plural. Say ‘data are’Anne Todd
2 Today’s Goals Coaches are able to: Clarification Prompt & support facilitator, minute taker and data analyst to prepare for meetingsMeeting Foundations ChecklistPrompt the use of the TIPS model during meetingsData-based Decision-making rulesHelp teams stay focused during meetingsElectronic Meeting Minute formatClarificationCoaches are NOT expected to be TrainersTrainers deliver TIPS team training & help Coaches anticipate errors while guiding them through the possible solutions & adaptationsSome participants will be coaches, some will be trainers, and some will be both. Today we want coaches tounderstand how the Meeting Foundations Checklist and Meeting Minute form are used to support team functioning and sustainability.Be able to use the TIPS problem solving model to simulated SWIS summary dataPlus the three goals above
3 Context Every school has teams Teams are being expected to do problem solvingSelect curriculaGet training and implement new ideas/programsProvide efficient leadership“Communities of Practice”Teams need to report data to administration, district, stateTeams NEED data to do good problem solving.Most teams are not skilled at running problem solving meetings and using data for decision-making.
4 Assumption: Coaching is Critical Teams will need more than a manual or brief training to become skilled at use of data for efficient problem solvingCoaching will be a key element to successful use of good problem solving procedures.
5 What do we need? A clear model with steps for problem solving Access to the right information at the right time in the right formatA formal process that a group of people can use to build and implement solutions.
6 Problem Solving Meeting Foundations Team Initiated Problem Solving (TIPS) ModelReviewStatus andIdentifyProblemsDevelop andRefineHypothesesEvaluate andReviseAction PlanCollectand UseDataDiscuss andSelectSolutionsThe outer circle ( Problem Solving Meeting Foundations) refer to the process and procedures and team uses to function. Roles are determined and defined, meetings are scheduled for the year, electronic equipment and internet access are available, and an agenda is established.The inner circles and arrows define a problem-solving model designed to improve the decision-making and problem solving of PBIS Team. This model is called “TIPS,” which stands for “Team-Initiated Problem Solving.” The model uses data for during problem solving and decision making, during meetings. We will also apply a metric for determining if rates of problem behavior at a school or below, at, or above the national average to determine if there is a problem or not. TIPS teaches teams to use their SWIS data to define precision problem statements. Once the problem statement is precise, a variety of solutions are discussed based on prevention, teaching, reward, correction and extinction and teams will determine which solution(s) they want to implement. The TIPS model then moves the team to action planning ,evaluation and measurement determination.We are finding it more difficult to use these skills in the REAL context, the purpose of this session is to build the skills and build the fluency of using those skills.Develop andImplementAction PlanProblem Solving Meeting Foundations
7 TIPS Model TIPS Training Team Meeting One full day team trainingTwo coached meetingsTeam MeetingUse of electronic meeting minute systemFormal roles (facilitator, recorder, data analyst)Specific expectations (before meeting, during meeting, after meeting)Access and use of dataProjected meeting minutesResearch tool to measure effectiveness of TIPS TrainingDORA (decision, observation, recording and analysis)Measures “Meeting Foundations” & “Thoroughness of Problem Solving”TIPS Training is a package across time starting with one full day of team training (the binder has all the materials, the thumb drive has the files) Followed by two coached meetings. Coaching includes:Prompting & supporting the facilitator, minute taker and data analyst to prepare for meetingsPrompting the use of the TIPS model during meetingsHelping the team stay focused during meetingsPoints to make: tough to build fluency if meeting once a month. The frequency is too lean.
8 This is the observation tool used to measure Meeting Foundations and Thoroughness of Problem-Solving for the research studies.DORA does not stand for Dora the Explorer!
9 Evidence of Effectiveness Evaluation Study ( )Newton et al.,Single-case Study ( )Todd et al.,Group Design Study ( )
10 TIPS Study: Todd et al., 2009 Meeting Foundations Score School A Baseline Coaching TIPSMeeting Foundations ScoreSchool ASchool B% DORA Foundations ScoreSchool CSchool D
11 TIPS Study: Todd, et al, 2009 . Thoroughness of decision-making Baseline Coaching TIPSThoroughness ofdecision-makingSchool A% DORA Thoroughness ScoreSchool CSchool D
12 Building Capacity and Sustainability For Social Competence,Academic Achievement, and SafetyOUTCOMES*Meeting time*Support*Report to FacultySWISSYSTEMSDATAElectronicMeeting Minutes FormBuilding Capacity and Sustainability using the TIPS model to:1. Conduct effective, efficient meetings when using SWIS data for problem solving and decision making.2. Implement and evaluate solutions that result in positive effects on student achievement, social behavior and safety.a. The Information system is the use of the School Wide Information System. The data are current, accurate, believableb. The Practices include the use of the meeting minute form and the problem solving processc. The System is the implementation of Meeting Foundations, the TIPS model, and the documentation of decisions, action plans, and evaluation plansPRACTICESSupportingStaff & Student Behavior and Decision Making
13 Improving Decision-Making via Problem Solving Action Planning & EvaluationProblemSolvingProblemSolutionSteps in the problem solving model.Information/ Data
14 Problem-Solving Meeting Foundations Structure of meetings lays foundation for efficiency & effectiveness
15 Using Meeting Minutes Documentation of Reviewing Meeting minutes Logistics of meeting (date, time, location, roles)Agenda items for today’s meeting ( and next meeting)Discussion items, decisions made, tasks and timelines assignedProblem statements, solutions/decisions/tasks, people assigned to implement with timelines assigned, and an evaluation plan to determine the effect on student behaviorReviewing Meeting minutesAn effective strategy for getting a snapshot of what happened at the previous meeting and what needs to be reviewed during the upcoming meetingWhat was the issue/problem?, What were we going to do?, Who was going to do it and by When?, and How are we measuring progress toward the goal?Visual tracking of focus topics during and after meetingsPrevents side conversationsPrevents repetitionEncourages completion of tasks
16 Organizing for an effective problem solving conversation A key to collective problem solving is to provide a visual context that allows everyone to follow and contributeProblemUse DataOut of TimeSolution
17 Building a system that is NOT person dependent We want to walk into a meeting having no previous history, review the previous meeting minutes be able to fit into any role neededFacilitatorMinute takerData analystActive team memberExample……
18 PBIS Team Meeting Minutes and Problem-Solving Action Plan Form Today’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:Team Members (bold are present today)Today’s Agenda Items Next Meeting Agenda Items01.02.03.Administrative/General Information and IssuesInformation for Team, or Issue for Team to AddressDiscussion/Decision/Task (if applicable)Who?By When?Problem-Solving Action PlanImplementation and EvaluationPrecise Problem Statement, based on review of data(What, When, Where, Who, Why)Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety)Who?By When?Goal, Timeline,Decision Rule, & UpdatesThis slide is animated to teach the different parts of the meeting minute form each click adds the next sectionMost schools have the title at the top and write/type as the meeting progressesMake a point that we don’t need to document everything that happened (i.e., NM rolled her eyes KJ entered the room, SW continued to repeat the same issue, we took at 5 minute bathroom break)Our RatingYesSo-SoNo1. Was today’s meeting a good use of our time?2. In general, did we do a good job of tracking whether we’re completing the tasks we agreed on at previous meetings?3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?4. In general, are the completed tasks having the desired effects on student behavior?Evaluation of Team Meeting (Mark your ratings with an “X”)
19 A completed example…IF a person knows how to use the meeting minute form, the person should be able to pick these minutes up from Jan 7, 2010 and be able to organize previous items to update and facilitate creation of the Feb 3, 2010 agenda
20 Important Structural Components Regular meetings & regular attendanceThe “right” peopleThe right rolesFacilitatorMinute TakerData AnalystActive Team MembersThe right information for problem solving & decision makingAccomplishments – Products of successful meetingMeeting Minutes (record of decisions & tasks concerning administrative/general issues)Problem-Solving Action Plan (record of decisions & tasks concerning problems identified by team)
23 Before the Meeting… Room reserved “New” items solicited for agenda Agenda producedTeam member roles determinedData reviewed by Data Analyst before the meeting; Analyst ready to lead team through discussion of (a) possible new problems and (b) effects of in-process solutions on “old” problemsComputer reserved; access to SWIS online database assuredLCD projector reserved & set up to project data (or team has some other strategy for ensuring team members can review data at meeting)Team members have individual TIPS Notebooks to bring to meeting (We’ll review the (a) before-meeting, (b) during-meeting, and (c) after-meetings responsibilities of individual team members later in this workshop)
24 At Close of and After Meeting… Meeting Minutes and Problem-Solving Action Plan completedCopy of Meeting Minutes & Problem-Solving Action Plan distributed to each member within 24 hrs.
26 Activity Complete the Foundations Checklist Use the PBIS team you know bestHow would you use the Foundations Checklist to help a school team that was preparing to adopt TIPS procedures?
27 Problem Solving Meeting Foundations IdentifyProblemsTeam Initiated Problem Solving (TIPS) ModelDevelopHypothesisEvaluate andReviseAction PlanCollectand UseDataAsk participants about knowledge and experience of SWISDiscuss andSelectSolutionsDevelop andImplementAction PlanProblem Solving Meeting Foundations
28 SWISTM (School-Wide Information System) DefinedSWISTM is a web-based information system for gathering, entering, summarizing, reporting and using office discipline referral informationPurposeA progress monitoring tool for improving the ability of school personnel to develop safe and effective learning environments
29 Three Key Elements of SWISTM Data Collection SystemCoherent system for assigning referralsProb. behavior definitions, referral form, rules for referralAllocation of FTE to enter data, build reports.Computer ApplicationWeb-based, continuously available, secureDecision-makingUse of dataSchool-wideIndividual Student
30 Features of SWISTM Only reports discipline data Major office referralsMinor discipline offencesHighly efficient (30 sec per referral)Local controlFormatted for decision-making (pictures)Information is available continuouslyConfidential, secureCan be combined with district data baseTeams need access to the data AND a person or two who know how to access and navigate through the data. Ideally, this should happen during a meeting when team members are asking questions about the data. Since SWIS is web-based, it is always available through an internet browser. We need to help people learn how to use the data in an interactive format during a meeting, when the data are most needed. moving the group through the simulations helps to build fluency of the skills needed.
32 How SWISTM works Data Entry School Address and Contact Enrollment/Ethnicity/Days per monthStaff InformationStudent InformationReferralsReportingAverage Referrals per Day per monthReferrals by Problem BehaviorReferrals by LocationReferrals by TimeReferrals by StudentOther ReportsTools
34 Total Office Discipline Referrals as of January 10 Let’s talk about accuracy of the data again. When you begin to use the data and draw comparisons, the data need to be comparable. Look at the data above. First, as a data analyst, you look and see, ‘wow.. Things are getting better, the graph is going down’…. Then you do what you are supposed to do first, and look at the label on the Y axis. This label says total office discipline referrals. It is great to compare the total ODRs, but now… look at the X-axis. There are a different number of days in each month and the number of schools day in each much has a wide range (Dec may have 10 school days, January may have 19 school days). These months, the way they are arrayed here, are not comparable and this data should not be used! If you aren’t using SWIS, do the math to get average referrals per day per month by using the total referrals and the total days each month. If you are using SWIS, do not fear….. (next slide)
35 SWIS does that calculation for you. look at the Y-axis label now SWIS does that calculation for you. look at the Y-axis label now. Average referrals per day per month allow us to compare months. Now look at the trend….. ‘we are going to have a wild spring term if we don’t do anything differently!). This is the same set of data on the previous slide and look at what the pattern of data does for the problem solving process.Accurate data and data that are formatted for purposes of making decisions is critical.I like to make this a bit dramatic by going back and forth between this slide and previous, telling them they are the team and they are reviewing this data…
44 Organizing SWIS Data for Decision-making Universal Screening ToolProportion of students with0-1 Office Discipline Referrals (ODRs)2-5 ODRs6+ ODRsProgress Monitoring ToolCompare data across timePrevent previous problem patternsDefine Problems with precision that lead to solvable problemsOK…. Building precision problem statements is a skill that is needed for using the data. organizing and interpreting the data requires another set of skills.Slides provide a sequence of slides to illustrate different precision statements based on different pictures of the same type of data.
45 Using office discipline referrals as a metric for universal screening of student social behavior ~5%~15%0-1 office discipline referral~80% of Students
46 Using the Referrals by Student report as a Universal Screening Tool
47 Jennifer Frank, Kent McIntosh, Seth May Cumulative Mean ODRs Per Monthfor 325+ Elementary Schools 08-09Cumulative Mean ODRs
48 Using ODRs to Identify Problems Build a picture for the pattern of office referrals in your school.Compare the picture with a national averageCompare the picture with previous yearsCompare the picture with social standards of faculty, families, students.GoalIdentify problems empiricallyIdentify problems earlyIdentify problems in a manner that leads to problem solving not just whining
49 Using ODRs to Identify Problems Build a picture for the pattern of office referrals in your school.Compare the picture with a national averageCompare the picture with previous yearsCompare the picture with social standards of faculty, families, students.
50 SWIS summary 2008-2009 (Majors Only) 3,410 schools; 1,737,432 students; 1,500,770 ODRs Grade RangeNumber of SchoolsAvg. Enrollment per schoolNational Avg. for Major ODRs per 100 students, per school dayK-62,162450.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days6-9602657.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days9-122158871.27 = more than 1 Major ODR per school day, or about 127 every 100 daysK-(8-12)4314081.06 = about 1 Major ODR per school day, or about 106 every 100 daysHow to use these numbers:Your enrollment (e.g. 400 students or 225 students)Divide by 100 (e.g. 400/100 = 4; 225/100 = 2.25Multiply by the National Average to get ODR per day(4 X .34 = X .34 = .76)
51 SWIS summary 2008-2009 (Majors Only) 3,410 schools; 1,737,432 students; 1,500,770 ODRs Grade RangeNumber of SchoolsAvg. Enrollment per schoolNational Avg. for Major ODRs per 100 students, per school dayK-62,162450.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days6-9602657.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days9-122158871.27 = more than 1 Major ODR per school day, or about 127 every 100 daysK-(8-12)4314081.06 = about 1 Major ODR per school day, or about 106 every 100 days
52 Compare with National Average 150 / 100 = X = .51Elementary School with 150 Students
53 Compare with National Average 450 / 100 = X = 1.53Elementary School with 450 Students
54 Application Activity: Absolute Value Is there a Problem?Middle School of 625 students?Compare with national average:625/100 = X .85 = 5.31Office Discipline Referrals per School Day
55 High School: Compare with National Average 1800 / 100 = X 1.27= 22.86High School of 1800 studentsBuild the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?
56 High School: Compare with National Average 450 / 100 = X 1.27= 5.17High School of 450 studentsBuild the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?
57 Middle School of 700 students Slides work as a set. This slide is a precursor to the next slide to show how to start with ‘this years’ data and then use it to compare to last year (slide 32)Build the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?
58 Build the routine when reviewing these slides: How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?What happened last year that we don’t want to repeat?
59 Final slide of this setBuild the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?What are we going to do next fall to maintain this success?
60 Identification of Problem (for example...) Our average Major ODRs per school day per month are higher than national average for a school of our enrollment sizeOur average ODRs per school day per month are higher this year than for corresponding months of previous yearOur average ODRs per school day per month are showing an increasing trendFaculty, parents, and students say our ODR levels are too high
61 More Precision Is Required to Solve the Identified Problem Define problem by identifying What problem behaviors are involved in ODRsClarify problem by identifyingWhen ODRs are occurring (time of day)Where ODRs are occurring (location)Who is engaging in problem behaviors that result in ODRsWhy are problem behaviors continuing to occur
62 Problem StatementsUltimately, you want to write a “problem statement” that precisely specifies the problem you identifiedThe more Ws (what, when, where, who… why) you incorporate into the problem statement, the more precise the problem statement will beThe more precise the problem statement, the easier it will be to generate a solution that “fits” the problem
63 Which Statement Is More Precise? 1a. Too many ODRs1b. Total of 22 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 15 different students involved; aggression appears to provide peer attention, and resolve unclear playground rules (who gets equipment),2a. Behavior in cafeteria is uncivil and unsafe.2b. Verbal threats and gender harassment in the cafeteria are increasing; 80% of events are from 4 students during second lunch; We are unclear what is maintaining these behaviors.3a. Hallway noise is unbearable.3b.4a. The number of ODRs per day has increased by 20% each month since school started.4b.
64 Which Statement Is More Precise? 1a. Too many ODRs1b. Too many instances of disrespect2a. Too many ODRs between 1:00pm and 1:30pm2b. Too many ODRs in the afternoon3a. Too many ODRs occurring outside the classrooms3b. Too many ODRs on the playground4a. 25% of students have at least 2 ODRs4b. Many students are experiencing ODRs5a. Too many ODRs on the playground5b. Total of 12 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 8 different students involved; aggression appears to provide peer attention.
65 Use Schoolwide Information System (SWIS) Data to Achieve Precision QuestionSWIS Table/GraphWhat problem behaviors are occurring?Referrals by problem behaviorWhen are problem behaviors occurring?Referrals by timeWhere are problem behaviors occurring?Referrals by locationWho is engaging in problem behaviors?Referrals by studentWhy do problem behaviors keep happeningReferrals by motivation
66 Solutions – Generic Strategies Prevent –Remove or alter “trigger” for problem behaviorDefine & Teach –Define behavioral expectations; provide demonstration/instruction in expected behavior (alternative to problem behaviorReward/reinforce –The expected/alternative behavior when it occurs; prompt for it, as necessaryWithhold reward/reinforcement –For the problem behavior, if possible (“Extinction”)Use non-rewarding/non-reinforcing corrective consequences – When problem behavior occurs Although not a “solution strategy,” Safety may need to be considered (i.e., procedures that may be required to decrease likelihood of injuries or property damage)
67 Trevor Test Middle School Hypothesis:Prevent “Trigger”Define & TeachReward/ReinforceWithhold RewardCorrective consequenceOtherSafety
68 Implementing Solutions Who is going to do it?When will they do it?Minute Taker writes this information down, facilitator follows up at next meeting on status of implementation
69 Evaluating Solutions Define the goal for solving the problem What will ‘it’ look like when you say it is not a problemDefine how you will know that the solutions were implemented as planned (with fidelity)?How often will you conduct a status review?Define how you will know that the solutions had a positive effect on student achievement, social competence, and/or safety?How often will you monitor student progress?
70 Achieving a Precise Problem Statement for Fictional Trevor Test School Middle School – Grades 6, 7, & 8565 students
71 Trevor Test Middle School n= 565 grades 6-8 Is there a problem Trevor Test Middle School n= grades 6-8 Is there a problem? Compare to national average, compare to last year, examine trend, examine peaks?565/100 = 5.65; X .85 = 4.8
72 Trevor Test Middle School Identified Problem for last 4 mos., Major ODRs per day higher than national avg.increasing trend across all 5 mos.
73 Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.)
75 What information do we need? Who is involved in problem behavior in the cafeteria?ODRs in the Cafeteria
76 Main problemThe sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention.
77 Trevor TestThe sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention.
78 Trevor Test Middle School Hypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in cafeteria expectations; attention from adults and peers rewarding disruptionPrevent “Trigger”Change lunch schedule so fewer students are eating between 11:30 AM & 12:00 PM?Define & TeachFocus on 6th graders; define cafeteria expectations; develop and post expectation signage in cafeteria; demonstrate/teach expectations in class periods occurring just prior to lunchReward/ReinforceSet up “Friday 5” (extra 5 mins. of lunch time on Friday, if no ODRs occur in cafeteria during lunch time)Withhold RewardEnsure staff don’t argue back and forth with student if instance of disruption occurs (may be an inadvertent reward); remind students that paying attention to a disruptive student can mess up Friday 5Corrective consequenceEnsure active supervision during lunch (add one supervisor between 11:30 AM and 12:00 PM?); ensure quick corrective consequence, per our handbookOtherDetermine whether Behavior Support Program has been initiated for Student #10; if it has, make sure it includes focus on disruption in cafeteriaSafety
79 Trevor Test Solution Actions Choose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective.Choose least amount of work that will have the biggest impact on decreasing the problem.Implementing the solution requires action and time linesProblems need goals so that we can measure progress and know when to move on.Use weekly 1-5 survey of cafeteria monitors to assess implementation of planAre we doing the plan?1 ….. 2 …..3 ….. 4 ….. 5No Yes
80 Trevor Test Solution Actions Choose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective.Choose least amount of work that will have the biggest impact on decreasing the problem.Implementing the solution requires action and time linesProblems need goals so that we can measure progress and know when to move on.Use weekly 1-5 survey of cafeteria monitors to assess implementation of plan
81 Problem Solving Action Plan Precise Problem StatementSolution ActionsWho?When?Goal, Timeline, Rule & UpdatesMany 6th grade students are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attentionPrevention: Maintain current lunch schedule, but shift classes to balance numbersTeach: Teach behavioral expectations in cafeteriaPrincipal to adjust schedule and send to staffTeachers will take class to cafeteria; Cafeteria staff will teach the expectationsChanges begin on MondayRotating schedule on November 15Goal: Reduce cafeteria ODR’s by 50% per month (Currently 24 per month average)Measure:1. SWIS ODRs2. Brief fidelity surveyTimeline: Review monthlyRecognition: Establish “Friday Five”: Extra 5 min of lunch on Friday for five good daysExtinction: Encourage all students to work for “Friday Five”… make reward for problem behavior less likelySchool Counselor and Principal will create chart & staff extra recessPrincipal to give announcement on intercom on MondayCorrective Consequence- Active supervision and continued early consequence (minor/major ODR’s)Hall and Cafeteria SupervisorsOngoingData Collection – Maintain ODR record & supervisor weekly reportSWIS data entry person & Principal shares report with supervisorsWeeklyThe minute taker/recorder needs to build fluency in getting relevant information documented. This is the plan for Trevor Test Middle School.
86 Application of model when monitoring individual student progress
87 Phoenix Elementary 265 Students K-5 A second simulation Judge the time you have left to determine how to go through each of these. The big idea here is to have data for one school in one spot for discussion to define a precision problem statement, solutions, action plan and evaluation plan. An action plan says who will do what by when. An evaluation plan is a plan for specifying how the team will know that their efforts have (1) been implemented as planned (fidelity) and (2) that student outcomes have been effected (decreased rates of problem behavior). A question that needs to be answered is ‘what is the goal?’ or ‘what will it look like for us to say, we don’t have a problem?’
89 Phoenix Elementary 265/100 = 2.65 2.65 x .34 = .901 Do we have a problem?03-0404-05Trends, patterns. Peaks?What can we anticipate?
90 Phoenix Elementary Problem Behaviors Build the precision problem statementAggression, disrespect, inappropriate language
91 Phoenix Elementary Locations Year OneYear TwoOn the playground
92 Phoenix Elementary - Time When are the problems happening? Get school schedule and look at recesses, who is out there, who is supervising, etc?
93 Phoenix Elementary Referrals Per Student (2 + Referrals) Lots of studentsThis leads me to wanting to build a custom report by grade level. Matching that data with the by time data gives more specific information
94 Problem Statement Do we have a problem? Build a precise problem statementGive best guess on hypothesisOther information sources lead to sharing equipment, taking turns on swings, different games rules for soccer during recess and during soccer games.Have people write the problem statement down. It sounds silly, but it is something people don’t want to take the time to do, but we need them to practice. We want to build fluency for writing out precision problem statements so that it is easier to do during a meeting context.
95 Solution Development problem statement & hypothesis: PreventionTeachingRewardExtinctionCorrective ConsequenceData CollectionHave participants come up with ideasThis table is a prompt for getting the discussion going. teams do not need to identify a solution for each category above, but the categories provide prompts for the team:What can we do to prevent the problem?What can we do to teach the students to do what we expect?What can we do to reinforce appropriate student behavior?What can we do to minimize reinforcement for the problem behavior?What can we do to correct inappropriate behavior?What data will we collect to measure effectiveness of solutions?Choose a solution or a package of solutions to implement. Do not default to doing everything listed. Always choose the least amount of tasks/actions that you think will produce the biggest effect.
97 Langley Elementary School 478 StudentsK-5Generate a discussion after 5 minute review of the data.What is the first thing we do? Calculate ODR per 100 studentsReview the other graphs, which of the big 5 are missing?How would you create that report? (custom graph by motivation).What else would you want to know about this school?
98 A completed example…IF a person knows how to use the meeting minute form, the person should be able to pick these minutes up from Jan 7, 2010 and be able to organize previous items to update and facilitate creation of the Feb 3, 2010 agenda
99 Pattern of problem behavior is getting better Pattern of problem behavior is getting better. what did we do in Feb and March to see this outcome? How will we celebrate this success? How will we maintain this success?However, there are two possible problems (as you look at location and at problem behavior. Students typically get tardies from the classroom not the playground) start with two problem statements:Classroom, tardiesPlayground, 10:30, 12:00, 12:30.To get more precise with these two statements, use SWIS custom reports to generate a by grade report and a by motivation for each of the two locations
100 Talk about options for this graph. Get names of students… Talk about options for this graph. Get names of students…. We have many students getting many referrals….. Classroom, playground, both?
101 Precision Statement/Hypothesis WhatWhereWhenWhoWhyWhat other info needed?Possible Solutions?What are Langley’s precision statements. Since you don’t have access to other data, make best guesses for the sake of the discussion or at a minimum talk about what other reports would be needed.Then move into discussing possible solutions. The more you can reinforce function-based solutions, the better. IF the possible motivation is escape, what solutions might work best?; what about if the possible motivation is peer attention?, how would that change the discussion?
102 Solution Development problem statement & hypothesis: PreventionTeachingRewardExtinctionCorrective ConsequenceData Collection
103 PBIS Team Meeting Minutes and Problem-Solving Action Plan Form Today’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:Team Members (bold are present today)Today’s Agenda Items Next Meeting Agenda Items01.02.03.Administrative/General Information and IssuesInformation for Team, or Issue for Team to AddressDiscussion/Decision/Task (if applicable)Who?By When?Problem-Solving Action PlanImplementation and EvaluationPrecise Problem Statement, based on review of data(What, When, Where, Who, Why)Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety)Who?By When?Goal, Timeline,Decision Rule, & UpdatesThis slide is animated to teach the different parts of the meeting minute form each click adds the next sectionMost schools have the title at the top and write/type as the meeting progressesMake a point that we don’t need to document everything that happened (i.e., NM rolled her eyes KJ entered the room, SW continued to repeat the same issue, we took at 5 minute bathroom break)Our RatingYesSo-SoNo1. Was today’s meeting a good use of our time?2. In general, did we do a good job of tracking whether we’re completing the tasks we agreed on at previous meetings?3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?4. In general, are the completed tasks having the desired effects on student behavior?Evaluation of Team Meeting (Mark your ratings with an “X”)
105 Sandhill High school 354 students Another simulation to work through as a small or larger group.Go through the same steps as in previous simulations.
106 Sandhill High School: 354 students What is our calculated rate per 100 students?How are we doing? As compared to national average? As compared to last year?What are our patterns, trends?What should we anticipate, try to prevent?
109 Sandhill –Referrals by Student Lots of students
110 Sandhill - Time7:00 was used as a default if ‘time’ was not written on the referral formFirst period and many ‘unknown’. The staff need to be more precise with ‘time of incident’ . 7:00am has been used as a default for ‘unknown’. THIS SWIS Facilitator did not hold the team to a compatible ODR form and didn’t put a place for time of incident. If the time was not noted on the ODR, the data entry person was taught to enter 7:00am as ‘unknown’
111 Sandhill hypothesisStudents are skipping class to avoid doing the work8: oversleeping?Other times: avoid class? Gain more social time with peers?We don’t know exactly when due to the 7:00 time used as a default
112 Precision Statement/Hypothesis WhatWhereWhenWhoWhyWhat other info needed?Possible Solutions?Verbalize and write down the precision problem statement. What more do you need to know and how will you get that information?
113 Solution Development problem statement & hypothesis: PreventionTeachingRewardExtinctionCorrective ConsequenceData CollectionSolutions for tardies in class?The best strategy I have heard from high school teachers is to model ‘being on time’, define what being on time looks and sounds like, and before each class period starts, have a potential test question on the board for students to copy as they construct a study guide.
115 Team Training & Follow Up Swift at SWIS TrainingTeam Meetings:First time: Simulated DataSecond time: Use First Month of DataThird time: Problem Solving StatementsWorking with the team overtime is essential. Plan how this will work with your teams. Will you teach the data analyst to create the big 5 report, or will you do it in your SAMI account and get it to them before the meeting? what will your coaching look like? How will you build fluency of team members to build precision problem statements, discuss possible solutions, and build an action and evaluation plan?
116 Next Steps As a field: Coaches: Add TIPS training to Trainer repertoireIntegrate messages, language, and processes for using data for problem solving and progress monitoring across the stateDetermine impact of TIPS on student outcomes (next grant proposal)Coaches:Prompt teams to not only define precision problem statements but to also define a goal for ‘what it will look like’ when we don’t have a problemPrompt team members to be effective and efficient in their rolesdata analysts create and summarize data to jump start the meetingminute takers record relevant information(not novels) about problems discussed, solutions determined and action plan to implement solutionsfacilitators ask questions to facilitate problem solving and decision makingAsk for supportTell the Network what you need in order to be successful in your role(s)
117 Getting a SWIS account www.swis.org Work with a SWIS Facilitator to complete a License AgreementTen readiness requirements includingPositive School culture is a priorityTeam identified to use the data at least monthlyConsistent, coherent procedures for dealing with problem behavior (process and documentation)Data entry time and person scheduledCost of SWIS$250 per year (additional $50 for Check in Check out)