Presentation on theme: "Susan Barrett www.pbis.org Working Smarter Susan Barrett www.pbis.org."— Presentation transcript:
1Susan Barrett www.pbis.org Working SmarterSusan Barrett
2Building the Plane in Flight 3-5 year processMoving Beyond Universals- same logic-Systems, Systems, SystemsOutcomesImplementing and Sustaining Evidence Based Practices through Data Based Decision MakingBuild skill development“Knowledge about and skill to do”Build capacity
3Spooner Elementary School: Working Smarter Activity Form Committee/ initiative/ work groupPurposeTarget GroupMembershipRelationship to school mission & School Improvement plan1= low 3= highOverall priority1= low 3= highDiscipline CommitteeTo deal with behavioral/discipline issues across campus – meets monthlyAll student’s across campusMrs. JonesMrs. SmithMr. GarciaMr. BlackSchool Safety CommitteeTo ensure safety across the school campus – meets monthlyAll students across campusMr. RogersMr. WilliamsMrs. FordSkills Streaming InitiativeTo create social skills curriculum for all grade levels across campus – meets mthlyMrs. HappyMr. O’ConnorMrs. GeorgeSocial InitiativeIs to schedule activities across the members- meets monthlyAll faculty and staff across campusMrs. O’ShannonMrs. PopeMr. SmithMrs. ColeHere is an example of the activity we’ll be working with.Committee, Purpose, Target group, etc.(FOCUS) who is on those committees and what is their relationship to school mission and SIP and overall priorityIf you think about the mission statement for Spooner Elementary in the previous slide, you can see that, for example, the purpose of the Skills Streaming Initiative has a high relationship to the school mission statement, but a lower overall priority. Does this suggest something to the PBS team?Which of these committees overlap?Do we need two committees doing discipline and safety? What’s the difference?
4WORKING SMARTERCommittee/ initiative/Work GroupPurposeTarget GroupMembershipRelationship to School Mission & School Improvement Plan1= low 3= highOverall Priority1= low 3= highGroups can do this on the provided form or can do this on flip charts if they are available. In that case they would begin listing committees and working on the purpose etc. and then come back and indicate the relationship to school mission and overall priority.( This tends to work well and keeps the group together and on track.)
5What is School-wide Positive Behavior Support? School-wide PBS is:A systems approach for establishing the social culture and individualized behavioral supports needed for schools to achieve both social and academic success for all students.Evidence-based features of SW-PBSPreventionDefine and teach positive social expectationsAcknowledge positive behaviorArrange consistent consequences for problem behaviorOn-going collection and use of data for decision-makingContinuum of intensive, individual interventions.Administrative leadership – Team-based implementation (Systems that support effective practices)
6Establishing a Social Culture Common LanguageMEMBERSHIPCommon ExperienceCommon Vision/Values
7Six Basic Recommendations for Implementing PBIS Never stop doing what is already workingAlways look for the smallest change that will produce the largest effectAvoid defining a large number of goalsDo a small number of things wellDo not add something new without also defining what you will stop doing to make the addition possible.
8Six Basic Recommendations for Implementing PBIS Collect and use data for decision-makingAdapt any initiative to make it “fit” your school community, culture, context.FamiliesStudentsFacultyFiscal-political structureEstablish policy clarity before investing in implementationLAUSD.pptx
9Systems for Students with High-Risk Behavior SCHOOL-WIDE Tertiary Prevention:SpecializedIndividualizedSystems for Students with High-Risk BehaviorSCHOOL-WIDEPOSITIVE BEHAVIORSUPPORT~5%Secondary Prevention:Specialized GroupSystems for Students with At-Risk Behavior~15%Primary Prevention:School-/Classroom-Wide Systems forAll Students,Staff, & Settings~80% of Students
109 Indicators of Effective Schools 1. A clear and shared focus2. High standards and expectations for all students3. Effective school leadership(create school culture conducive to student learning)4. High levels of collaboration and communication5. Curriculum, instruction and assessments aligned with state standards6. Frequent monitoring of learning and teaching7. Focused professional development8. A supportive learning environment9. High levels of family and community involvementOffice of the Superintendent of Public Instruction, 2003
11Competing or Coordinated Need for a framework, the anchor, for all school improvement effortsCommon language, Common logic
13Financial Crisis or Opportunity? Tiered Logic offers platform for connecting best practice to achieve academic and social emotional outcomes for school improvementMust braid resources and work smarterMust be better consumers of the research
14Student Outcomes- Academic and Socal TechnicalIntegrated & CompensatoryPerformance Assessment (Fidelity)CoachingTrainingSelectionSystems InterventionFacilitative AdministrationDecision Support Data SystemInnovationOrganizationLeadershipAdaptiveFixsen, Naoom, Blase, Friedman, & Wallace, 2005(c) Dean Fixsen and Karen Blase, 2008
15Stages of Implementation Exploration Installation Student BenefitsTechnicalIntegrated & CompensatoryPerformance Assessment (Fidelity)CoachingTrainingSelectionSystems InterventionFacilitative AdministrationDecision Support Data SystemInnovationOrganizationLeadershipAdaptiveStages of ImplementationExplorationInstallationInitial ImplementationFull ImplementationInnovationSustainability2 – 4 YearsFixsen, Naoom, Blase, Friedman, & Wallace, 2005(c) Dean Fixsen and Karen Blase, 2008
17Educational Initiatives Guiding Principles (Coyne 2008)Promoting evidence based practicesSupporting change at the systems level (feasible, consistent and relevant to local needs)Developing local capacity to sustain effective practices over time
18Maryland’s Tiered Instructional and Positive Behavioral Interventions and Supports (PBIS) Framework Academic SystemsBehavioral SystemsIntensive, Individually Designed InterventionsAddress individual needs of studentAssessment-basedHigh IntensityIntensive, Individually Designed InterventionsStrategies to address needs of individual students with intensive needsFunction-based assessmentsIntense, durable strategies1-5%1-5%Targeted, Group InterventionsSmall, needs-based groups forat risk students who do not respondto universal strategiesHigh efficiencyRapid response5-10%5-10%Targeted, Group InterventionsSmall, needs-based groups for at risk students who do not respond to universal strategiesHigh efficiency/ Rapid responseFunction-based logicAs you all know, this is the public health framework for behavior and instructional supports in Maryland through PBIS and RtI..Core Curriculum andDifferentiated InstructionAll studentsPreventive, proactiveSchool-wide or classroomsystems for ALL students80-90%Core Curriculum andUniversal InterventionsAll settings, all studentsPreventive, proactiveSchool-wide or classroom systems for ALL students and staff80-90%18
19Suspension Reduction Strategies Maryland’s Tiered Instructional and Positive Behavioral Interventions and Supports (PBIS) FrameworkSchool Mental HealthSocial Emotional Learning CurriculumSuspension Reduction StrategiesStudent ServicesTruancy Reduction StrategiesFOR ALL OF THE ABOVE-----There are strategies at all three levels.The students on which this group is focused have access to those interventions at all three levels. When universal strategies are implemented they target ALL STUDENTS, the ED student may need additional support in one area so would be identified for a smaller group, targeted intervention. If individual supports were determined to be necessary based on the ongoing monitoring of interventions, they would be put in place.Onoe of our mantras is : If we can predict it we can prevent it (this requires some function-based thinking____ NOT FBA, but thinking through the motivation for the behaviors. The more we can prevent the behavior that takes the student out of the classroom, the better access to quality instruction the student has/In fact students move up and down this triangle based on their competence in specific areas. We know that students with emotional issues get better and get worse, depending on their diagnosis, level of support, stressors, etc.As I understand it there are no competencies established in Maryland for Social Emotional Learning. That is the gap in the integration of SEL into the framework.Special Education Assessment and ReferralBullying Reduction Strategies
20Applying the Three-Tiered Logic to Charles Carroll Triangle Activity:Applying the Three-Tiered Logic to Charles CarrollTier 3 FBA / BIP IEP / 504 Wilson ReadingInclusion PD Assistive TechnologyFraming Your Thoughts Touch MathTier 2 Focused Guidance Groups IST AEL, EIR, SOARCheck-in / Check-out SuccessMaker+Behavior Contracts Double Dose InstructionPST; PPW RIT ResourceFundationsStudent Intervention PlansTier 1 PBIS Expectations Leadership Team Explicit InstructionChippy Coupons Inclusive School Gradual Release ModelSchoolwide Celebrations Data Binders Specific Learning TargetsPBIS Lesson Plans & Matrix PTA Partnership Objective DeconstructionPBIS Brochure & Home Connection Formative Assessment PDCharacter Education Community Partners Habits of Mind FocusGuidance Counselor Progress Monitoring-SITAligned Discipline Referrals Grades 3-5 SuccessMakerCelebree Daycare Assessment-Data PDSACheck Club VolunteersYearbook Club Gr. 4-5 Geography Club
21Applying the Three-Tiered Logic to Your School Triangle Activity:Applying the Three-Tiered Logic to Your SchoolTier 3Tier 2Tier 1
23Working Smarter For each practice… Outcome: How is the practice linked to overall outcome outlined in your school improvement plan?Systems: Identify System of support for teachers?(Service Delivery Team- Leadership Team, Student Services Team, School Counseling)What are roles and responsibilities of each team? How do teachers and support staff access these supports?What are the structures that support skill development?Data:What tools to measure fidelity and progress monitoring tools used to measure effectiveness-How do you know the practice makes the impact?
24Building Solutions: Using Data to Drive Decisions to Improve Behavior in Your School Presented by Susan BarrettRob Horner, Anne Todd, Steve Newton,Bob Algozzine, Kate Algozzine
25Goals Logic for school-wide Positive Behavior Support Design of “decision-systems” for schoolsCollection and use of data for decision-makingWhole schoolTargeted groupsIndividual student
26Main MessagesSupporting social behavior is central to achieving academic gains.School-wide PBS is an evidence-based practice for building a positive social culture that will promote both social and academic success.Implementation of any evidence-based practice requires a more coordinated focus than typically expected.
27Get honest about issues concerns in your building- Administrator is key!! Establish a kind of “haven”- place that individuals can get feel safe about reporting concerns, supported by school community and empowered to be a part of the decision making process- “Community of Practice”Tools: EBS Self assessment, climate surveys, satisfaction surveys- provide data summaries within a week of return – decide best approach to deliver feedback
28Elements to the Data Process A. Establish A Coherent Process for DisciplineBehavior DefinitionsMinor v. MajorWritten procedures for staffFlow chart showing processOffice referral form ( includes poss. Motivation)Other tracking formsTime during staff meetings to get agreement, learn about process and follow through all year!!
30B. Computer Application Easy, efficientNo more than 30 seconds to enterAble to generate reports quicklyAvailable in picture form (bar graphs)Custom ReportsThis is a tough one given that FCPS has SASI as a procedure. We do have more principals going to their own Excel files.You can emphasize that schools need to get their data entry person to SASI/PBS training.
31C. Data For Decision Making Generate reports for various meetingsAction PlanBuild Precision StatementsDetermine InterventionTrack Data, Continue, Modify, TerminateShare with FacultyCelebrate!!!!!
32Triangle of Student Referrals Intensive, Individual InterventionsIndividual StudentsAssessment-basedHigh Intensity6+ referrals1-5%1-5%Targeted Group InterventionsSome Students (at-risk)High EfficiencyRapid Response2-5 referrals5-10%5-10%Universal InterventionsAll StudentsPreventive, proactive80-90%80-90%0-1 referral
33Triangle of Student Referrals: August/September 2005
34Michigan: Distribution of Elementary Reading Intervention Level 24%56%33%24%43%20%(n = 4074)(n = 201)Reading Intervention Level (based on DIBELS)Dr. Steve Goodman
35Using Data within PBIS pbssurveys.org Use data to assess current statusEBS Self-assessment SurveyUse data to assess implementation fidelityTeam Implementation Checklist (TIC)School-wide Evaluation Tool (SET)Benchmarks of Quality (BoQ)Use data to assess impact on studentsOffice Discipline referralspbssurveys.org
36Use Data for Decision-making “We are all continually faced with a series of great opportunities, brilliantly disguised as insoluble problems.”John Gardner
37Main IdeasDecisions are more likely to be effective and efficient when they are based on data.The quality of decision-making depends most on the first step (defining the problem to be solved)Define problems with precision and clarity
38Main IdeasData help us ask the right questions…they do not provide the answers: Use data toIdentify problemsRefine problemsDefine the questions that lead to solutionsData help place the “problem” in the context rather than in the students.
39Main Ideas The process a team uses to “problem solve” is important: Roles:Facilitator; Recorder; Data analyst; Active memberOrganizationAgenda; Old business (did we do what we said we would do); New business; Action plan for decisions.What happens BEFORE a meetingWhat happens DURING a meetingWhat happen AFTER a meetingAgenda, data summary, positive nagUpdates, identify problem, problem solveMinutes posted, tasks completed
40Decision-making at many levels Whole schoolSmall groups or school areasIndividual studentSame basic processCarolSWIS
42Using Data Use data to identify a possible problem? Use data to build a precise “problem statement?Use data to select a solution (intervention)Use data to assess if a solution is (a) being implemented, and (b) being effective.
43Use data within “decision systems” not “data systems” QuestionDecisionData SourceIs there a problem?Primary/Precise problem statement.Should we allocate resources to this problem?Compare “observed” with “expected” (ODRs, ORF)Is there a reasonable solution?Do we have a reasonable intervention, and should it be implemented?Technical adequacyContextual FitIs the intervention being implemented as planned?Do we need more resources to implement the intervention?On-going data related to fidelity of implementationIs the intervention proving effective?Should we continue the intervention, modify it, or terminate it?On-going data related to impact of the intervention on student behavior
44Using Data Use data in “decision layers” Don’t drown in the data It’s “OK” to be doing well
45Six things to avoid Define a solution before defining the problem Build solutions from broadly defined, or fuzzy problem statementsFailure to use data to confirm/define problemAgree on a solution without building a plan for how to implement or evaluate the solutionAgree on a solution but never assess if the solution was implementedSerial problem solving without decisions
46Using Data for Decision-making at the Whole School Level Identifying “problems” with “precision”Well defined problems prompt functional solutionsPoorly defined problems prompt discussions in which the problem is admired, but not addressed.“Well begun is half done”Aristotle, quoting a proverb
47Precise Problem Statements (What are the data we need for a decision?) Solvable problem statements include information about the five core “W” questions.What is problem, and how often is it happeningWhere is it happeningWho is engaged in the behaviorWhen the problem is most likelyWhy the problem is sustaining
48“Everything is vague to a degree you do not realize till you have tried to make it precise.” Bertrand Russell
49Primary versus Precision Statements Primary StatementsToo many referralsSeptember has more suspensions than last yearGang behavior is increasingThe cafeteria is out of controlStudent disrespect is out of controlPrecision StatementsThere are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
50Primary versus Precision Statements Primary StatementsToo many referralsSeptember has more suspensions than last yearGang behavior is increasingThe cafeteria is out of controlStudent disrespect is out of controlPrecision StatementsThere are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
51Precise or Primary Statement? Children are using inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the schoolJames D. is hitting others in the cafeteria during lunch, and his hitting is maintained by peer attention.
52Precise or Primary Statement? ODRs during December are higher than in any other month.Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by escape from work (but may also be maintained by peer attention… we are not sure).
53Precise or Primary Statement? Three 5th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention and possibly unsophisticated sexual expression.Boys are engaging in sexual harassment
54Organizing Data for Decision-making Compare data across timeMoving from counts to count/month
55Using DataIdentifying a possible problem: A difference between what you want and what you have.What data to review?Office discipline referrals per day per monthAvoid simple countsAvoid one data point (look at trends)How to use dataCompare with national standardsCompare with local standardsCompare with prior experience
56Total Office Discipline Referrals as of January 10
57Average Office Discipline Referrals per day per month as of January 10
58SWIS summary 07-08 (Majors Only) 2,532 schools; 1,300,140 students; 1,139,119 ODRs Grade RangeNumber of SchoolsMean Enrollment per schoolMean ODRs per 100 per school dayK-61762444.35 (sd=.46)(1 /300 / day)6-9482653.92 (sd=1.42)(1/ 110 / day)9-121769141.06 (sd=.1.57)(1/105 / day)K-(8-12)3124011.00 (sd=.1.86)(1/ 155 / day
59Interpreting Office Referral Data: Is there a problem? Absolute level (depending on size of school)Middle, High Schools (> 1 per day per 100)Elementary Schools (>1 per day per 300)TrendsPeaks before breaks?Gradual increasing trend across year?Compare levels to last yearImprovement?
60Application Activity: Absolute Value Is there a Problem?Middle School of 625 students?Compare with national average: /100 = X .92 = 5.75Office Discipline Referrals per School Day
61Compare with National Average 150 / 100 = 1.50 1.50 X .35 = .53 Elementary School with 150 Students
62Compare with National Average 1800 / 100 = 18 18 X 1.06 = 19.08 High School of 1800 students
70What are the data you are most likely to need to move from a Primary to a Precise statement? What problem behaviors are most common?ODR per Problem BehaviorWhere are problem behaviors most likely?ODR per LocationWhen are problem behaviors most likely?ODR per time of dayWho is engaged in problem behavior?ODR per studentWhy are problem behaviors sustaining?No graph
71What other data may you want? ODR by staffODR by IEPODR by gradeODR by gender by grade
72Test precision problem statement Use precision problem statement to build and test hypotheses.Problems are most common in D-Hall wingProblems are more likely during second recessProblems are most common during assembly scheduleProblems are more likely during state testing periods
82Moving to fine-grained analysis SWIS Custom Reports
83Describe the challenges Neal appears to be experiencing? Data Student Staff Time Location Behavior09/20/2007Neal Anderson4Dean Smith10:15AMPlygdDisresptUnknownNoneConf209/22/200710:00AM309/23/2007StaffLoss priv09/25/2007Ob a attn509/30/2007610/02/2007710/07/2007In-sch susp810/09/2007Out-sch susp911/03/2007David Anderson-JonesJason ClineClassM-Contact1001/05/2008Dale Cocker10:30AMPeersOffice
84Describe the challenges we are experiencing in the Parking lot Date Student Staff Location Behavior8Debbie Calhoun3:15:PMPark lotAgg/FightOb p attnPeersOut-sch susp11904/01/2008Mark BanksDale Cocker8:00:AMTobaccoIn-sch susp12003/13/2008Brian Bender7Sally Post3:00:PMM-DisresptDKParent12104/21/2008Dottie DennerBrenda FrankenDressStaff12201/27/2008Joe FranklinFrannie James12302/01/2008Carol Earley3:30:PMVandalOb a attn12401/12/2008Samual Fullerton12510/30/2007Bruce GilStarla Paulson2:15:PMSkipAvoid a12603/08/2008Willie LomanAnne Harrison
85A Context for Designing Solutions Behavior support is the redesign of environments, not the redesign of individualsAttend as much, or more, to what happens between problem behavior bouts as what happens during instances of problem behavior.(Edward Carr)Positive Behavior Support plans define changes in the behavior of those who will implement the plan.
87Building Solutions Packages versus Practices Many good ideas, packages and procedures existMost are expensive to implement, and not a perfect fit with your problemBe efficient as well as effectiveMost good interventions will be multi-componentBe wary of the one-trick strategy.
88Using Data to Build Solutions: Four Elements to Consider Prevention: How can we avoid the problem context?Who, When, WhereSchedule change, curriculum change, etcTeaching: How can we define, teach, and monitor what we want?Teach appropriate behaviorUse problem behavior as negative exampleRecognition: How can we build in systematic reward for desired behavior?Extinction: How can we prevent problem behavior from being rewarded?Consequences: What are efficient, consistent consequences for problem behavior?Monitoring: How will we collect and use data to evaluate (a) implementation fidelity, and (b) impact on student outcomes?
89Solution Development Prevention Teaching Reward Extinction Corrective ConsequenceData Collection
90Problem solving Mantra Do we have a problem?(Identify the primary problem)What is the precise nature of the problem?(Define, clarify, confirm/disconfirm inferences)Why does the problem exist, & what should we do about it?(Hypothesis & solution)What are the actual elements of our plan?(Action Plan… what will we do, who will do it, when it get done)Is our plan being implemented?(Evaluate & revise plan)Is the plan having the desired effect?(Maintain, modify or terminate the plan)
91Quick example of the five step process with East Elementary 500 studentsK-5
92Step one: review status and identify problems Rose ElementaryEnrollment: 500 studentsGrades: K – 5National Mean Level of ODRs for Elementary Schools is .35 per 100 students per day, or 1.75 for a school of 500 (downloaded from Aug 30, 2007)
93Rose Elementary Problem Identified For all but one month during last year ( ), the rate of ODRs per day exceeded the national average reported by elementary schools across the United StatesThe data show a minimal trend across months, but there are noticeable increases in the ODR level in December and March.For each month of the last school year, the level of ODRs per day exceeded the level from the corresponding month during the prior school year ( )Teachers, families, and students themselves have reported that student problem behavior is unacceptable and presents a barrier to effective instruction.
97Rose Elementary Hypothesis Statement A large proportion of students are engaging in disruption & aggression on the playground during recess becauseWe have not developed playground specific expectations and taught them to studentsPlayground supervisors have not been included as participants in the planning, teaching and evaluationDisruption and aggression are resulting in access to peer attention and time with preferred equipment.
98Step 3: Discuss and Select Solutions Rose Elementary ExamplePrevent problem behavior situationEnsure that supervisors are on the playground and are engaged in active supervision .Teach appropriate behaviorTeach the school-wide behavior expectations of being safe, respectful, and responsible, and do the teaching on the playground where problem behaviors are most likely.Reward appropriate behaviorProvide a formal system for playground supervisors to recognize appropriate play on the playground.Reduce reward for problem behaviorTeach all students to signal “stop” when they are treated disrespectfully.Teach playground supervisors to ensure that aggression and disruption are not allowed to gain access to preferred activities or materials.Deliver corrective consequences for problem behaviorReview continuum of consequences for problem behavior on playground with students and supervisors and make sure continuum is in effect.Collect data to assess if the intervention (a) is implemented with fidelity and (b) produces desired impact on student behavior.Collect, summarize and report data.
99Designing SolutionsIf many students are making the same mistake it typically is the system that needs to change not the students.Teach, monitor and reward before relying on punishment.An example (hallways)