Portfolio & Assessment Tool Contents, Guidelines & Rating KIPBS Module 10 Kelcey Schmitz

Background Feedback from students, facilitators, parents, school personnel, direct service staff and other stakeholders regarding written plan indicated there was a need for change PC-PBS checklist reviewed by current KIPBS facilitators and items were removed (based on consensus) to reduce size of checklist Checklist went from 68 items to 45 items New checklist with 45 items were organized into 6 categories 6 Categories became “sections” of Portfolio Portfolio Assessment Tool adapted from PC-PBS checklist that would serve as portfolio fidelity checklist Draft example of hard copy portfolio created from written plan A portfolio committee was created to develop content, instructions and documents related to the portfolio

Portfolio vs. Written Plan Students, facilitators and team members report that written plan is time consuming/cumbersome to write and read. Portfolio is an organized file of the documentation gathered/created throughout the PBS process Pieces of the portfolio can be easily shared as needed without the entire written plan Should be less time consuming than written plan. Can be organized in a three ring binder, accordion folder or even on a CD or flash drive Facilitators could have an option of either completing the portfolio or written plan since both are evaluated with the same checklist items

Instructions on Mandatory First PA Portfolio Process Once case study has been approved through the prior authorization process a KIPBS Instructor will be assigned to work with you. You must complete the Mandatory Case Study checklists with approximate dates and submit it to the assigned Instructor. As you reach “stopping points” on the Mandatory Case Study checklist, an Instructor will meet with you and review progress and provide feedback.

© 2008 Freeman, et al. Mandatory 1 st PA Checklist

Excerpts from Mandatory 1 st PA Checklist Consumer’s Name: Date Completed KIPBS Sign off Activity Contact KIPBS Evaluation Coordinator to discuss timeline for evaluation Schedule an initial meeting with caretakers o Introduce yourself, state why you are there, and describe the process o Obtain informed consent to videotape o Find out what their major concerns are with respect to the individual o Obtain previous plans, if there were any * If the individual spends a significant amount of the time in a different setting (e.g. school, work), schedule a meeting with the people in that setting and obtain the same information Schedule and conduct meetings and observations Conduct enough meetings and observations for you to assess and get to know the individual, his/her environment, and his/her team

Schedule and conduct PCP interviews with caretakers and team members Conduct the necessary interviews to obtain information for the PCP meeting. Information gathered should include the individual’s history (medical and behavioral), strengths, preferred lifestyle, and possible challenges Schedule the PCP meeting Contact the KIPBS Evaluation Coordinator to schedule observation and evaluation of PCP meeting [Onsite Evaluation Observation or alternative] Contact the KIPBS staff person/Instructor-Mentor who will be doing your onsite evaluation observation, to discuss arrangements & details

Prepare for PCP meeting (selected steps excerpted) Some goals will involve environmental-type of changes, while other goals will involve behavioral-type of changes (to avoid the negative focus, you may approach the issue by discussing appropriate behaviors to increase or skills to teach). Since environmental-type of changes will likely impact the behavioral issues, it is a wise idea not to make these changes until functional assessment activities and baseline measures on the target behaviors are gathered. However, in the meantime, the team can work towards these goals by gathering information and resources so that once baseline measures are completed, preferred lifestyle interventions can be started. Conduct PCP meeting [Onsite Evaluation Observation or pre-arranged alternative]. At the end of the meeting (or at the next meeting where all team members are present) distribute/conduct the PCP Satisfaction Survey Schedule and conduct Functional Assessment Interviews with caretakers and team members. Obtain information about behavioral issues. Establish what the major behavioral issues may be. Examine the Functional Assessment Interviews, as well as any other information you may have obtained from discussions, previous plans, records reviews, and observations, to establish what the major behavioral issues may be and what you should focus on and observe for in your assessment

Additional Excerpts from Mandatory Checklist… Schedule and conduct observations (e.g., Functional Behavior Assessment Observation form, ABC Chart, Scatter Plot) Conduct enough observations (3-5) to have a good idea of what the problem behaviors are, when they are more/less likely to occur, and what their function might be. Develop a sound hypothesis. Note: YOU should conduct these observations. Caretakers may gather some data if behaviors occur when you cannot observe, but you should conduct the majority of observations so that you have a good sense of what is going on Develop a behavioral definition for each target behavior and determine how each will be measured once you have a good sense of what these behaviors look like (from your observations). Send your Operational Definitions to the KIPBS Evaluation Coordinator for review/evaluation. If we find the descriptions/definitions of the behaviors acceptable, videotaping will not be necessary; however if there needs to be more clarification you may be asked to submit video clips or have a mentor come observe. Develop a data collection sheet to measure all target behaviors (including replacement behaviors) Send your Data Collection Sheet to the KIPBS Evaluation Coordinator for review/evaluation.

Prepare any additional Miscellaneous materials (section 6), as needed/ as appropriate, for your Portfolio. Contact KIPBS Evaluation Coordinator to arrange timeline for submitting your first PA case portfolio and evaluating it, using the shortened version of the PC-PBS Checklist. Portfolio must score 80% or higher on shortened (45 item) checklist with all “yes” responses to the five Critical Features in order for KIPBS Facilitator to be approved for additional PA cases. From this point on, you should: - Obtain data collected from caretakers on a regular basis, and closely monitor behaviors - Conduct fidelity and reliability observations periodically - Hold regular follow-up meetings to go over data and discuss the status of behaviors and goals with caretakers o If an intervention does not seem to have the intended effect and enough time has been allowed for change, discuss modifying or changing the intervention strategy o If PCP goals are not being met, problem-solve the issues o As PCP goals are met, update the plan accordingly o As behavioral goals are met, modify the procedures gradually to get closer and closer to the “regular” environment (i.e. gradually fade prompts, gradually thin reinforcement) o Plan for sustainability and major changes o Distribute/conduct the Quality of Life survey at regular intervals

Portfolio Assessment Tool Items were categorized from the 45 item checklist into six main sections –Identifying information –Assessment –PCP –Interventions/Supports –Data –Plan Evaluation

Portfolio Assessment Tool is multi-functional 1.Section Contents 2.Item locator 3.Self Assessment Checklist 4.Plan Evaluator Checklist 5.Self Assessment Score Sheet 6.Plan Evaluator Score sheet 7.Reliability

Portfolio Sections Section 1: Intake/Referral Section 2: Assessment Section 3: Person Centered Planning Section 4: Interventions/Supports Section 5: Data (raw/graphed) Section 6: PBS Plan Evaluation

Section 1: Intake/Referral Identifying info –facilitator name, consumer name, address, DOB, age, contacts, referral source) A brief history of the consumer’s life is provided Behavioral and/or environmental issues are identified Important places for the consumer at school/work, home, and in the community are described Opportunities for the consumer to interact with friends and/or family are described Current schedules and routines are described (quality, predictability)

Section 2: Assessment Indirect assessment data include at least 2 –Caretaker interviews –Record reviews –Preferred lifestyle info –Assessment tools to collect QOL setting events and other related info Data from 3-5 student conducted direct observations are described Problem behaviors are operationally defined; definitions are clear Hypothesis statement is provided for each function of the problem behavior

Section 3: PCP –The consumer’s strengths are described –Opportunities for choice in the consumer’s current environment are described –Current health and physiology issues are described –Planning and assessment participants are listed –A global statement of the consumer’s dream is made –Type of preferred living setting for the consumer is described –With whom the consumer wants to socialize is clearly stated –Social leisure or religious activities the consumer wants to participate din are described –Goals or skills to be achieved are described –Plan outlines how achievement of goals or skills will be assessed

Section 4: Interventions & Supports –Goals or skills to be achieved are described –Plan outlines how achievement of goals or skills will be assessed (yellow) –Possible function of problem behavior is included –Teaching of adaptive skills as a replacement behavior is included –Replacement behaviors are operationally defined –Each part of the hypothesis statement is addressed –Environmental interventions address at least 3 of the following Opportunities for choice Schedule predictability Instructional interaction approaches Prompts for appropriate behavior School or day program curriculum –Possible function of problem behavior is addressed –Teaching of adaptive skills as a replacement behavior is included –Replacement behaviors are operationally defined –Each part of the hypothesis statement is addressed (Setting event, antecedent, behavior, consequence)

Section 4 cont’d –Environmental interventions address at least 3 of the following –Minimizing positive and/or negative reinforcement for problem behavior is included –Selecting effective reinforcers and/or maximizing positive reinforcement for appropriate behavior is included –Safety emergency procedures for what to do it/when crisis occurs is addressed –Training needs are id and or system of support established (yellow) –Process for monitoring the intervention plan is described Timelines for meetings What needs to be done When By whom –Each part of the hypothesis statement is addressed –Plan for sustainability includes Plan for transitions or major setting events and/or Training plan for new staff –Sample fidelity checklist –KIPBS Intervention and Supports Plan At a glance sheet (not need to be labeled “KIPBS)

Section 5: Data Baseline data are clearly graphed (include labels, axis values, titles, and legends) Data to support each hypothesis statement are presented Baseline and intervention data for each target behavior are graphed Graphs are clear (include labels, axis values, titles, and legend) Indirect or direct measures of replacement behavior are provided Evaluation of achievement of goals or skills includes (at least 1) –Number of goals of skills achieved –Other measures of achievement of goals or skills

Section 6: Plan Evaluation Contextual Fit Quality of Life PCP Satisfaction

P.A.T. Score Sheet Self Assessment- first Submission –Self Assessment Re-submission after feedback (if needed) Plan Rater: first Submission –Plan Rater: Re-submission after feedback (if needed) Reliability- first Submission –Reliability Re-Submission (if needed) *Sections scored include Critical Features, Section 1, Section 2, Section 3, Section 4, Section 5 and Section 6

Discussion