Presentation on theme: "Improving Practice to Wraparound Through Use of Fidelity Measures"— Presentation transcript:
1Improving Practice to Wraparound Through Use of Fidelity Measures Enrica Bertoldo, Quality Support ManagerEleanor Castillo, Director of Outcomes & Quality AssuranceMary Ann Wong, Research SpecialistVeronica Padilla, Manager of Outcomes & Evaluations26th Annual California Mental Health Advocates for Children & Youth Conference, Asilomar, CaliforniaMay 4, 2006
2EMQ Children & Family Services Our MissionTo work with children and their families to transform their lives, build emotional, social and familial well-being, and to transform the systems that serve them.Our PhilosophyFamily Voice Team BasedCollaborative/Integrative Community-BasedCulturally Competent IndividualizedStrength-Based Natural SupportsPersistence Outcomes-BasedBackground on EMQEMQ Children & Family Services (a.k.a. Eastfield Ming Quong) is a leading provider of children's mental health, Wraparound, and social services in No. & So. California.. Headquartered in Campbell, California, EMQ offers an integrated network of services that range from community-based services such as Wraparound to residential treatment, support and educational services, and foster placements. Our philosophy incorporates many of the elements and values seen in wraparound into our other programs. We are recognized for innovation in family-centered, strengths-based programs for children and adolescents. We have wraparound in 3 sites…Community Based (Wraparound, Matrix, SOC): Wraparound Services in 3 sites: Santa Clara, Sacramento, and San BernadinoResidential Treatment (Long-term, Short-term)Support Services (Outpatient, Giaretto)School-based (Day Treatment Program)Addiction Prevention Services (School-based and Agency-Based Prevention Programs)24 Hour Mobile CrisisFoster Family Agency
3PurposeGiven the complexity of Wraparound, existing fidelity measures are utilized to improve practice.This workshop will share successes and challenges in the implementation of fidelity measures (WFI-3.0 & WOF-2.0) that affect practice and supervision to Wraparound.A pilot in utilizing a Wraparound supervisory adherence measure (W-SAM) will also be discussed.
4What is Wraparound?Team-based, individualized service planning processPrinciples of community-based, strength-based, family-centered, and culturally competentWraparound is described as a promising practice (Burns, Hoagwood, & Maultsby, 1998)
511 Core Elements of Wraparound Voice and ChoiceYouth/Family TeamCommunity-Based Services/SupportsCultural CompetenceIndividualized Services/SupportsStrengths-Based Services/SupportsNatural Supports8. Continuation of Services/SupportsCollaborationFlexible Resources/ FundingOutcome-Based Services/Supports
6Essential Elements of Wraparound Element 1: Voice and ChoiceFamilies must be full and active partners at every level of the wraparound process. If the team cannot reach consensus, the final decision should be up to the caregiver.Element 2: Youth and Family TeamWraparound is a team-driven process involving caregivers, youth, natural supports, and community services working together to develop, implement, and evaluate the individualized plan.Element 3: Community-based Services and SupportsServices and supports that the youth and family receive should be based in their community. The family should not have to leave their community if more restrictive services are necessary.Element 4: Cultural CompetenceThe team should not only be respectful of the family’s beliefs and traditions, but also actively seek to under-stand the family’s unique perspectives and convey them to others.
7Element 5: Individualized Services This means that services and supports are tailored to the unique situation, strengths, and needs of teach individual, and may involve existing categorical services and informal sup-ports; modifying existing services and supports; and or creating new services and supports. Further, the team should create a specific plan to meet the family’s goals and crisis/ safety plan to manage potential emergencies.Element 6: Strengths-Based ServicesThe focus of the team should be on what is working and going well for the family. While goals may be drawn up based on the family’s needs, the plan should capitalize on the family’s positive abilities and characteristics.Element 7: Natural SupportsServices and supports should reflect a balance of formal and informal community and family supports rather than a reliance on formal professional services.Element 8: Continuation of CareServices and supports must be provided unconditionally. In a crisis, services and supports should be added rather than placing the youth with a new provider.
8Element 9: Collaboration The team should coordinate services and supports so they seem seamless to the family rather than disjointed.Element 10: Flexible Funding and ResourcesSuccessful wraparound teams are creative in their approach to service delivery and have access to flexible funds and resources to implement their ideas.Element 11: Outcome-Based ServicesSpecific, measurable out-comes should be monitored to assess the youth and family’s progress toward goals.The description of each element is taken directly from Wraparound Fidelity Index 3.0 (2002), produced by the Wraparound Evaluation and Research Team at the University of Vermont.
9What is Fidelity to Wraparound? Treatment Fidelity = The degree to which a program is implemented as intended (Rast & Bruns, 2003; Moncher & Prinz, 1991)Adherence to the 11 Core Wraparound Elements (e.g., WFI-3.0; Suter et al., 2002)Adherence to the Child and Family Team Process (e.g., WOF-2.0; Epstein et al., 2002)
10Why Measure Fidelity?Measuring fidelity is essential to families, providers, policy makers and researchersWithout measuring fidelity, how do you ensure the Wraparound process is occurring?Without outcomes, the Wraparound process is just one more fadHigh fidelity has been associated with positive outcomes in some studiesIt improves quality assuranceIt helps agencies secure more funding by proving outcomesIt can even help create legislation on how families and kids receive services
11More Why’s…..All nationally recognized Wraparound programs have extensive fidelity measurements in place.Why?Without fidelity measurement standards firmly in place, Wraparound programs and team members have difficulty knowing what they are doing well and what they need to improve in order to achieve the ideal as presented in the Wraparound vision.
12Fidelity Implementation Measures Wraparound Fidelity Index-3.0 (WFI-3.0)Measures fidelity to Wraparound principles through structured interviewsAdministered to Youth, Caregiver, & Resource FacilitatorHigher Score = Greater FidelityWraparound Observation Form-2.0 (WOF-2.0)Measures fidelity to Wraparound through observation of the Child and Family Team meetings
13Fidelity to Wraparound EMQ Sacramento has collected the Wraparound Fidelity Index (WFI) since 2001Data from WFI-3.0: January 2003 – September 2005EMQ Sacramento has collected a fidelity measure since 2001, beginning with the WFI 2.0 and then the WFI 3.0. Our measure of fidelity in this study was the WFI Version 3. We conduct WFI Interviews with the Wraparound resource facilitator, caregiver, and youth six months after admission and every six months thereafter for each youth until discharge.(The responses from each interview results in a total fidelity score, ranging from 0 to 8, with 0 indicating low adherence to the Wraparound philosophy and 8 indicating high fidelity to the Wraparound philosophy. An overall fidelity score is also calculated by combining the reports of the three respondents)ParticipantsParticipants in this study included youth (N=146), caregivers (N=124), and resource facilitators (N=183) for families receiving Wraparound services by a large family service agency in the Sacramento region.
15WFI Total Scores by Respondent: EMQ & National Data *National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the 18th Annual Research Conference on Systems of Care and Children's Mental Health in Tampa on 3/7/05 by Eric J. Bruns, Ph.D. for "Is it Wraparound Yet?" - Bootstrapping wraparound fidelity standards using the WFI .
16WFI Elements: EMQ & National Data *National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the Technical Assistance Partnership Webinar on 4/5/04 by Eric J. Bruns, Ph.D. for "Ensuring High-Quality Wraparound"
17WFI Elements: EMQ & National Data *National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the Technical Assistance Partnership Webinar on 4/5/04 by Eric J. Bruns, Ph.D. for "Ensuring High-Quality Wraparound"
19The Relationship Between Fidelity & Outcomes Previous research has found an association between greater Wraparound fidelity and better child and family outcomes (Bruns, 2004), but the relationship has not been clearly understoodEMQ (2005) completed a study to further understand the relationship between fidelity to the 11 core elements of Wraparound and treatment outcomesOver the past couple of years, EMQ has been increasingly interested in examining the relationship between fidelity and outcomes. Previous studies have used the WFI to demonstrate a relationship between fidelity and outcomes (Bruns, 2004, presentation given at 11th Annual Building on Family Strengths Conference) but the exact nature of the relationship has not been clear.(Past studies have shown WFI Scores to be associated with BERS, CAFAS, and CBCL concurrent and future outcomes) The purpose of this study was to further understand the relationship between fidelity to the 11 core elements of wraparound and treatment outcomes. Specifically, we were interested in knowing what elements of the wraparound process, if any, are associated with outcomes.
20Demographics Average Age at Admission: 14 years 63% Male; 37% Female 62% Caucasian; 26% African-American; 8% Latino; 2% Asian/Pacific Islander; 1% Native-American; 1% OtherAverage Length of Stay: 15 monthsThe demographics for youth in this sample are as follows:The average age of these youth at the time of admission was 14 years (SD=2.55). Gender composition was 63% male and 37% female. Ethnicity was 62% Caucasian, 26% African-American, 8% Latino, 2% Asian/Pacific Islander, 1% Native-American, and 1% other. The average length of participation in Wraparound services was 15 months (SD=11, range months).
21Fidelity to Wraparound Implementation Measure:Wraparound Fidelity Index – 3.0 (WFI-3.0)Collection since 2003Sample in study included the following:146 WFI’s collected from Youth124 WFI’s collected from Caregivers183 WFI’s collected from Resource FacilitatorsEMQ Sacramento has collected a fidelity measure since 2001, beginning with the WFI 2.0 and then the WFI 3.0. Our measure of fidelity in this study was the WFI Version 3. We conduct WFI Interviews with the Wraparound resource facilitator, caregiver, and youth six months after admission and every six months thereafter for each youth until discharge.(The responses from each interview results in a total fidelity score, ranging from 0 to 8, with 0 indicating low adherence to the Wraparound philosophy and 8 indicating high fidelity to the Wraparound philosophy. An overall fidelity score is also calculated by combining the reports of the three respondents)ParticipantsParticipants in this study included youth (N=146), caregivers (N=124), and resource facilitators (N=183) for families receiving Wraparound services by a large family service agency in the Sacramento region.
22Outcome Measures at Discharge Child and Adolescent Functional Assessment Scale(CAFAS; Hodges, 2000)Measures youth’s level of functioning; completed by Resource FacilitatorLower Score = Greater FunctioningExit Total CAFAS ScoreChild Behavior Checklist (CBCL; Achenbach, 2001)Measures youth’s level of behavior problems; administered to CaregiverLower Score = Lower Behavior ProblemsExit Total, Externalizing, & Internalizing CBCL ScoresLiving Arrangement at Discharge: Community or FacilityWe routinely collect the CAFAS & CBCL at admission, every 6 months, and discharge. For this study we looked at CAFAS Total score at discharge, the CBCL Total, Externalizing, and Internalizing scores at discharge. In addition, we examined the youth’s living arrangement at discharge, living in the community or facility. We define community as living with bio/adoptive parents, kinship care, foster family, or independent living program.(The Child and Adolescent Functioning Assessment Scales (CAFAS; Hodges, 2000) measure a youth’s emotional and behavioral functioning across eight domains (School/Work Role Performance, Home Role Performance, Community Role Performance, Behavior Toward Others, Moods/Emotions, Self-Harmful Behavior, Substance Use, and Thinking). Subscale scores on these dimensions are summed to produce a total CAFAS score, with a higher value indicating more severe impairment.(The Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) measures a youth’s competence and problem behaviors and is administered to caregivers. Scoring produces a Total Problems score as well as raw scores within two groupings of syndromes (Internalizing and Externalizing Problems) A higher value indicates more severe impairment.
23Correlations Among WFI Scores and Outcomes at Discharge CBCLTotalCBCL ExternalizingCBCL InternalizingCAFASLiving in Community1. Voice and Choice.12.07.15*.472. Youth and Family Team-.04.01.103. Community-Based Services-.19*-.25**-.21*-.40**.21**4. Cultural Competence.06.02.155. Individualized Services.09.11.056. Strengths-Based-.10-.09-.14.087. Natural Supports-.12-.13-.07.138. Continuation of Services-.06-.029. Collaboration-.03-.17*-.1110. Flexible Resources/Funds.04.20**11. Outcome-Based Services-.16-.18*.14*12. RF Total Fidelity-.20-.08-.15.1613. Caregiver Total Fidelity14. Youth Total Fidelity-.01-.0515. WFI Total FidelityNon-parametric correlations (Spearman’s rho) were conducted to demonstrate relationships between WFI total fidelity and element scores and outcome measures at discharge. The average score of each element across the respondents was calculated for the analysis (youth does not have elements 9, 10, 11). The analysis also factored in the pre-test scores on the outcome measures to account for the differences in the pre and post scores on youth functioning.Significant Correlations:Between fidelity to Community-Based Services and Supports and various outcome measures:CBCL Total Problem Score at Discharge ( r = -.189, p<.05) N=143)CBCL Externalizing Problem Score at Discharge, (r = -.254, p<.01 N=143)CBCL Internalizing Problem Score at Discharge (r = -.210, p<.05 N=143),CAFAS Total Score at Discharge (r = -.40, p<.01 N=171)Living in Community (r = .21, p<.01 N=?)Other significant relationships occurred between Natural Supports & CBCL Total (r=-.19, p<.05 N=143); Collaboration & CBCL Internalizing (r = -.17, p<.05 N=142); Outcome-Based Services & CBCL Internalizing (r = -.18, p<.05 N=141) & CAFAS (r=.14, p< .05, N=?) & Living in Community (r=.21, p<.01 N=?) ; Flex Funds & Living in Community (r=.20, p<.01 N=?); Voice-Choice & CAFAS (r=.15, p<.05, N=?)The difference in the pre and post scores on youth functioning did not impact the relationship with fidelity.Note: *p<.05; **p<.01
24Community-Based Services Significant ResultsCommunity-Based ServicesCBCL Total-.19*CBCL Externalizing-.25**CBCL Internalizing-.21*CAFAS Total-.40**Living in Community.21**Note: *p<.05; **p<.01Greater community-based services is related to lower impairment on the CBCL and CAFAS at dischargeGreater community-based services is related to community living arrangement at dischargeAs you can see from the table, community-based services and supports had the most consistent relationship to outcomes. Significant relationships were found with all 5 outcomes measures used in this study….Means:CBCL Total= (SD=19.29, N=242)CBCL Externalizing=87.16 (SD= 19.71, N=242)CBCL Internalizing= (SD=25.20, N=242)CAFAS(7)=94.78 (SD=43.78, N=324)WFI Total=5.81 (SD=.82, N=186)RF Total WFI =6.30 (SD=.78, N=183)CG Total WFI=5.39 (SD=1.21, N=124)Youth Total WFI= (SD=1.22, N=146)Elements are the average across 3 respondentsCommunity-Based=5.00 (SD=1.72, N=186) Flexible=5.2 (SD=1.32, N=184)Voice/Choice=6.66 (SD=1.16, N=184) Outcomes=6.83 (SD=1.37, N=184)CFT=4.5 (SD=1.26, N=186)Cultural Com=6.64 (SD=1.04, N=186)Individualized=6.85 (SD=1.28, N=186)Strengths=6.43 (SD=1.14, N=186)Natural Supports=3.78 (SD=1.52, N=186)Continuation=6.04 (SD=1.30, N=186)Collaboration=6.10 (SD=1.34, N=185)
25ResultsGreater fidelity to the provision of Community Services/Supports in Wraparound is related to positive outcomesNo other elements had as significant correlations to outcomes as Community Services/SupportsNo significant relationship found between Total Fidelity Scores and outcomes at dischargeCorrelations were consistently low even when significantIn summary, greater fidelity to community-based services to wraparound is related to positive outcomes. No other elemnts had as significant or consistent relationships to outcomes. Unlike previous studies, no relationship was found between total fidelity scores and outcomes at discharge. Other significant correlations were found but not as consistent as Community-based services and supports. Significant findings with use of flex funds and focus on outcome-based services with the community living arrangement at discharge.
26ImplicationsImplications for teams to support a youth’s school attendance and involvement in work, training, and other community activities (e.g., church, sports, art, etc.) as well as support community-based living situations for youth (as an alternative to residential or institutional care) in order to improve outcomes.Implications for teams to consider the questions that comprise Element 3 of the WFI- Community-Based Services and SupportsHow many hours a week does the youth spend at a regular community school, working at a paying job, in a job training program?Are services and supports for the family accessible?Does the team help the youth get involved with activities in the family’s community? (Give 2 examples)Different placements for youth over past 30 days?This study’s finding that effective Wraparound services focus on the enhancement of community adjustment is consistent with the results of a study of system reform performed by Hyde, Burchard, and Woodworth (1996). These investigators assessed community adjustment and involvement of four groups of high-risk youth, two of which received Wraparound services. Positive findings revealed that youth in the Wraparound groups displayed higher levels of adjustment to their communities (as evidenced by living in the community, attending school, working, engaging in relatively low rates of self-harmful behavior). Furthermore, a higher percentage of these youth participated in community activities, such as church, sporting events, and volunteer work.
27Challenges of the WFI Direct feedback to teams was not provided The utility of the WFI in its relationship to outcomes was limited based on EMQ’s analysis of the dataTime intensive and limited resources available to conduct 3 structured interviews per youth
29Initiative to Improve Practice Quality Improvement Project launched in January 2004 to implement practice standards and feedback tools across positionsFamily FacilitatorFamily SpecialistFamily PartnerGoal was to standardize job positions, provide role clarity, and improve consistency in job performance and supervisionUse of the WOF-2.0 to observe the CFT processDevelopment of practice standards for facilitators and other team members, including supervisors, will look at adherence to fidelity to wraparound and outcomes.
30Example of EMQ Feedback Tool: Family Facilitator Scoring:Opportunity for Improvement (1)Acceptable (2)Exemplary (3)Not Observed (*)3 Dimensions:A. Families receive services that are consistent with wraparound valuesB. Families receive services that are collaborative, integrated and adhere to best practicesC. Children and Families served are safe and stable at home and in the community
31Facilitator Field Feedback Tool A. Families receive services that are consistent with wraparound values1. Arrived on time and was prepared for the CFT2. Started the CFT with “what’s working” to maintain strength based focus and future orientation3. Facilitated the CFT to validate and incorporate ideas, requests, and concerns of all members4. Evidence that needs and strategies were reviewed within the CFT to assure the family voice and preference guide all activities5. Evidence that plans were reviewed for progress on current measurable, observable goals and updated as needed to address new and changing needs6. Evidence that the next CFT and staff visits were scheduled according to family preference
32Facilitator Field Feedback Tool B. Families receive services that are collaborative, integrated and adhere to best practices1. For children open six months or longer, evidence that the CFT was comprised of at least 50% non-paid informal members2. Facilitated the CFT to allow pertinent agenda items to be covered according to family preference and time availability3. Demonstrated engagement and rapport with all team members including system partners4. Therapy, medications and adjunctive services were evaluated relative to plan goals as neededSpecific tasks were assignedC. Children and Families served are safe and stable at home and in the community1. Evidence that team has a plan B to back up plan A2. If the CFT was for a child in out of home placement, this was addressed as a priority with specific plan, tasks, and target date3. Evidence that strategies and interventions developed emphasize the use of typical resources in the child’s community
33EMQ Feedback ToolsData analysis showed that the tools were not measuring what was intended; therefore, tools were discardedLed to an examination of the national standards for WraparoundEMQ Sacramento decided to collect data on the Child and Family Team process using the Wraparound Observation Form – Version 2.0 (WOF-2.0)
34Wraparound Observation Form – Second Version (WOF-2) Measure to assess the implementation of the Wraparound approach by direct observation of youth and family team meetingsWOF has demonstrated good inter-rater reliability (kappa=.886; Nordess & Epstein, 2003)48 items that measures 8 characteristics:Community-Based Unconditional CareIndividualized Measurable OutcomesFamily-Driven Management of Team MeetingInteragency Collaboration Care Coordinator
35Implementation of the WOF WOF data collection began July 2005Feedback provided in individual supervision if completed by direct supervisorVerbal feedback provided to facilitators after the child and family team meeting if completed by neutral observer. Copy of tools also provided to supervisors
36Wraparound Observation Form – Second Version (WOF-2) Community-based resources (5 items)1. Information about resources / interventions in the area is offered to the team.2. Plan of care includes at least one public and/or private community service/resource.3. Plan of care includes at least one informal resource.4. When residential placement is discussed, team chooses community placements for child(ren) rather than out-of-community placements, whenever possible.5. Individuals (non-professionals) important to the family are present at the meeting.
37Individualized services for the family (9 items) If an initial plan of care meeting, the parent is asked what treatments or interventions he/she felt worked/didn’t work prior to WRAP.Facilitator advocates for services and resources for the family (e.g., identifies and argues for necessary services).All services needed by family are included in plan (e.g., no needed services were not offered).Barriers to services or resources / interventions are identified and solutions discussed.The steps needed to implement the plan of care are clearly specified by the team.Strengths of family members are identified and discusses at the meeting.Plan of care that includes life domain(s) goals, objectives, and resources/interventions is discussed (or written).Plan of care goals, objectives, or interventions are based on family/child strengths.Safety plan/Crisis plan developed/reviewed.
38Family-driven services (10 items) 15. Convenient arrangements for family’s presence at meeting are made (e.g., location, time, transportation, day care arrangements).16. The parent/child is seated or invited to sit where he/she can be included in the discussion.17. Family members are treated in a courteous fashion at all times.18. The family’s perspective is presented to professionals from other agencies.19. The family is asked what goals they would like to work on.20. The parent is asked about the types of services or resources/interventions he/she would prefer for his/her family.Family members are involved in designing the plan of care.In the plan of care, the family and team members are assigned (or asked) tasks and responsibilities that promote the family’s independence (e.g., accessing resources on own, budgeting, maintaining housing).The team plans to keep the family intact or to reunite the family.Family Members voice agreement/disagreement with plan of care.
39Interagency collaboration (7 items) Staff from other agencies who care about or provide resources/interventions to the family are present at the meeting.Staff from other facilities or agencies (if present) have an opportunity to provide inputInformal supports (if present) have an opportunity to provide input.Problems that can develop in an interagency team (e.g., turf problems, challenges to authority) are not evident or are resolved.Staff from other agencies describe support resources interventions available in the community.Statement(s) made by a staff member or an informal support indicate that contact/ communication with another team member occurred between meetings.Availability of alternative funding sources is discussed before flexible funds are committed.
40Unconditional Care (3 items) Termination of Wrap services is discussed because of the multiplicity or severity of the child’s/family’s behaviors /problems.Termination of other services (not Wrap) is discussed because of the multiplicity or severity of the child’s/family’s behaviors /problems.For severe behavior challenges (e.g., gangs, drugs), discussion focuses on safety plans/crisis plans (e.g., service and staff to be provided) rather than termination.Measurable Outcomes (3 items)35. The plan of care goals are discussed in objective, measurable terms.36. The criteria for ending Wrap involvement are discussed.Objective or verifiable information on child and parent functioning is used as outcome data.
41Management of team meeting (5 items) 38. Key participants are invited to the meeting (e.g., family members, CPS worker, teacher, therapist, others identified by family).39. Current information about the family (e.g., social history, behavioral and emotional status) is gathered prior to the meeting and shared at meeting (or beforehand).40. All meeting participants introduce themselves (if applicable) or are introduced.41. The family is informed that they may be observed during the meeting.42. Plan of care is agreed on by all present at the meeting.
42Facilitator (6 items)Facilitator makes the agenda of meeting clear to participants.Facilitator reviews goals, objectives, interventions and/or progress of plan of care.Facilitator directs (or reflects) team to discuss family/child strengths.Facilitator directs (or reflects) team to revise/update plan of care.Facilitator summarizes content of the meeting at the conclusion of the meeting.Facilitator sets next meeting date/time.
44Results of WOFEMQ Sacramento shows higher scores on elements of Family-driven, Collaboration, Unconditional Care & Management of CFT MeetingsEMQ Sacramento continues to be challenged by lower scores on Community-based Resources, inclusive of natural supportsIn contrast to the results of the WFI for Outcomes (high scores), the WOF showed low scores on this elementSlight increase in Total WFI scores since the implementation of the WOF
45Back to the WFI Could the use of the WOF impact WFI scores? A look at WFI scores across time
46WFI Total Scores Across Time WOF Implementation Begins 7/05
47WFI Respondent Scores Across Time High Scores on Voice and Choice, Cultural Competence, Individualized Services, Strengths,Outcome-Based ServicesLow Scores on Natural Supports, Community-Based Services, and CFT Element
48Challenges of the WOFLimited resources in staff to become neutral observersManagers wanted to use the WOF as a supervision toolWillingness and comfort level of staff to participate in direct observationStaff perception of fear of families being judged by observersSponsorship by managementCompeting priorities in data collection
49Next StepsSupervisors will continue to utilize the WOF to provide feedbackOperational realities limit the use of neutral observersEMQ will no longer collect the WFI due to limitations in resources, the utility of the tool for feedback, and its relationship to outcomesContinue to develop practice standards
50ReferencesAchenbach, T. M., & Rescorla, L.A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.Epstein, M.H., Nordess, P.D., & Hertzog, M. (2002). Wraparound Observation Form-Second Version.Epstein, M.H., Nordess, P.D., et al. (2003). Assessing the Wraparound Process During Family Planning Meetings.Hodges, K. (2000). Child and Adolescent Functional Assessment Scale (CAFAS). CAFAS Self-Training Manual and Blank Scoring Forms (2003). Ann Arbor, MINordness, P.D., & Epstein, M.H.(2003). Reliability of the Wraparound Observation Form Second Version.Suter, J., Force, M., Bruns, E., Leverentz-Brady, K., & Burchard, J. (2002). Manual for training, administration, and scoring of the WFI 3.0. Burlington: University of Vermont.
51Contact InformationEnrica BertoldoQuality Support Manager, SacramentoMary Ann WongResearch Specialist, Sacramento
52Wraparound Supervisor Adherence Measure (W-SAM): A Pilot Eleanor Castillo, Ph.D.Veronica Padilla
53Rationale Supervision identified as a critical aspect of Wraparound No tool to date to measure this aspect of WraparoundConcept adapted from Multisystemic Treatment’s (MST) Supervisor Adherence Measure (SAM)
54RationaleWFI-3.0 measures adherence to Wraparound principles from the Youth, Caregiver, and Resource Facilitator perspectiveWOF-2.0 measures the Child and Family Team aspect of Wraparound
55Description Facilitator rates the Clinical Program Manager (CPM) 40 ItemsLikert Scale (1 = Never; 5 = Almost Always)Administered quarterlyHighly reliable (alpha = )
56Process Implementation Sponsorship from the DirectorsProgram quality improvement versus “mandate”Immediate feedback
57Quality Management Training Evaluation Performance Appraisal Overall Program ResultsIndividual CPMPerformance Appraisal360Annual
58Quality Management 3-Tiered: Agency-wide, Program, and Individual Training and CoachingEvaluationPerformance AppraisalQuarterly 360 feedbackAnnualCorrelation between supervisor focus on the Wraparound process and youth outcomes
59Quality Management Trend analyses Fidelity consistency over time What factors affect fidelityLearning curve with new manager or facilitators
60Next Steps Continue pilot Conduct further analyses on the psychometric properties of the scaleFactor analysesItems reduction
61Director of Outcomes & Quality Assurance Contact Information:Eleanor Castillo, Ph.D.Director of Outcomes & Quality Assurance