Quality Services Reviews: A process for understanding and promoting best child welfare practice Florida Department of Children and Families Quality Assurance Managers Meeting March 14-15, 2011

2 Goals for Reviewer Training Learn how pilot QSR reflects the practice model Review the scoring criteria Learn about interviewing techniques to enhance information gathering Learn how to plan and lead an effective caseworker debriefing

3 Family-Centered Practice Engagement Team Formation Assessment & Understanding Planning Implementation Tracking & Adaptation

4 What is the Qualitative Case Review? The Qualitative Case Review process is a method for organizing the conversation with our communities in terms of : the results we want for children and families served, and understanding how child welfare practice can be improved to get those results.

5 Key attributes of the QSR Process Language used is transparent to a wide audience Underlying values as to child and family safety and well-being are broadly held Practice model reflects a coherent approach to supporting and sustaining change in families Process promotes learning at all levels in the organization, as well as in the larger community

6 QSR Uses a Record Review and Interview Format We evaluate outcomes and best practice, along with compliance to policy Record is reviewed at the beginning to gain and understanding of the case Information is gained through interviews with family members, the child, the caseworker, service providers, etc. Last interview with caseworker to share and update information and findings.

7 Quality Service Reviews: Common Understanding of Good Results for Children Served Traditional Audit: Were investigations timely in accordance with policy? Are child’s educational needs reflected in case plan? Is there documentation in the record that ongoing therapy (if recommended) is being provided? Quality Review: Are children safe from current, manageable risks of harm? Is the child leaning and progressing at a rate commensurate with his/her age and ability? Is the child doing well, emotionally and behaviorally?

8 Quality Service Reviews: A Different Way of Looking at Practice Traditional Audit Is there a plan in the file? Was the plan signed by the parents? Was the permanency goal presented to the court at the dispositional hearing? Qualitative Review Is the plan relevant to family needs and goals, coherent in selection and assembly of strategies, supports, services, and timelines? Was the family effectively engaged in assessment and service planning, and tracking? To what degree are service implementation and results routinely monitored, evaluated and modified?

9 Group Exercise 1: Understanding the QSR Indicators What are we measuring? How do we currently assess this issue? How is this new description different? Who would need to be interviewed to learn more about this? Provide some examples of circumstances that would fall into the “No” category of practice? (NO, standard not met equals a 1, 2, or 3 rating) (Teams of 3-4 persons)

10 How the scoring works 6 – Optimal 5 – Substantially Acceptable 4 – Minimally Acceptable 3 – Partially Unacceptable 2 – Substantially Unacceptable 1 – Completely Unacceptable Each indicator is scored on a scale of 1 to 6.

11 Quality Service Reviews: Focus on Most Recent Segment of Practice Child and family status indicators look at last 6 months – Exceptions are stability and permanence – Recent child/family status improvements may influence scores – Any child/family may have positive status regardless of length of involvement System performance looks at past six months – Recent performance improvements have some influence on scores – Any case can meet quality standards regardless of time open

12 Safety always trumps Safety is the only indicator that trumps Safety is paramount to any case. Overall status is acceptable only when Safety is rated in the 4 – 6 range.

13 “Groundhog Day” Rule Difference between a Rating of 3 and 4 (or a “Yes” or “No” If this case were frozen in time as it is today, would it be acceptable?

14 How the Review Process Works Purposeful sample of cases selected for each agency (younger and older children, children in care for varying lengths of time and with different permanency goals) Family permission obtained to participate in review and information releases obtained Key team members in each case identified and appointments scheduled (place, time frames) Lead reviewers and partners conduct the interviews, beginning and ending with the caseworker

15 How the Review Process Works, continued Answers are reviewed and scored in each indicator and domain Feedback session with caseworker and supervisor to provide immediate feedback and elicit further suggestions for next steps on case Presentation of stories at debriefing sessions during the week Exit conference at end of week to present and discuss findings Case story for each case reviewed Final report which includes aggregate data and case stories.

16 Expectations of QSR Reviewers Prepare for interview by being oriented to basic facts Thoughtfully explain the QSR process Consider how to best engage children/families Be attentive to clues about family culture Know what you need to learn from person being interviewed Allow each person to tell their story Gather each person’s perspectives

17 Expectations of QSR Reviewers, continued Ask each person interviewed about child safety Collaborate with your review partner Prepare for special challenges Stay within the role of reviewer Remember the obligation to report child safety issues Reviewers call it as they see it – score results and affirm efforts

18 Some “Don’ts” for QSR Reviewers Don’t be premature in making judgments Don’t share information across informants Don’t ask any leading questions Don’t drift from the protocol or definitions Don’t go unprepared into debriefing with Case Manager Don’t overwhelm staff with too many suggestions Don’t do “TA, Supervision or Training” during debriefing sessions

19 Basics for Interviewing Introducing yourself, partner and the process Starting the interview Allow the interviewee to tell the story Gathering necessary information Planning for the less than perfect situation (place and/or time constraints) Closing the interview

20 The Core “Script” for each interview. Why your perspective matters What is the story of your involvement? What are both the strengths and needs of this child and family? How is the system working to help this child and family? What are the goals? What things are working well? What would you like to see working differently? How might that happen?

21 Interviewing Core ConditionsSkills Genuineness Respect Empathy Clarification Closed questions Open questions Solution Defining questions Past Successes Exception Finding Miracle Questions Summarization

22 Caseworker Debriefings: Making it Positive Begin with a strength and affirmation Ask how the caseworker achieved something you learned that was positive (“How did you make that happen?”) Summarize and clarify child and family story, share any new information, clear up confusion Review child and family status findings (begin with strengths) Review system findings (begin with strengths)

23 Caseworker Debriefings: Making it Positive Elicit from caseworker/supervisor their thoughts as to next steps As appropriate, suggestions might be stated as, “We wondered what might have happened if at that point, (particular team members) might have had a team meeting to figure out…” Provide suggestions on some other options for next steps that MAY be useful Direct more difficult system questions to the supervisor

24 Group Exercise: Planning a Caseworker Debriefing As a team, identify three strengths about Jorge and why they are important. (5 min.) As a team, identify main concerns for Jorge. (5 min.) As a team, identify the issues that remain a practice challenge. (5 min.) As a team, develop a few suggestions for creating a positive caseworker debriefing. (10 min.)