1. Achieving Our Mission The Role of CQI in Public Safety Kimberly Gentry Sperber, Ph.D.

2. Efforts To Date “ What Works” Literature –Principles of Effective Interventions –Growing evidence based on individual program evaluations and meta-analyses Continuing Gap Between Science and Practice –Few programs score as satisfactory on CPAI/CPC

3. Importance of CPAI/CPC Criteria Differences in recidivism rates based on CPAI scores: –Scores of 0-49% demonstrated 1.7% reduction compared to comparison group. –Scores of 50-59% demonstrated 8.1% reduction. –Scores of 60-69% demonstrated 22% reduction. What are implications for ODRC standards for correctional programs?

4. Why Isn’t “It” Working? Latessa, Cullen, and Gendreau (2002) 4 common failures of correctional programs: –Failure to use research in designing programs –Failure to follow appropriate assessment and classification practices –Failure to use effective treatment models –Failure to evaluate what we do

5. What Happens When We Don’t Measure Ourselves? The Role of Fidelity

6. Washington State Example (Barnoski, 2004) For each program (FFT and ART), an equivalent comparison/control group was created Felony recidivism rates were calculated for each of three groups, for each of the programs Youth who received services from therapists deemed ‘competent’ Youth who received services from therapists deemed ‘not competent’ Youth who did not receive any services (control group)

7. Functional Family Therapy Results: % New Felony Results calculated using multivariate models in order to control for potential differences between groups

8. Project Greenlight Short-term prison-based reentry program in New York –CBT Skills Training –Employment Services –Housing Services –Drug Education and Awareness –Family Counseling –Practical Skills Training –Community-Based Networks –Familiarity With Parole –Individualized Release Plans

9. Project Greenlight Benefits Participants received more service referrals Participants reported more contacts with community services after release Participants demonstrated significantly more familiarity with parole conditions Participants were more positive about parole

10. But Did It Work?

11. What Went Wrong? Violation of the risk principle –Ceased use of risk assessment instrument when staff deemed process too cumbersome Violation of the need principle –All offenders received same services whether needed or not Violation of the fidelity principle –Staff modified delivery of the CBT curriculum (shortened the duration, increased frequency, increased class size) Differential staff competence –Certain case managers produced worse outcomes

12. UC Halfway House/CBCF Study in Ohio: A Look at Fidelity Statewide (2002) UC Halfway House/CBCF Study in Ohio: A Look at Fidelity Statewide (2002) Average Treatment Effect was 4% reduction in recidivism Lowest was a 41% Increase in recidivism Highest was a 43% reduction in recidivism Programs that had acceptable termination rates, had been in operation for 3 years or more, had a cognitive behavioral program, targeted criminogenic needs, used role playing in almost every session, and varied treatment and length of supervision by risk had a 39% reduction in recidivism

13. 2010 UC Halfway House/CBCF Study in Ohio: Adherence to CBT in Groups and Changes in Recidivism

14. What Do We Know About Fidelity? Fidelity is related to successful outcomes (i.e., reductions in recidivism, relapse, and MH instability). Poor fidelity can lead to null effects or even iatrogenic effects. Fidelity cannot be assumed Fidelity can be measured and monitored.

15. Bringing Mission to the Forefront

16. Becoming a Mission-Driven Organization Knowing the mission statement What does it mean to you? What does it mean for your role? Who is responsible for fulfilling the agency’s mission statement? How can QA/QI help us accomplish our mission statement?

17. Providing Managed Change Achieving and maintaining quality is not accidental. It requires a deliberate effort on the part of everyone.

18. Achieving Quality Responsibility for quality falls on both the organization and the individual. The individual and the organization should be linked in a formal framework designed to continually improve quality.

19. Quality Assurance (QA) Retrospective review process Emphasis on regulatory and contract compliance Catching people being bad leads to hide and seek behavior

20. Continuous Quality Improvement (CQI) CQI is a prospective process Holds quality as a central priority within the organization Focus on customer needs; relies on feedback from internal and external customers Emphasizes systematic use of data Not blame-seeking Trust, respect, and communication Move toward staff responsibility for quality, problem solving and ownership of services

21. Focus on Causes not Symptoms Focus on processes/systems rather than individuals or specific errors Identification of risk points and their contribution to the problem Identify changes in these processes that reduce risk of re- occurrence

22. Objectives of CQI To facilitate the Agency’s mission To ensure appropriateness of services To improve efficiency of services/processes To improve effectiveness of directing services to client needs To foster a culture of learning To ensure compliance with funding and regulatory standards

23. Common Elements Documentation Review Performance Indicators –Process vs. Outcomes Satisfaction – Client, Employee, Referral Source/Stakeholders Protocols for Data Use Infrastructure

24. Focused Discussion: Why Examine Documentation? Clinical/Service Implications –Documentation is not separate from service delivery. –Did the client receive the services he/she needed? Operational Implications –Good documentation should drive decision-making. –Means of communication Risk Management Implications –If it isn’t documented, it didn’t happen. –Permanent record of what occurred in the program/unit. Source of Staff Training Reflection of the provider and organization’s competency: –EBP –Outcome of care

25. Action Plans Plan of correction Proactive approach to problem- solving Empowers staff Using objective data to inform decision making

26. Who Creates Action Plans? Anyone and everyone can create action plans Focus should be on who has knowledge or expertise to contribute Focus should not be on the person’s title

27. Minimum Requirements Buy-in from staff at all levels of the organization Sufficient resources allocated for staff training Sufficient resources allocated for staff to participate in the process –Meetings –Data collection, reporting, use Sufficient information systems

28. Why Invest the Resources? What makes this effort too compelling to ignore?

29. A CEO’s Perspective Why invest in CQI? –Because it’s the right thing to do! –Better for clients (i.e., better outcomes) Mission-driven –Increased staff satisfaction –Increased staff retention –Improved referral source satisfaction –More business for related projects –Outcomes to sell to business community and other payers –Demonstrates fiscal responsibility (i.e., effective use of dollars)

30. Strategic Use of CQI Data CQI data used to provide testimony before legislature CQI data and infrastructure used to secure new contracts and grants CQI data used in newsletters, media relations, levy campaigns, etc. CQI data used to negotiate programmatic changes with stakeholders

31. Performance Examples Men’s halfway houses employment with benefits –33% to 67% over 5 years Changes in HIT over time per strategic plan –57.4% to 72% over 5 years CCC’s employment indicator performance –<10% to 75% over 5 years How do these changes in performance link to mission?

32. Top 10 Practices for Reducing Recidivism in Drug Courts (Carey & Finigan, 2013) 1.Program caseload less than 25 2.Participants have greater than 90 days clean before graduation 3.Judge spends average of 3 minutes or more per person during status review hearings 4.Treatment communicates with court via email 5.Representative from treatment attends drug court team meetings

33. Top 10 Practices for Reducing Recidivism in Drug Courts (Carey & Finigan, 2013) 6.Review of the data/program stats has led to modification in operations 7.Representative from treatment attends court sessions 8.Drug court allows non-drug charges 9.Law enforcement is member of drug court team 10.Results of program evaluations have led to modification in operations

34. Top 10 Practices for Increasing Cost Savings in Drug Courts (Carey & Finigan, 2013) 1.Review of the data/stats has led to modifications in operations 2.Results of program evaluations have led to modification in operations 3.Sanctions are immediate 4.Defense attorney attends drug court team meetings 5.Must have a job or be in school to graduate

35. Top 10 Practices for Increasing Cost Savings in Drug Courts (Carey & Finigan, 2013) 6.Representative from treatment attends court sessions 7.Team members are given copy of sanctioning guidelines 8.Drug test results back in 48 hours or less 9.Drug tests collected at least 2x/week during first phase 10.Law enforcement attends court sessions

36. NPC Research on Drug Courts

37. The Role of QA/QI in Community Corrections (based on UC Halfway House and CBCF study) The Role of QA/QI in Community Corrections (based on UC Halfway House and CBCF study)

38. Bottom Line Many programs are implementing evidence- based practices with little evidence of strong fidelity. Result is an ongoing gap between science and practice. This gap often results in null or even iatrogenic effects. Correctional organizations have a responsibility to ensure effective services. Responsibility for EBP needs to be aligned at all levels – administration, management, line staff. Need to focus on creating formal infrastructure to support EBP and mission achievement

39. Continuous Quality Improvement Questions & Answers