1. Continuous Quality Improvement: Ideas from The Field Becca Sanders, Carolyn Sullins Evaluators, Kalamazoo Wraps System of Care beccaanddan@gorge.netbeccaanddan@gorge.net; csullins@wmich.educsullins@wmich.edu

2. Objectives From Theory to Practice in the Field Challenges to implementing the model Reorienting Evaluation –Onlooker vs. In the Mix –CQI: “The Big 3” to be Tackled –Technical and Adaptive Components How CQI? Our “Non-model Model” –New roles for evaluators –The Foundation: Key Components –What Matters –What Does it Look Like?

3. Continuous Quality Improvement: Ordered Chaos Ummm… there is no “model”… Linear Schminear Too much to wrangle! Just pick a spot Then start to untangle Tis not “a” thing There is no “it” There’s no such clarity Does not fit! More tangles will appear This is true Untangle them Each one anew

4. Model Challenges: CQI in an SOC Context The Scope of SOC Not a program The Scope of the SOC Audience Relevance, right people at the right table- not so easy The SOC Aggregate Data Only Problem Moving target, changing system The Nature of the SOC Problem Technical and adaptive problems require technical and adaptive solutions CQI looks good on paper: Moving from paper to action… It’s another story… What are the barriers? “Check out our tight indicators and efficient quarterly review process… yeah… we’ll work on that…”

5. More Model Challenges: CQI in an SOC Context Most SOCs Already Have: Data demands from other agencies Plenty of (inconsistently collected, underutilized) data Concerns about how data may be misinterpreted Skepticism about the usefulness of more data

6. Reorienting Evaluation: A Key Shift for Doing CQI Well Data as “beside” the (systems) problem; a contributor to the solution. Help figure out if good “stuff” is happening in an initiative. If not, why not? If not, what should we do about it? Clean, linear, solid aggregate indicators Data as “within” the (systems) problem; a contributor to the problem (and solution). Data culture chronically nonfunctioning or nonexistent. Data culture in need of CQI in and of itself. “Readiness for CQI:” Precedes all else

7. Reorienting Evaluation Onlooker vs. In the Mix for a Fix Evaluation of Change Efforts Multiple Change Efforts Practices/ Services Procedures Policies Etc… Multiple Change Efforts Practices/ Services Procedures Policies Etc… Data Culture Onlooker In the Mix for a Fix Does the data stuff… vs. Helps us do the data stuff

8. What does Reorientation Look Like? CQI: “The Big 3” To Be Tackled 1)Data Existence: Collect relevant, useful data, consistently and accurately. Simplify this process! 2) Data Access: MIS system extraction, MIS data input, resources, expertise, power structures, trust, bureaucratic structures, agency culture, regulations, interest/ competence levels, data silos, data sharing agreements, understanding. 3) Data Dissemination: report on time, clearly and simply, link to individuals when appropriate, level “local” enough to be helpful. When ducks in a row, data use follows

9. CQI: The Big Picture Outcome Big Picture Outcome: Build a data-guided culture across organizations/ stakeholders by helping communities improve “The Big 3:” 1.Data Existence 2.Data Access 3.Data Dissemination A Mix of Technical and Adaptive Issues See ya’ll in 10 years… this is going to take me a while.

10. Technical versus Adaptive What do we mean? Technical: “WHAT there is to know.” Requires a knowledge transfer The “teachables” Adaptive: “WHY this is important.” “HOW you can use it.” Requires a value shift The “less-teachables”

11. “The Big 3:” Require a Mix of Technical and Adaptive Solutions Welcome to the Kalamazoo Wraps luxury! Hi! I’m an evaluator… I focus mostly on the adaptive stuff related to building a data guided culture: coaching, working one on one with stakeholders, helping people understand the value of data, upping interest and engagement in data related activities, making sure people understand reports and that they’re pretty. My budget is small. Hi! I’m an evaluator… I focus mostly on the technical stuff related to building a data guided culture- collecting data, creating and managing databases, analyzing data, helping create and review instrumentation. I also run the national evaluation. My budget is large. Hey… lets support one another’s roles!

12. Strategies for CQI Capacity Building Technical and Adaptive Solutions The Big 3Highly Technical Highly Adaptive Data Existence Data Access Data Dissemination Coaching Instrument Review and Recommendations Help with MIS systems set up Data Collection Technical Assistance MIS system access Analysis Data management MOU establishment with partnering organizations Catered Reporting Content Timing Stakeholder driven Data splits Product Review

13. CQI: What Matters? The Foundation: Key Components Initiative/ Leadership: Who Cares? Relationship building/ personality: What’s the engagement / hook Data splits for targeted responses By clinician, by individual… Fluidity: Product, Timing, Whose at the Table Tweak, Tweak and Tweak some more Beware of polite toleration of CQI efforts! (Adaptive) Offer a service! (Adaptive) Aggregate lacks utility! (Technical) pi= 3.141592653…and on it goes (Adaptive and Technical)

14. CQI: What Can it Look Like A “Close Proximity” Family Vacation: A few key people have a vague notion of the kind of place they want to go… So they hop in the car… But there’s no map… So you get lost… But you know enough to get found… Then you arrive… At something close to what you were going for… But you swear there are measures you can take so next year will be better… So you start planning… Then you hop back in the car… If You Give a Mouse a Cookie…

15. In Conclusion Foundation/ key components to quality CQI- clear Various facets of the CQI problem- clear Various facets of the CQI solution- clear Roles/ who does what regarding CQI- clear How to apply CQI: (intentionally) not so clear. Why? –fluid –catered –responsive –highly localized x