1. C ontinuous Q uality I mprovement Michelle P. Hill mhill@pa.gov 717.772.4850 1 Departments of Education and Public Welfare | www.education.state.pa.us | www.dpw.state.pa.us MIECHV Grantee Meeting 6/5/2013

2. Session Agenda 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 2

3. 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 3

4. Brainstorm: What is CQI? 4

5. Quality improvement comes from a desire to make things better At first glance, quality improvement might seem like more work piled on to an already impossible work load But if quality improvement can ultimately lessen that workload, and allow you to provide more efficient/better home visits for your families, leading to better outcomes, isn’t it worth it? 5 The BIG picture

6. 6 Quality AssuranceQuality Improvement MotivationMeasuring compliance with standards Continuously improving processes to meet standards MeansInspectionPrevention AttitudeRequired Defensive Chosen Proactive FocusOutliers: "bad apples" Individuals Processes Systems ScopeAgency OverallClient Care ResponsibilityFewAll Quality Improvement  Quality Assurance

7. Quality assurance (QA) measures compliance against certain necessary standards. QA can be reactive, retrospective, policing, and in many ways punitive. It often involves determining who is at fault after something goes wrong. However, standards and measures developed for quality assurance can inform the quality improvement process. 7 Shifting Away from Quality Assurance

8. o a process of continuous improvement o involves both prospective and retrospective reviews o improvement -- measuring where you are, and figuring out ways to make things better o create systems to prevent errors from happening o improving how things work o trying to find where the “defect” in the system is o figuring out new ways to do things o “think outside the box” 8 So, what is quality improvement?

9. 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 9

10. An Opportunity Comprehensive, multi-level CQI is a federal grant requirement and a commitment Pennsylvania has made in its MIECHV CQI Plan... but more importantly, it is an opportunity: To refine and share best practices; To enhance decision-making; To individualize services for participating families across sites and programs 10

11. A Shared Responsibility CQI leverages the expertise and perspective of project participants across roles, levels, and agencies To work well, CQI needs to be a safe space for constructive input from all It can’t just be something the state, the programs, or the “data people” are doing alone Community input is vital Communication is key – challenges and successes need to be freely shared 11

12. Pennsylvania’s MIECHV CQI Plan 3 teams Bottom-up approach 12 MIECHV State CQI TeamMIECHV Program CQI TeamMIECHV Local CQI Teams

13. Survey Results 30 different agencies implementing MIECHV programs 25 agencies responded to the survey 21 have a CQI plan, CQI lead and/or a CQI team 3 have a CQI lead and/or a CQI team, but no CQI plan 1 does not have a CQI lead/team or a plan 13

14. Discussion: Share current CQI processes 14

15. 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 15

16. Initial Activities (of all Teams) learn about CQI define the role and function of a CQI team determine the frequency of CQI team meetings and activities identify the people/positions responsible for elements of the CQI plan provide oversight of the CQI Plan define quality for the MIECHV program identify goals and objectives 16

17. Team Members should include the following: Lead – oversees the team; facilitates the meetings Scribe – captures all documentation Member – provides information and helps make good decisions Technical Expert – subject matter experts 17

18. MIECHV State CQI Team Provides an opportunity to address statewide issues and consolidate information and issues from all other teams. Meetings will be held quarterly in-person. 18

19. 19 1 Members are optional attendees at meetings. 2 Denotes subcontractors. MIECHV State CQI Team

20. Possible issues to be addressed are: Monitor data collected across programs Discuss unresolved issues from Program and Local Teams Ongoing comprehensive program evaluation Monitors comprehensive program model fidelity Communicate shared successes and resolved issues Report on aggregated data Assist in the development of the PA MIECHV State CQI plan Through tracking the output and process indicators related to the constructs, the MIECHV State CQI Team will be able to identify specific agencies that require additional support. CQI activities will focus on ensuring that state goals are met. Data will be analyzed for progression with enrollment, provider training, frequency of home visits, and data entry compliance, in addition to monitoring whether one site or model is doing better than others on certain benchmarks and why, including identification of best practices and lessons learned. 20 MIECHV State CQI Team

21. MIECHV Program CQI Team Provides an opportunity to address issues that impact the MIECHV programs as a whole and any unresolved issues presented by the MIECHV Local CQI Teams. Meetings will be held every other month via webinar/phone (in-person when possible). 21

22. 22 MIECHV Program CQI Team

23. Possible issues to be addressed by the MIECHV Program CQI Team are: Discussion of common issues (and shared successes) across agencies and models Discussion of unresolved issues brought up from MIECHV Local CQI Teams Discussion of data Discussion of model fidelity Review and provide feedback to State on the use of assessment tools, program questionnaires, and implementation of the MIECHV program The overall goal of the MIECHV Program CQI Team is to continuously improve the CQI process so that the MIECHV Local CQI Teams have fewer challenges and greater success in optimally serving participants. Members of the MIECHV Program CQI Team will work directly with the MIECHV Local CQI Teams. 23

24. MIECHV Local CQI Teams Provide an opportunity to address issues that impact individual MIECHV agencies. Meetings will be held monthly in-person. 24

25. Discussion: Local CQI Team Members 25

26. MIECHV Local CQI Teams 26 Program DirectorHome Visiting SupervisorData Entry Staff Home VisitorsAll Support Staff

27. MIECHV Local CQI Teams The MIECHV Local CQI Teams will be required to: Have a CQI plan, which will be revised, at a minimum, annually Focus on agency level data Monitor model fidelity at the agency level Determine areas of improvement Address quality improvement challenges (i.e. staff retention, participant attrition) Identify issues with program evaluation(s), communication, information systems, hiring, safety, etc. Issues that are not resolved can be brought to the MIECHV Program CQI Team. The EDUCATIONAL RESEARCH ASSOCIATE 2 and STATISTICAL ANALYST 1 will be tasked with regular dialogue with local sites around CQI. 27

28. 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 28

29. Components of a Culture of Quality Generate buy-in at all levels: movement from an ‘us’ to a ‘we’ Understanding the system: process steps that lead to the outcome Data collected that are relevant and meaningful Utilization of data to monitor progress towards established target Small-scale tests of interventions 29

30. Understanding of Current Culture Mission, vision and values support culture Determination of where we are today vs. where we want to be Consistent and frequent methods of communication Lines of accountability Expectations for performance Processes and procedures 30

31. 31 Culture of Quality AttitudeDataTransparencyOutcomesCommitment Agencies strive to reach pre- determined targets. Every situation can be a learning experience. Data are essential to day-to-day work. Data collection infrastructure exists. Reporting metrics are meaningful. Data elements are relevant, accurate, and important. Reports are timely and recent. Reports show performance relative to targets. Reports show changes over time. Reports break out small units. All members are committed. Key members are trained in QI methods and support strategic initiatives. Frontline practitioners are supported by leadership. Improvement is guided by team approach and accountability. Data are readily accessible. Practices and methods are shared. Results and outcomes are shared. Individual contributions are recognized. Open communication and critical scrutiny are welcomed. Shared learning decreases the need to ‘reinvent the wheel.’ Outcomes are clearly articulated and understood. Outcomes are measured against an established target. All process measures are tied to outcomes. There is a deep understanding of systems, processes, and procedures. Performance is reported frequently.

32. Organizational Culture of Quality Checklist Transparent Curious and eager to learn Open to failure as a learning opportunity Committed to improving processes and outcomes Data-driven with sufficient MIS capacity Trained from top to bottom in CQI methods 32

33. MIECHV CQI Process Map 1: Data Collection Process MIECHV CQI Process Map 1 provides an overview of the data collection process, from client intake to data extracts from PA MIECHV and NFP ETO data systems. Client enrolled Intake Data collected by home visitors Data brought back to local MIECHV agency Data entered into PA MIECHV & NFP ETO data systems Data extracts from PA MIECHV & NFP ETO data systems

34. MIECHV CQI Process Map 2: Data Sharing Process Once all data are received, the STATISTICAL ANALYST 1 will analyze the data and create trend reports. These reports will be shared each month with the MIECHV Local CQI Teams. After reviewing the data at the agency level, the MIECHV Local CQI/Data lead will work with the STATISTICAL ANALYST 1 to improve data quality. The STATISTICAL ANALYST will then upload all data into an Excel spreadsheet to perform data clean- up while observing for outliers and continuing to follow-up as needed with sites for missing / questionable data. The INFANT TODDLER SPECIALISTS and Model TAs will also follow-up with sites on any program issues/TA needs. The STATISTICAL ANALYST 1 will conduct data analysis and create reports for the MIECHV Local CQI Teams; this will enable the local agencies to see how data quality can affect results, as well as to see their final results. Data extracts from PA MIECHV (pulled on 10th of each month) and NFP ETO data systems (pulled on 17th of each month) Preliminary data analysis by STATISTICAL ANALYST 1 Share data with MIECHV Local CQI Teams (last Thursday of the month) Follow up with MIECHV Local CQI Teams re: data issues (STATISTICAL ANALYST 1) Final data review, clean up, and analysis by STATISTICAL ANALYST 1 Follow up with MIECHV grantees re: TA needs (REGIONAL HOME VISITATION SPECIALISTS & Model TAs)

35. MIECHV CQI Process Map 2: Data Sharing Process Reports will also be shared with the MIECHV Program CQI team (every other month), MIECHV State CQI team (quarterly), Home Visitation Stakeholder Committee (quarterly), and the Early Learning Council (twice a year). While reports for the MIECHV Program and State CQI teams will include data that is identified at the agency level, the other two teams will only see data that is de-identified. The goal of sharing data with the Home Visitation Stakeholder Committee and the Early Learning Council is to provide them information about the children and families being served through MIECHV as well as Pennsylvania’s progress towards meeting the benchmarks. Share data with MIECHV Program CQI Team (every other month) Data review and discussion with MIECHV State CQI Team (quarterly) Share data with Home Visitation Stakeholder Committee (quarterly) Share data with Early Learning Council (twice a year)

36. 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 36

37. Discussion: CQI plan 37

38. 1.Introduction to Quality Improvement 2.Pennsylvania’s MIECHV CQI Plan 3.CQI Team 4.Culture of Quality 5.CQI Plan 6.Model for Improvement 38

39. Model for Improvement 2 parts: (1) THREE fundamental questions –What are we trying to accomplish? Set an aim. –How will we know that a change is an improvement? Establish measures. –What changes can we make that will result in improvement? Take action. –We know that improvements require change, however not all changes are an improvement. We have to test whether a change is an improvement, and that’s where the PDSA cycle comes in. 39

40. Model for Improvement 2 parts: (2) Plan-Do-Study-Act (PDSA) cycle –Plan a change. Clarify the objective, make a prediction, figure out the who, what, where, when, and how. –Do (Test) the change. Carry out the plan, document any unexpected problems or challenges, begin analyzing data. –Study (Review) the tests. Complete data analysis, compare results to theory and prediction, summarize and present data. –Take Action based on what has been learned. –If the change did not work, go through the cycle again with a different change. If the change was successful, use what has been learned to begin planning new improvements. 40

41. How Do Tests Lead to Improvements? 41 Implementation of change Wide-scale tests of change Follow-up tests Very small scale test Changes that result in improvement The cycles build on each other…

42. Things to Remember Keep it small – design the first test for one home visitor, 5 clients Scale down the time frame (weeks, days, hours) Keep the first tests simple See what happens, act on that knowledge, and then scale-up the test And remember You’ll never make a change if you don’t… actually make a change. 42

43. 43

44. 44

45. Time to Practice 45

46. Survey Results Top eight ranked areas of improvement (in order from the highest to the lowest ranked): 1.retention of families 2.improvements in family self-sufficiency/stability 3.families are connected to needed services and social supports 4.recruitment of families 5.increase in screenings for domestic violence 6.improved parenting skills 7.community partnerships 8.prevention of injuries and maltreatment 46

47. Next Steps  Identify agency-level CQI team members  Share current CQI plan  Plan initial agency-level CQI meeting  Identify initial measures and areas of focus  Seek technical assistance 47