Improving the Practice and Performance of Public Health Departments: Results from the Evaluation of CDC’s National Public Health Improvement Initiative (NPHII) Cassandra Martin Frazier, MPH Applied Systems Research and Evaluation Branch Division of Public Health Performance Improvement Office for State, Tribal, Local and Territorial Support Keeneland Conference, April 22, 2015 Centers for Disease Control and Prevention Office for State, Tribal, Local and Territorial Support

Objectives  Describe progress toward achieving outcomes of increased accreditation readiness, quality improvement, and performance management capacity  Illustrate the reach of NPHII program funding and support to help improve public health practice and performance  Discuss plans to sustain current accreditation readiness, quality improvement, and performance management activities

NPHII: The Program at a Glance  Advance public health departments’ readiness for accreditation  Improve organizational efficiency and effectiveness through quality improvement activities  Increase performance management capacity Supported through the Prevention and Public Health Fund of the Affordable Care ActAffordable Care Act FY2010: $42.5M to 76 awardees FY2011: $33.5M to 74 awardees FY2012: $33.5M to 73 awardees FY2013: $32.5M to 73 awardees

NPHII Evaluation Approach  Purpose: extent to which NPHII supported improved public health practice and performance through  Increased readiness for accreditation  Improved efficiencies/effectiveness  Increased performance management capacity  Design: non-experimental program evaluation  Multi-strand, sequential mixed methods  Quantitative data: annual surveys,* progress reports  Qualitative data: stakeholder interviews,* focus groups* * OMB No

Data Sources for Presentation Data SourceProgram Progress Reports DescriptionSemi-annual reports submitted by awardees detailing progress toward completing required program activities Response RateYear 3 IPR—n=73 (100%) Year 3 APR—n=73 (100%) Year 4 IPR—n=72 (100%) Year 4 APR—n=68 (94%) LimitationsSelf-reported data Limited validation Question revisions over time AnalysisCross-sectional, frequencies—Year 4 APR Cross-time, aggregate—all sources (for select measures)

ACCREDITATION READINESS

Public Health Accreditation Board (PHAB)Prerequisites * Year 4 APR

For domains 1–5, at least 55% of awardees that completed a self-assessment against the PHAB Standards and Measures (n=62) met the standards* *Year 3 IPR and APR; Year 4 IPR and APR Organizational Self-Assessment Against PHAB Standards and Measures

At least half of awardees (52%) that completed a self-assessment against the PHAB Standards and Measures (n=62) met the standards for domains 6, 7, 10, 11, 12* *Year 3 IPR and APR; Year 4 IPR and APR

QUALITY IMPROVEMENT

Improvements Made in Efficiency Overall, 41 out of 72 awardees (57%) increased their program and/or organizational efficiency through quality improvement initiatives* *Year 3 APR; Year 4 IPR and APR

Improvements Made in Effectiveness Overall, 57 out of 72 awardees (79%) increased their program and/or organizational effectiveness through quality improvement initiatives* *Year 3 APR; Year 4 IPR and APR

REACH OF NPHII SUPPORT

Support to Other Public Health Agencies Accreditation Readiness Performance Improvement Quality Improvement Performance Management Number of Public Health Agencies Receiving Support Non-awardee agencies received indirect support to conduct NPHII-related activities; 52% of awardees used NPHII funds to provide this support (n=68)* * Year 4 APR

INFRASTRUCTURE AND SUSTAINABILITY OF PERFORMANCE IMPROVEMENT ACTIVITIES

Performance Management Capacity More than half of awardees established at least one of the four performance management system components required by NPHII* (n=66) * Year 4 APR

Sustaining Staff/Performance Improvement Efforts Most awardees plan to sustain their NPHII-related activities and performance improvement office* * Year 4 APR

Summary  Awardees increased their readiness for accreditation  Most established key documents and plans for their organization and partners  Most moved toward meeting standards necessary for accreditation  Awardees demonstrated more efficient processes that save time, and enhanced the quality of their services  Awardees plan to continue working on or maintaining these activities  NPHII not only supported its direct awardees, but reached agencies within their jurisdictions

Acknowledgments  Centers for Disease Control and Prevention  Anita McLees  Andrea Young  Craig Thomas  Garry Lowry  Saira Nawaz *  Laura Hsu *  NPHII Program Leadership and Staff  National Network of Public Health Institutes  Nikki Rider  Sarah McKasson  Brittany Bickford  Sarah Gillen  NPHII awardees * Former CDC contractor and evaluation fellow

For more information, please contact CDC’s Office for State, Tribal, Local and Territorial Support 4770 Buford Highway NE, Mailstop E-70, Atlanta, GA Telephone: CDC-INFO ( )/TTY: The findings and conclusions in this presentation are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Questions? Please contact Cassandra Martin Frazier Centers for Disease Control and Prevention Office for State, Tribal, Local and Territorial Support