North Carolina Institute for Public Health Measuring Public Health Agency Preparedness Mary V. Davis, DrPH, MSPH December 20, 2006

What is the Role of Public Health Agencies in Disasters/Bioterrorism?

Feedback Poll Is it possible for me to know whether my health department is prepared? A. Yes B. No

Measuring Preparedness What does it mean to be prepared? How can preparedness be measured? –Definitions –Metrics –Instruments What is the relationship between individual competency and agency preparedness?

Measurement 1.0 Evaluation of Public Health Response to Hurricanes Finds North Carolina Better Prepared for Public Health Emergencies January–June 2004

Purpose of NC Study Identify how BT funds improved public health capacity in North Carolina to prepare for and respond to bioterrorism events and other public health emergencies. Examine how improvements in public health capacity for a specific public health emergency can translate to all public health emergencies.

NC Context Capitalize on “natural experiment” –1999 Hurricane Floyd –2002 Infusion of Federal funds to address bioterrorism –2003 Hurricane Isabel Strong partnership history between NC DPH and UNC SPH No standard definition of “preparedness”

Evaluation Questions—Resources Phases of Disaster Framework—Noji Public Health Management of Disasters— Landesman Criteria for evaluating management of disasters—Quarentelli

NC Hurricane Evaluation Study Handout Page 1 Evaluation Questions

Data Collection Key informant interviews—Professional role involvement in Floyd and/or Isabel Document review—state and federal protocols, documents, records

Results Floyd Planning—no specific activities Preparation –Communication with local health departments Response –Command and Control –Surveillance and Assessment

Floyd and “The Box”

DPH Use of Federal Funds Created Infrastructure –Office of Public Health Preparedness & Response –Public Health Command Center –Public Health Regional Surveillance Teams Employed Staff –BT Coordinator –Office of Public Health Preparedness & Response staff –Public Health Regional Surveillance Team staff: 28 in 7 regions statewide –Public health epidemiologists: 12 statewide Implemented Health Alert Network Implemented NC Emergency Department Database

Actions (cont’d) Enhanced Laboratory Capacity –Biosafety Level 3 and Level 2 Purchased Equipment –8 vehicles –Fixed vehicle-mounted and hand-held radios Strengthened Communication –Bioterrorism Steering Committee, reps from multiple agencies Provided Training –Emergency preparedness and core public health competencies –Exercises and drills

Resulted in Capacities Used in Isabel Effective, centralized communication through Public Health Command Center –Between state agencies –With local public health departments –With community and academic partners –With citizens

Capacities (cont’d) Command and Control Incident Management –Operational and functional plans –Systematic response –Decision making in real-time emergency conditions –Epidemiology and surveillance Rapid needs assessment Medical surveillance

Preparedness Logic Model Handout Page 2 NC Model

Preparedness Logic Model Handout Page 3

Preparedness Logic Model RAND Corporation. RAND Health. Learning from Experience, Page 9 Goal: mitigate morbidity, mortality, stress Objectives –Early identification –Early and effective response –Earliest return to normal function Functional capabilities meet objectives Capacity-building activities result in capabilities

Measurement 2.0 Mississippi Department of Health Hurricane Katrina After Action Report January–June 2006

Target Capabilities List U.S. Department of Homeland Security Target Capabilities List version 1.1 Defines 36 prevention, protection, response, and recovery capabilities (measures) Capability measures: capacity (processes, infrastructure) Performance measures (demonstrate task execution)

Why Target Capabilities? Becoming benchmarks for state performance Provide a framework for performance measures Create a performance plan –Specific –Measurable –Obtainable

TCL Pilot Test MS Department of Health Katrina After Action Review –Specific type of evaluation –Identifies strengths and weaknesses in response –Defines specific action plan for improvements Examined response in 21 TCLs –Emergency Support Function 8—public health –General TCL’s—planning

Target Capabilities Examined in MDH Katrina AAR Handout 4

NC Involvement MS requested NC to construct a team to assess ESF 8 response MDH planning with Region IV, specifically NC OEMS prior to Katrina NC mobilized SMAT to Waveland, MS during Katrina response NC personnel familiar with gulf communities and MDH structure

MDH Katrina AAR Identify major points what worked/did not work well during MDH Katrina response Katrina in Emergency Support Function 8 Data collection (all new instruments) –Responder Online Assessment –Responder Interviews –Community Assessment Results used to: –Inform Performance Improvement Plan –Revise MS State Emergency Operations Plan

Online Survey Example Interoperable Communications (Communications and Information Management) * 1.2.0: Were you a part of Interoperable Communications (Communications and Information Management)? Please choose only one of the following: Yes No [Only answer this question if you answered 'Yes' to question '1.2.0 '] 1.2.2: Were sufficient back-up equipment and power sources available? Please choose only one of the following: Yes No Unknown [Only answer this question if you answered 'No' to question '1.2.2 '] 1.2.2n: Please explain why you answered no.

Online Survey and Interview Data Feedback available for all TCLs –Least for Animal Health Emergency Support –Most for Planning, Mass Care Substantive open-ended feedback from survey “no” responses Interviewees provided substantive information on what worked and areas for improvement

AAR Recommendations Overarching capability and specific TCL Results from online survey, interviews, community assessment Experience and knowledge of MDH and NC team Federal guidelines

Limitations First use of TCL Performance Measures as data collection instruments Limited time to create and validate data collection instruments Respondents lacked knowledge of performance measure MDH identified participant samples Challenge to digest the wealth of information from the results

Take Home Points First use of Target Capability List as an AAR framework First instance of one state government conducting an AAR for another state NC team drew upon EM, OEMS, PHP&R, UNC SPH, and other expertise Approach yields data in performance measure framework

Preparedness Measurement Next Steps Modify Target Capability measurement instruments Pilot in an After Action Review Continuously Refine Measures Use RAND Public Health Preparedness Logic Model as guide Measures should be –specific about responder function –specific about activities and capabilities

Public Health Preparedness Continuous process—constant quality improvement Agencies need to test preparedness in real conditions –Exercises and drills –Outbreak investigations –Disaster, BT responses

Individual Competency and Agency Preparedness Using RAND Public Health Preparedness Logic Model Individual Competency –Conduct capacity-building activities –Demonstrate competency to perform functional capabilities Agencies –Implement functional capabilities –Achieve preparedness objectives and goals

Resources NC Hurricane Evaluation Article in PH Reports in Jan/Feb 2007 issue MDH AAR Report Contact