Preparedness Project Lessons NC AWWA / WEA 2015 Annual Conference Jack Moyer

Types of Projects Included Project Locations General Lessons and Observations Security and VA Lessons Emergency Planning Lessons Business Continuity Planning Lessons Closing Points Overview

3 Preparedness Lessons Learned Vulnerability assessments (VA) Emergency response plans (ERP) Continuity of operations plans (COOP) and business continuity plans (BCP) Tabletop exercises and games (enhanced tabletop exercises) Physical security standards development Other security and preparedness planning projects Types of Projects Included

4 Preparedness Lessons Learned Drought preparedness planning Emergency operations center (EOC) and joint information center (JIC) preliminary design Continuity of government (COG) planning Public information office (PIO) planning Dam emergency action plan (EAP) tabletop exercises Other Types of Projects

5 Preparedness Lessons Learned Project Locations

6 Preparedness Lessons Learned Lack of a culture of security and preparedness Opportunity to address “low-hanging fruit” Importance of visible management commitment Importance of engaging stakeholders IT engagement challenges General Lessons and Observations

7 Preparedness Lessons Learned Inadequate policies and procedures Lack of training and awareness Lack of enforcement Lack of Security / Preparedness Culture

8 Preparedness Lessons Learned Many have good disaster recovery plans (DRP) Often difficult to get IT leadership engaged with the rest of the preparedness project team The project champion or upper management must get the IT experts to participate IT Engagement Challenges

9 Preparedness Lessons Learned Lack of maintenance Fence weaknesses Camera weaknesses Need to address cyber security and process control systems Other weaknesses in security equipment and procedures VA Lessons

10 Preparedness Lessons Learned Inadequate maintenance of security improvements, resulting in inoperable cameras, damaged fences, etc. Inadequate budget and resources for the maintenance of security systems Competing priorities for funding such as rehabilitating degraded infrastructure or decreasing revenues Inadequate Maintenance

11 Preparedness Lessons Learned Gaps underneath or at gates Unrepaired damage Vegetation and other compromises to the fences Cheap padlocks, chains, and daisy-chaining of padlocks Fence Weaknesses

12 Preparedness Lessons Learned Where present, cameras and camera systems nearly always have weaknesses, including: Camera systems that don't work as intended, and often never did Cameras that are intended to be monitored, but are not Cameras that are no longer compatible with computers in use Camera Weaknesses

13 Preparedness Lessons Learned Rapidly evolving threats Stuxnet / Germany Presidential Executive Order February 2013 AWWA Process Control (Cyber) System Security Guidance Document Need to Address Cyber Security

14 Preparedness Lessons Learned Doors propped open that are supposed to be closed and locked Unresolved concerns regarding disgruntled past or current employees Poor housekeeping in some areas, leading to safety and security compromises Lack of enforcement of existing policies and procedures Other Common Weaknesses - 1

15 Preparedness Lessons Learned Vulnerable to potential malevolent acts by both contractors and disgruntled employees Background checks on contractors are generally inadequate Contractors often have unsupervised access Password protection and key control programs at many systems are often lacking Other Common Weaknesses - 2

16 Preparedness Lessons Learned ERPs not up-to-date, particularly contact information Insufficient emergency response training and exercises Few ERPs include NIMS and ICS Better inter-agency coordination needed Emergency Planning Lessons

17 Preparedness Lessons Learned National Incident Management System Incident Command System Few ERPs include NIMs and ICS

18 Preparedness Lessons Learned Better Inter-agency Coordination Needed

19 Preparedness Lessons Learned Few plans include crisis communication plans for critical notifications Few plans address the threat of armed intruders or active shooters Emergency Planning Lessons - 2

20 Preparedness Lessons Learned Pandemic plans are often lacking or too focused on the flu Often Lack Pandemic Plans

21 Preparedness Lessons Learned Employees are a water utility’s most valuable and most vulnerable resource They are only as valuable at work as their families are prepared at home Many systems do not have adequate provisions to help employees and their families prepare Weak Employee Preparedness

22 Preparedness Lessons Learned Few water systems have BCPs or COOPs Stakeholder engagement is critical in BCP and COOP projects Mission essential functions (MEF) are often very challenging for systems to identify and prioritize in BCP development The importance of succession plans is often a challenge to convey and seldom done Emergency procurement needs to be addressed BCP Lessons

23 Preparedness Lessons Learned Plans often lack provisions for emergency procurement and to address critical interdependen -cies Emergency Procurement

24 Preparedness Lessons Learned Water and wastewater systems have done much to prepare There are many opportunities for improvement and security preparedness in most water and wastewater systems Many of those opportunities are neither difficult nor expensive What is needed is a commitment to improvement in those areas Closing Points

Questions or Comments? November 17, | Jack Moyer