Presentation is loading. Please wait.

Presentation is loading. Please wait.

Let’s Get Real: Disaster Recovery and Business Continuity in Public Safety Is Yours Just a Paper Plan or a Real Way to Prepare and Respond to Incidents.

Similar presentations

Presentation on theme: "Let’s Get Real: Disaster Recovery and Business Continuity in Public Safety Is Yours Just a Paper Plan or a Real Way to Prepare and Respond to Incidents."— Presentation transcript:

1 Let’s Get Real: Disaster Recovery and Business Continuity in Public Safety Is Yours Just a Paper Plan or a Real Way to Prepare and Respond to Incidents and Disasters?

2 Presentation Overview Key DR/BC Concepts and Issues –Report card and dashboard –Scenarios –Requirements: What has to operational by when for work to be done by how many at what locations serving what customers who are where? –Facilities –People –Systems –Integration –Coordination –Daily readiness and simulated escalations –Testing and independent verification and validation –Implementation and triage –Recovery, discovery, and improvements Player Scorecard: Who Is In the Game and Why? DR/ BC Framework Action Steps to a Real Plan –First steps –Critical functions –Funding and leveraging scarce resources –Think out of the box –Integration with the big picture DR/BC plan and activities of your jurisdiction Conclusions

3 Key DR/BC Concepts and Issues

4 The Report Card and Dashboard All aspects of the plan, test, and implementation should be scored simply (Red, Yellow, and Green) Key indicators of planning and readiness need a dashboard to enable assessment and action –Score or status –Trend –Key issue

5 Public Safety Scenarios Public safety entities have a more difficult challenge Your IT DR/BC plan is intertwined with risk scenarios You may be affected by the risks of a given scenario and your IT plan must address those risks appropriately to maintain operations You also have a role in response to the scenario so the events will affect your operational requirements

6 Scenarios Overview Threat driven geographic circles of impact Kinds of threats and events Responsibility –What will you do, what is shared, what do others have to do for themselves Tolerance for risk and uncertainty Lesson learned: if you have a well known and documented local risk: –Have a real plan or get ready for a career change…

7 Source: IBM

8 Scenarios Identify Possible and Likely Natural Disasters and Environmental Conditions By Kind and Duration of Effects –Tornado –Hurricane –Tsunami –Flood –Snowstorm –Drought –Earthquake

9 Scenarios Identify Possible and Likely Natural Disasters and Environmental Conditions By Kind and Duration of Effects –Electrical storms –Fire –Subsidence and landslides –Freezing Conditions

10 Scenarios Identify Possible and Likely Natural Disasters and Environmental Conditions By Kind and Duration of Effects –Contamination, Toxic releases and environmental hazards –Epidemic –Pandemic –Animal or crop disease outbreak

11 Scenarios Organized and/or Deliberate Disruption –Act of terrorism WMD –Acute and short lived (bomb) –Acute and long lived (dirty bomb) –Chronic »Long term (contaminants and biohazards) »Permanent (radioactivity, etc.) WLD (suicide bombers, car bombs, utility sabotage) Bioterrorism or genetically modified or inorganic organisms –Direct contact –Infectious »Contact »Airborne

12 Scenarios Organized and/or Deliberate Disruption –Act of Sabotage –Product or food tampering –Act of war –Theft –Arson –Labor Disputes / Industrial Action

13 Scenarios Loss of Utilities and Services –Electrical power failure –Loss of gas supply –Loss of water supply –Petroleum and oil shortage Raw materials Refined materials –Communications services breakdown –Loss of drainage / waste removal and trash pickup

14 Scenarios Equipment or System Failure –Internal power failure –HVAC failure –Equipment failure (excluding IT hardware)

15 Scenarios Serious Information Security Incidents –Cyber crime –Malware –Zombie attacks –Denial of service –Loss or alteration of records or data –Disclosure of sensitive information

16 Scenarios IT system failure (local or hosted) –Hardware –Software Commercial application Locally developed application –Data –Communications

17 Scenarios Other Emergency Situations –Workplace violence –Public transportation disruption –Neighborhood hazard –Health and safety issues

18 Scenarios Multiple and compound hazards and events –Purposeful –Coincidental –Causally connected –Interrelated

19 IT Requirements What systems need to function How fast –Maximum and optimum time frame for each system or function to be restored How well –Sometimes minimal functionality is sufficient

20 IT Requirements Where will it be used and by whom and will the communications infrastructure support it? –Employees –Users or beneficiaries By what priority will systems be restored The priority will be modified by what contingencies –E.g. a long term total evacuation changes the operational needs for criminal justice systems and personnel

21 Facilities Hot, warm, cold Mirrored, recoverable, reload-able Properly located EOC Non-EOC Operational IT facilities For user interaction with IT systems

22 Facilities New kinds of mutual aid and sister city/county/state arrangements –Work with friends, colleagues, associations, and vendors –To match you with a comparable entities that are located outside the various geographic threat circles –Who can mirror your IT operations (hardware, software, operating systems, and culture)

23 People The right numbers, skills, location, redundancy, etc. –Skills and abilities inventory Employees Contractors Vendors Mutual aid and “the cavalry”

24 People Force in depth—who is the backup to the backup to the backup? Consider the actual health and physical abilities and disabilities of a person when assigning tasks for a disaster scenario –The disaster is not the time to find out the electrician in the hazmat suit has a heart condition What family and personal duties may interfere with performing official duties (e.g. save your own kids or save a stranger)?

25 Systems Daily operational Interdependent systems Emergency only Identity security and access management for physical and logical security –Follow FIPS 201 for federal/state/local interoperability

26 Integration With whom should you work closely? Identify integration issues between: –Internal systems and public safety entities –Other governmental systems –Related actors –Non-governmental systems and processes Example: 911 and 311or its equivalent –Normally separate but related –Emergencies blur the line –Co-location, cross training, and system integration

27 Coordination Within organization Within unit of government Across units of government Across levels of government Across public and private boundaries

28 Daily Readiness and Simulated Escalations A disaster a day (“What, that’s not normal?”) Realistic scenarios Captured lessons Learning and actually responding to lessons learned within risk framework A quality and security framework for daily operations has substantial overlap with DR/BC

29 Causes Effects Strategy Management Knowledge Support Technologies Security Leadership Security Sponsorship Security Strategy Security Program Security Program Structure Security Program Resources and Skillsets Security Policies Security Policies, Standard and Guidelines Security Management Security Administration Security Monitoring User Management User Awareness Information Asset Security Application Security Database / Information Security Host Security Internal Network Security Network Perimeter Security Technology Protection and Continuity Physical and Environment Controls Contingency Planning Controls Like similar capability models from the Carnegie Mellon SEI, SCMM models brings benefits: –Helps close security holes –Serves as a foundation for growth –Guides security leadership –Is evolutionary, not chaotic –Supports point solutions Security Capabilities Models KPMG SCMM Model

30 Capability Maturity Like the SCI CMM models, the KPMG Security Capability Model has five levels of maturity: Initial (1) Initial (1) Continuously improving process Repeatable (2) Repeatable (2) Defined (3) Defined (3) Managed (4) Managed (4) Optimizing (5) Optimizing (5) Predictable process Standard, consistent process Disciplined process Informal process

31 Testing and Independent Verification and Validation Does the planned response or action step actually work? Who verifies that it does? What do you do if it fails the test?

32 Implementation and Triage Someone better be in charge Dispute resolution processes Who will be your Sensibility and Sanity Checker (off site, not affected by the disaster, and actually getting enough sleep to make sound decisions)? Baton Rouge example with Mayor Holden

33 Recovery, Discovery, and Improvements What will the new normal be and when will it happen Learn from history, both recent and long past Document while the event occurs if at all possible (make it someone’s job) or soon after before memories fade

34 Player Scorecard Who Is In the Game and Why

35 Overlapping and Inter- Related Responsibilities Disaster Preparedness and Recovery and Business Continuity Quality Assurance Methodologies Cyber Security Physical Security Public Safety

36 The Usual Suspects in Public Safety Police Fire Other sworn officers (transit, game, building or branch based, etc.) National Guard Public Health Public Works Transportation Environmental Protection

37 The Usual Suspects in Emergency Management Federal, state and local emergency management entities National Guard NOAA, NWS, NSSL, other National Laboratories, Corps of Engineers

38 IT Entities CIO, CTO, and Enterprise IT Shops Distributed IT Departments and leadership Government IT contractors –DR/BC specific entities –Applications developers and software –Hardware –Service providers (ASP, MSP, call centers, etc. Communications providers

39 Policy Makers Executive, legislative, and judicial –Those who hold the seat and those who actually make the decisions… –Go below the top level to ensure clarity, alignment, and redundancy EOC designees Emergency authorizers

40 Non-Governmental Organizations Media –Broadcast and satellite Emergency Broadcast System Members –Print –New media The Web –Government site mangers –Commercial site managers –Citizens and bloggers –Self-organizing communities (e.g. Craig’s List)

41 Non-Governmental Organizations Charities Businesses and business associations Community organizations Vital private services (hospitals, nursing homes, etc. )

42 A DR/BC Framework

43 Business Operations and Technology Create a matrix, not a linear or organizational view Strategy Organization Processes Applications and data Technology Facilities

44 Source: IBM

45 Action Steps to a Real Plan

46 First Steps

47 Leadership: clarity, alignment, and commitment Authority or consensus? Stakeholders roles and responsibilities Be clear about risk tolerance Applications and IT assets inventory –If needed, dust off and update your Y2K work Good data on plan status, readiness, test results, response, and compliance

48 First Steps Make a friend in accounting—actuarially accurate threat scenarios are more likely to be funded as risk and cost can be properly balanced Review existing plan or make a plan Borrow or buy a template Review peer plans and conduct site visits Communicate until it hurts

49 Critical Functions

50 Nail Down Your Critical Functions Law and order essentials (people, mobility, tools, survival basics, etc.) Communications Personnel management (policies, scheduling, notification trees and systems, counseling, etc.) Data and the connections to data and people Transactional systems

51 Nail Down Your Critical Functions Rescue and response Pipeline to the health care system Building/location/hazmat information for fire and first responders Justice processing and incarceration Dispatch

52 Nail Down Your Critical Functions Records Mobility –Devices and local storage if communications are intermittent or fail (e.g. mobile maps and databases) Know what you can actually cover (and what you are just waiving your hands at and hoping it either works or is never needed)

53 Funding and Leverage

54 Work within your risk/threat/cost/benefit matrix and follow your own rules How serious are you about being prepared?

55 Funding and Leverage Stop building single purpose infrastructures and reuse what you have – “Ask not, what an infrastructure can do for you, but what it can do for your taxpayers” Use shared services Follow standards or help create them if lacking

56 Funding and Leverage Determine what pre-existing, unmet needs can be addressed by a new investment Determine whether existing public safety or enterprise systems will do the job and if you can use them Invest wisely –Vendors over inventors –COTS over customization –Web services over hard coding

57 Think Out of the Box

58 Think Third World Hand crank your computers Bike generators Solar and wind power Portable water purifiers Emergency shelter Runners and mountain bikes Hand tools

59 Think New World Internet Protocol (IP) everything –Bridge between radio, wireless data/WI-FI and use each as IP conduits as needed Gigs of portable flash memory Satellite data and telephony

60 Think New World Instant Message Text and mobile email Cell On Wheels/Boat/Balloon Negotiate/legislate priority and bumping rights in telecommunications provisioning

61 Integrate With the Big DR/BC Picture

62 The Big Picture Consult EM before, during, and after Once essential public safety systems have a DR/BC IT and overall plan it can be incorporated into the overall EM plan for the jurisdiction Tie it all together in formal and informal agreements Create a focal point such as your EOC

63 Not located in a hazard area (floodway) 500 square feet minimum floor space Communications section adjacent to EOC Three methods of communications with state EMA and local responders UPS and generator systems located above flood level Sleeping space for identified staff Kitchen space/food or meal contract New construction to International Building Code EOC Basics Source: Alabama EMD

64 Conclusion: Essential Public Safety Systems and Organizations Must Be Disaster Resistant, Flexible, Diversified, and Redundant (Or We Are All In Big Trouble) Contact Information Richard J. H. Varn Center for Digital Government

65 Model Plan Outline What follows is a private sector based, but broadly applicable tool that sells for $199 To buy a copy of the business continuity plan generator see http://www.eon-

66 Model Plan Outline Business Continuity - Preparing the Plan Initiating the BCP Project Project Initiation Activities BC 010101 Review of Existing BCP (if available)

67 Model Plan Outline BC 010102 Benefits of Developing a BCP BC 010103 BCP Policy Statement BC 010104 Preliminary BCP Project Budget BC 010105 Procedure for Approving BCP Content

68 Model Plan Outline BC 010106 Communication on BCP Project to All Employees Project Organization BC 010201 Terms of Reference for BCP Project Manager BC 010202 Appoint BCP Project Manager and Deputy BC 010203 Select and Notify BCP Project Team

69 Model Plan Outline BC 010204 Initial BCP Project Meeting BC 010205 Project Objectives and Deliverables BC 010206 Project Milestones BC 010207 Project Reporting Requirements and Frequency BC 010208 Required Documents and Information

70 Model Plan Outline Assessing Business Risk and Impact of Potential Emergencies Emergency Incident Assessment BC 020101 Environmental Disasters BC 020102 Organized and / or Deliberate Disruption

71 Model Plan Outline BC 020103 Loss of Utilities and Services BC 020104 Equipment or System Failure BC 020105 Serious Information Security Incidents BC 020106 Other Emergency Situations Business Risk Assessment

72 Model Plan Outline BC 020201 Key Business Processes BC 020202 Establish Time-Bands for Business Service Interruption Measurement BC 020203 Financial and Operational Impact IT and Communications

73 Model Plan Outline BC 020301 Specifications of IT and Communication Systems and Business Dependencies BC 020302 Key IT, Communications and Information Processing Systems BC 020303 Key IT Personnel and Emergency Contact Information BC 020304 Key IT and Communications Suppliers and Maintenance Engineers BC 020305 Existing IT Recovery Procedures

74 Model Plan Outline Existing Emergency Procedures BC 020401 Summary of Existing Procedures for Handling Emergency Situations BC 020402 Key Personnel Responsible for Handling Existing Emergency Procedures BC 020403 External Emergency Services and Contact Numbers

75 Model Plan Outline BC 020500 Premises Issues BC 020501 Responsibility and Authority for Building Repairs BC 020502 Back-up Power Arrangements Preparing for a Possible Emergency

76 Model Plan Outline Back-up and Recovery Strategies BC 030101 Alternative Business Process Handling Strategy BC 030102 IT Systems Back-Up and Recovery Strategy BC 030103 Premises and Essential Equipment Back-up and Recovery Strategy

77 Model Plan Outline BC 030104 Customer Service Back-up and Recovery Strategy BC 030105 Administration and Operations Back-up and Recovery Strategy BC 030106 Information and Documentation Back-up and Recovery Strategy BC 030107 Insurance Coverage Key BCP Personnel and Supplies

78 Model Plan Outline BC 030201 Functional Organization Chart BC 030202 BCP Project Co-coordinator and Deputy for Each Functional Area BC 030203 Key Personnel and Emergency Contact Information BC 030204 Key Suppliers and Vendors and Emergency Contact Information BC 030205 Manpower Recovery Strategy

79 Model Plan Outline BC 030206 Establishing the Disaster Recovery Team BC 030207 Establishing the Business Recovery Team Key Documents and Procedures BC 030301 Documents and Records Vital to the Business Process BC 030302 Off-site Storage

80 Model Plan Outline BC 030303 Emergency Stationery and Office Supplies BC 030304 Media Handling Procedures BC 030305 Emergency Authorization Procedures BC 030306 Prepare Budget for Back-up and Recovery Phase

81 Model Plan Outline Disaster Recovery Phase Planning for Handling the Emergency BC 040101 Identification of Potential Disaster Status BC 040102 Involvement of Emergency Services BC 040103 Assessing Potential Business Impact of the Emergency

82 Model Plan Outline BC 040104 Project Management Activities Notification and Reporting During Recovery Phase BC 040201 Mobilizing the Recovery Team BC 040202 Notification to Management and Key Employees

83 Model Plan Outline BC 040203 Handling Personnel Families Notification BC 040204 Handling Media during the Disaster Recovery Phase BC 040205 Maintaining Event Log during Disaster Recovery Phase BC 040206 Disaster Recovery Phase Report Business Recovery Phase

84 Model Plan Outline Managing the Business Recovery Phase BC 050101 Mobilizing the Business Recovery Team BC 050102 Assessing Extent of Damage and Business Impact BC 050103 Preparing Specific Recovery Plan

85 Model Plan Outline BC 050104 Monitoring Progress BC 050105 Keeping Everyone Informed BC 050106 Handing Business Operations Back to Regular Management BC 050107 Preparing Business Recovery Phase Report Business Recovery Activities

86 Model Plan Outline BC 050201 Power and Other Utilities BC 050202 Premises, Fixtures and Furniture (Facilities Recovery Management) BC 050203 Communication Systems BC 050204 IT Systems (Hardware and Software)

87 Model Plan Outline BC 050205 Production Equipment BC 050206 Other Equipment BC 050207 Warehouse and Stock BC 050208 Trading, Sales and Customer Service

88 Model Plan Outline BC 050209 Human Resources BC 050210 Information and Documentation BC 050211 Office Supplies BC 050212 Operations and Administration (Support Services)

89 Model Plan Outline Testing the Business Recovery Process Planning the Tests Develop Objectives and Scope of Tests Setting the Test Environment Environmental Disasters

90 Model Plan Outline Organized and / or deliberate disruption Loss of Utilities and Services Equipment or System Failure Serious Information Security Incidents Other Emergency Situations Prepare Test Data Identify Who is to Conduct the Tests

91 Model Plan Outline Identify Who is to Control and Monitor the Tests Prepare Feedback Questionnaires Prepare Budget for Testing Phase Training Core Testing Team for each Business Unit

92 Model Plan Outline Conducting the Tests Test each part of the Business Recovery Process Test Accuracy of Employee and Vendor Emergency Contact Numbers Assess Test Results Training Staff in the Business Recovery Process

93 Model Plan Outline Managing the Training Process Develop Objectives and Scope of Training Training Needs Assessment Training Materials Development Schedule Prepare Training Schedule Communication to Staff Prepare Budget for Training Phase Assessing the Training

94 Model Plan Outline Feedback Questionnaires Assess Feedback Keeping the Plan Up-to-date Maintaining the BCP

95 Model Plan Outline Change Controls for Updating the Plan Responsibilities for Maintenance of Each Part of the Plan Test All Changes to Plan Advise Person Responsible for BCP Training

Download ppt "Let’s Get Real: Disaster Recovery and Business Continuity in Public Safety Is Yours Just a Paper Plan or a Real Way to Prepare and Respond to Incidents."

Similar presentations

Ads by Google