Presentation is loading. Please wait.

Presentation is loading. Please wait.

Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE.

Similar presentations


Presentation on theme: "Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE."— Presentation transcript:

1 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE

2 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Realizing the Need In June, 2004, the Department of State (CDOS) experienced a significant interruption in its data center services. Following an extensive outage, testing revealed the presence of “zinc whiskers”, zinc fibers that can be extruded over time by electroplated metal coatings, such as the coating applied on the electrostatic-dissipating raised floor tiles used in the CDOS data center. The facility was rendered unusable for computer operations. The Department’s equipment had been internally contaminated and as a result was subject to additional disruptions. This catastrophic event spotlighted a paramount need for an achievable business continuity and disaster recovery capability.

3 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Disaster Recovery Survey In March of 2005 the Colorado Information Management Commission (IMC) through the Governors Office of Innovation and Technology (OIT) conducted a survey of the State Agency disaster recovery preparedness. The survey found that:  Of all 23 state agencies surveyed, none reported complete IT disaster recovery capability.  Risk was high for all State agencies because disaster recovery capabilities were low.  Threats were rising & the risk profile was getting worse.  Majority of agencies were planning and/or seeking individualized rather than collaborative disaster recovery investments. An overwhelmingly common misconception about disaster recovery planning existed where it was thought that having a backup strategy, doing backups everyday and running anti-virus software was the same as having a Disaster Recovery Plan.

4 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Strategic Solution The Colorado Legislature approved a $3.6 million appropriation with $2.3 million in continuation funds from the CDOS cash fund to build and maintain a disaster recovery facility. Colorado Department of State, with the support of the Governor’s Office of Innovation and Technology successfully implemented a strategic solution for disaster recovery known as the enterprise Facility for Operational Recovery/Readiness/Response and Transition services, ( “e-FOR 3 T”.) The e-FOR 3 T initiative established a statewide disaster recovery facility for use by state agencies and government entities. The facility is operated under the provisions of a cooperative agreement between the Colorado Secretary of State's Office and participating agencies. The facility allows agencies to recover critical functionality in the event of a disaster or an event resulting in a significant interruption of service by:  Providing a tier 3+ IT facility for operational recovery  Promoting economies of scale through uniformity of service & infrastructure  Providing participating member agencies with a cost effective & reliable level of expected service

5 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE E-For 3 t Disaster Recovery Facility  Enterprise Solution – Uniform application of best practices for operational recovery, readiness & response  Funding Strategy – Colorado Department of State utilizing its cash funds to maintain portions of the disaster recovery infrastructure that are common in facility infrastructure & operations  Participating Member Agency Flexibility – Allows the flexibility to architect specific required recovery solutions  All State of Colorado Government Entities Invited – Participation is voluntary for any of these entities  Facility Infrastructure Components – Power, connectivity, and 12,167 sq ft of space to house equipment and personnel Enterprise Facility for Operational Recovery Readiness Response & Transition services Addresses the following strategic objectives by providing

6 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE e-FOR 3 T Specifications Data Center Space  6,549 sq ft raised floor  Individually locking (front and rear) cabinets  Private cages as required Migration & Transition Support  Staffing and resources for physical migration  Documented processes and procedures  Relocation and standup of state agency hardware  Review post-transition analysis internally and externally  Off-site tape vaulting

7 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE e-FOR 3 T Specifications Office Space  5,618 sq ft fully furnished (desks, workstations, phones, etc) Security  24 x 7 magnetic card key access with secondary pin code  Digital motion activated security cameras and intercoms Cable Management  Provide all cabling for each agency deployment  Cabling tagged and documented

8 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE e-FOR 3 T Specifications Network Operations Center – Telecom & Network Support  MNT network services provided on dual path system with redundant IP router & LAN switches  20 remote access internet accounts with static IP addresses for each participating member agency (10 DSL & 10 dial up)  Remote hands services allowing State to use facility personnel to perform basic tasks as instructed by State engineers  Insurance coverage for State assets  MNT network failover to redundant IP network  Private Network Transit (PNT) via backbone connection between Denver and alternate site

9 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE e-FOR 3 T Specifications Power  Facility power, delivery, infrastructure, generator and fuel  UPS systems and megawatt diesel generators Cooling  HVAC service to support requirements  Anti-static raised floor with cooled air delivery Fire Detection & Suppression  Certified incipient smoke detection (VESDA)  Dual-Interlock Pre-Action dry pipe sprinkler system

10 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE

11 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Continuity of Operation Plan The Governor’s Office of Information Technology (OIT) provides the leadership in planning for recovery of State of Colorado functions after a man-made or natural disasters through their Continuity of Operation/Continuity of Government Program. The Program Manager assists State of Colorado agencies in developing COOP plans using Federal standards for content per FPC 65. Internet-based LBL Contingency Planner software is used to create and update COOP documentation. The templates included in the Contingency Planner software include suggested procedures for carrying out a disaster recovery plan. Following the federal model, these plans will address identification and prioritization of agency essential functions, succession, delegation of authority, interoperable communications, identification of alternate facilities, identification of vital records and databases, human capital aspects, reconstitution of facilities and the testing, training and exercising of these plans. Each program within CDOS was tasked with providing specific details regarding their operations. By creating documents with possible scenarios agencies were able to determine how specific needs would change based on the circumstances.

12 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE

13 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE

14 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE

15 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE

16 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Scenario I – SOS and Other Agencies Affected/Internet Functional A.Assumptions: 1.Activation of the COOP is the result of a disaster that affects more than the Secretary of State. 2.The Secretary of State’s office is uninhabitable. 3.Other affected agencies will want to file emergency rules with Secretary of State. 4.Internet and SOS systems operational at e-For3t within 36 to 48 hours after COOP activation. 5.IT has implemented VPN so that employees can work from home rather than at e-For3t. B.Critical functions: In a normal work environment rulemaking agencies make all filings electronically, paper filings are not accepted. With the current procedures, all electronically filed rules need to be reviewed and the formatting needs to be modified by the Administrative Rules Staff in order to be published online. This process requires extensive review and formatting changes by the Administrative Rules Staff to ensure that documents are compatible with the online publishing system. Normally, the rulemaking process is spread out over a number of months and many agencies file infrequently. Only a handful of agencies file every month. In an emergency situation, many more agencies may be forced to adopt rules to accommodate the situation. The additional load on the system may create problems and the total volume of rules will necessarily increase but the amount is unknown.

17 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Scenario I – Continued C.Minimum personnel: 1.Program Mgr 2.Program Asst II 3.Administrative Asst III D.Resource Requirements: 1.Personal desktops, laptops or laptops provided by SOS. 2.Access to high speed Internet lines (May require subsidization of additional costs to employee). 3.VPN connectivity. 4.Telephone. 5.Scanner. E.Space Requirements: 1.None – employees work from home. F.Travel costs: 1.Program Mgr may be required to travel from home to eFort on occasional basis.

18 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Scenario II – SOS and Other Agencies Affected/Internet Nonfunctional A.Assumptions: 1.Activation of the COOP is the result of a disaster that affects more than the Secretary of State. 2.The Secretary of State’s office is uninhabitable. 3.Other affected agencies will want to file emergency rules with Secretary of State. 4.Internet is NOT functional but e-For 3 t is operational with an Intranet capability. 5.IT has NOT implemented VPN. Employees cannot work from home. B.Critical functions: The normal process of electronic filing of administrative rules is disrupted. Agencies must default to the previous procedure of delivering rules directly to the Secretary of State. Although the Internet is not working, the Intranet will allow for rules to be entered into the database for use when the Internet becomes available. Emergency rules will be the major portion of the workload with normal rules making up the balance. In this scenario, mail or hand carried documents must be delivered to a single location. The Administrative Rules program will have to receive the documents (either in electronic or paper format, or both), date/time stamp them and then process them into the system. For an interim period the official register will be published in print rather than electronically. Materials will be sent electronically or mailed to the outside vendor that produces the Colorado Register in print.

19 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Scenario II – Continued C.Minimum Personnel: 1.Program Mgr. 2.Program Asst. I 3.Admin. Asst. III D.Resource Requirements: 1.Computer workstations with Intranet access to the database for three individuals. 2.Telephone (at least one). 3.Copy machine. 4.Fax machine. 5.Scanner. E.Space Requirements: 1.Three workstations at eFort. One option would be to coordinate with the Arapahoe County Clerk’s office and use it as the mail reception point and initial processing site. That could eliminate the need for two of the three workstations at eFort. F.Travel Costs: 1.Staff will be required to work from an alternate location.

20 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Scenario III – Only SOS Affected/Internet Functional A.Assumptions: 1.Activation of the COOP is the result of a disaster that only affects the Secretary of State. 2.The Secretary of State’s office is uninhabitable. 3.Agencies will only need to file rules in the normal course. These may be regular or emergency rules but the situation is no different than normal operations. 4.Internet and SOS systems operational at e-For 3 t within 36 to 48 hours after COOP activation. 5.IT has implemented VPN so that employees can work from home rather than at e-For 3 t. B.Critical functions: In a normal work environment rulemaking agencies make all filings electronically, paper filings are not accepted. With the current procedures, all electronically filed rules need to be reviewed and the formatting needs to be modified by the Administrative Rules Staff in order to be published online. This process requires extensive review and formatting changes by the Administrative Rules Staff to ensure that documents are compatible with the online publishing system. In this circumstance, it will more or less be business as usual for the Administrative Rules Program. The only thing different will be that staff will either work from home or work at e-For 3 t.

21 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Scenario III – Continued C.Minimum Personnel: 1.Program Mgr. 2.Program Asst. I 3.Admin. Asst. III D.Resource Requirements: 1.Personal desktops, laptops or laptops provided by SOS. 2.Access to high speed Internet lines (May require subsidization of additional costs to employee). 3.VPN connectivity. 4.Telephone. 5.Scanner. E.Space Requirements: 1.Personnel could work from home per Scenario I or work at eFort per Scenario II. F.Travel Costs: 1.Staff may need to work from an alternate location.

22 Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE Links to COOP Info  Continuity of Operation Planning/ Continuity of Government  enterprise Facility for Operational Recovery Readiness Response & Transition (e-FOR 3 T)  LBL Technology Partners Contingency Planner software


Download ppt "Colorado Secretary of State EMERGENCY PREPARATION AND RESPONSE."

Similar presentations


Ads by Google