1. Principles of Incident Response and Disaster Recovery
Chapter 7
Disaster Recovery: Preparation and Implementation

2. Objectives
Understand the ways to classify disasters, both by speed of onset and source
Know who should form the membership of the disaster recovery team
Understand the key functions of the disaster plan
Explain the key concepts included in the NIST approach to technical contingency planning
Describe the elements of a sample disaster recovery plan
Principles of Incident Response and Disaster Recovery

3. Objectives (continued)
Understand the need for simultaneous wide access to the planning documents as well as the need for securing the sensitive content of the DR plans
Principles of Incident Response and Disaster Recovery

4. Introduction
Disaster recovery planning: preparation for and recovery from a disaster
Disaster may be an escalated incident or may be immediately classified as a disaster
In general, a disaster is an incident that cannot be contained or whose impact is not controllable
All business units of an organization need to be involved in disaster recovery planning, not just IT
Principles of Incident Response and Disaster Recovery

5. Disaster Classifications
Disasters can be classified by cause:
Man-made: war, terrorism, cyberterrorism, etc.
Natural: fire, flood, earthquake, hurricane, lightning, tornado, etc.
Disasters can be classified by speed of development:
Rapid onset: occur suddenly with little warning
Slow onset: occur over time and deteriorate the capacity of the organization to withstand
Principles of Incident Response and Disaster Recovery

6. Disaster Classifications (continued)
Principles of Incident Response and Disaster Recovery

7. Disaster Classifications (continued)
Principles of Incident Response and Disaster Recovery

8. Forming the Disaster Recovery Team
Disaster recovery team is assembled by the CPMT
Should include members from IT, InfoSec, and other departments
DR team is responsible for planning for DR and for leading the DR process when a disaster is declared
Must consider the organization of the DR team and the needs for documentation and equipment
Principles of Incident Response and Disaster Recovery

9. Organization
DR team
Should include representatives from every major organizational unit
Should be separate from other contingency-related teams
May include senior management, corporate support units, facilities, fire and safety, maintenance, IT, InfoSec
May be advisable to divide the team up into subteams
Principles of Incident Response and Disaster Recovery

10. Organization (continued)
Subteams may include:
Disaster management team: command and control, responsible for planning and coordination
Communications: public relations and legal representatives to interface with senior management and general public
Computer recovery (hardware): recovers physical computing assets
Systems (OS) recovery: recovers operating systems
Network recovery: recovers network wiring and hardware
Principles of Incident Response and Disaster Recovery

11. Organization (continued)
Subteams (continued):
Storage recovery: recovers storage area networks and network attached storage
Applications recovery: recovers applications and reintegrates users back into the systems
Data management: recovers and restores data
Vendor contact: works with suppliers and vendors to replace damaged or destroyed materials, equipment, or services
Damage assessment and salvage: provides initial assessments of damage and recovers salvageable items
Principles of Incident Response and Disaster Recovery

12. Organization (continued)
Subteams (continued):
Business interface: works with remainder of organization to assist in recovery of non-technology functions
Logistics: provides supplies, space, materials, food, services, or facilities needed at the primary site
Other teams needed to reestablish key business functions as needed
Principles of Incident Response and Disaster Recovery

13. Special Documentation and Equipment
All team members
Should have multiple copies of the DR and BC plans at home and office for immediate use when disaster occurs
Should have access to certain disaster recovery materials, including software, hardware, building blueprints, key phone numbers, emergency supplies, etc.
Principles of Incident Response and Disaster Recovery

14. Disaster Planning Functions
Guidelines are found in NIST Contingency Planning Guide for Information Technology Systems
Planning process steps:
Develop the DR planning policy statement
Review the business impact analysis (BIA)
Identify preventive controls
Develop recovery strategies
Develop the DR plan document
Test, train, and rehearse
Plan maintenance
Principles of Incident Response and Disaster Recovery

15. Develop the DR Planning Policy Statement
DR policy should contain these key elements:
Purpose
Scope
Roles and responsibilities
Resource requirements
Training requirements
Exercise and testing schedules
Plan maintenance schedules
Special considerations
Principles of Incident Response and Disaster Recovery

16. Develop the DR Planning Policy Statement (continued)
Purpose:
Provide for the direction and guidance of any and all DR operations
Must include executive vision and commitment
Business disaster recovery policy should apply to the entire organization
Scope:
Identifies the organizational units and groups of employees to which the policy applies
Roles and responsibilities:
Identifies the key players and their responsibilities
Principles of Incident Response and Disaster Recovery

17. Develop the DR Planning Policy Statement (continued)
Resource requirements:
Identifies any specific resources to be dedicated to the development of the DR plan
Training requirements:
Details training related to the DR plan
Exercise and testing schedules:
Specifies the frequency of testing of the DR plan
Plan maintenance schedules:
Details the schedule for review and update of the plan
Principles of Incident Response and Disaster Recovery

18. Develop the DR Planning Policy Statement (continued)
Special considerations:
May include issues such as information storage and retrieval plans, off-site and on-site backup schemes, or other issues
Principles of Incident Response and Disaster Recovery

19. Review the Business Impact Analysis
Review the BIA within the DR context
Ensure that the BIA is compatible with the DR specific plans and operations
BIA is usually acceptable as it was prepared and released by the CPMT
Principles of Incident Response and Disaster Recovery

20. Identify Preventive Controls
This function should have already been performed as part of ongoing information security posture
DP team should review and verify that data storage and recovery techniques are implemented, tested, and maintained
Principles of Incident Response and Disaster Recovery

21. Develop Recovery Strategies
May be impossible to prepare for all diverse contingencies, but recovery strategies should be in place for the most likely disasters
DR strategies:
Go substantially beyond the recovery portion of database backup and recovery
Must include the steps to fully restore the operational status of the organization
Includes personnel, equipment, applications, data, communications, and support services (power, water, etc.)
Principles of Incident Response and Disaster Recovery

22. Develop Recovery Strategies (continued)
DR strategies must include the enlistment and retention of qualified general contractors capable of assessing damage and rebuilding the facility
May want to include the general contractor in the DR training and rehearsals
If the primary site is a leased facility, include the leasing agency
Principles of Incident Response and Disaster Recovery

23. Develop the DR Plan Document
DR planning document should contain specific and detailed guidelines and procedures for restoring lost or damaged capabilities
Steps:
DR team takes the IR plan and converts incidents to disasters
DR team adds additional disasters not in the IR document, and creates disaster scenarios
DR team develops 3 sets of activities for each scenario
Activities during the disaster are placed first, then follow-up activities, and finally occasional activities
Principles of Incident Response and Disaster Recovery

24. Develop the DR Plan Document (continued)
Procedures during the disaster:
Procedures that must be performed during the disaster, if any
Grouped and assigned to individuals
May include evacuation plans, locations of shelters, fire suppression systems, other emergency reaction items
Must be readily available for use during a disaster
Procedures after the disaster:
Procedures performed immediately after
May include crisis management procedures
Principles of Incident Response and Disaster Recovery

25. Develop the DR Plan Document (continued)
Before the disaster:
Procedures to prepare for the disaster
May include data backup, disaster recovery preparation, training schedules, testing plans, copies of service agreements, business continuity plans, etc.
DR addendums
One for each type of anticipated disaster
Includes the trigger, notification method, response time
Principles of Incident Response and Disaster Recovery

26. Develop the DR Plan Document (continued)
Principles of Incident Response and Disaster Recovery

27. Develop the DR Plan Document (continued)
Trigger: point at which a management decision to react is made
Planning for actions taken during the disaster:
Most important part is planning the actions before phase
Should create reaction scenarios
Planning for events occurring after the disaster:
Includes recovery operations, identification of potential follow-on attacks, and forensics analysis
Must conduct an action-after review (AAR)
Principles of Incident Response and Disaster Recovery

28. Develop the DR Plan Document (continued)
Forensics analysis: process of systematically examining information assets for evidentiary material that can provide insight into the cause
After-action review (AAR): detailed examination of the events that occurred from detection to final recovery
Planning for actions taken before the disaster:
Includes preventive controls, risk management, team preparedness, stocking of critical consumables, execution of service and support contracts
Principles of Incident Response and Disaster Recovery

29. Plan Testing, Training, and Exercises
Training can be used to test the validity and effectiveness of the DR plan
Testing should be an ongoing activity, at least semiannually at the walk-through level
Final assembly of the DR plan can take place after testing and training
Principles of Incident Response and Disaster Recovery

30. Plan Maintenance
Plan must be a dynamic document that is updated regularly
Revisit the DR plan at least annually to update plans, contracts, and agreements
Make necessary personnel and equipment modifications
Any change in the organization’s size, location, or business focus must be incorporated into the DR and CP plans, and the BIA should also be reviewed
Principles of Incident Response and Disaster Recovery

31. Technical Contingency Planning Considerations
Technical contingency planning is based on the type of IT platforms:
Desktop computers and portable systems
Servers
Web sites
Local area networks
Wide area networks
Distributed systems
Mainframe systems
Principles of Incident Response and Disaster Recovery

32. Technical Contingency Planning Considerations (continued)
For each platform type, two perspectives are considered:
Technical requirements that should be considered, including preventive and recovery measures
Technology-based solutions that may be used
Some contingency measures are common to all IT systems
Principles of Incident Response and Disaster Recovery

33. Technical Contingency Planning Considerations (continued)
Common considerations include:
Frequency of backup and off-site storage of data, applications, and operating systems
Redundancy of critical system components
Documentation of system configurations and requirements
Interoperability between system components and between primary and alternate site equipment to expedite system recovery
Appropriately sized and configured power management systems and environmental controls
Principles of Incident Response and Disaster Recovery

34. Desktop Computers and Portable Systems
Contingency considerations should emphasize data availability, confidentiality, and integrity
Should consider these practices:
Store backups off-site
Encourage individuals to back up data
Provide guidance on saving data on PCs
Standardize hardware, software, and peripherals
Document system configuration and vendor information
Coordinate with security policies and controls
Use results from BIA
Principles of Incident Response and Disaster Recovery

35. Desktop Computers and Portable Systems (continued)
Contingency strategies may include:
Document system configuration and vendor information
Standardize hardware, software, and peripherals
Provide guidelines on backing up data
Ensure interoperability among components
Coordinate with security policies and controls
Backup applications and store off-site
Use alternate hard drives
Image disks and standardize images
Principles of Incident Response and Disaster Recovery

36. Desktop Computers and Portable Systems (continued)
Contingency strategies (continued):
Implement redundancy in critical system components
Use uninterruptible power supplies
Principles of Incident Response and Disaster Recovery

37. Servers Address server vulnerabilities by considering these practices:
Store backup media and software off site
Standardize hardware, software, and peripherals
Document system configuration and vendor information
Coordinate with security policies and controls
Use results from BIA
Principles of Incident Response and Disaster Recovery

38. Servers (continued) Contingency strategies may include:
Document system configuration and vendor information
Standardize hardware, software, and peripherals
Coordinate with security policies and controls
Ensure interoperability among components
Backup data and store off-site
Use uninterruptible power supplies
Implement redundancy in critical system components
Principles of Incident Response and Disaster Recovery

39. Servers (continued) Contingency strategies (continued):
Implement fault tolerance in critical system components
Replicate data
Implement storage solutions
Principles of Incident Response and Disaster Recovery

40. Web Sites
In addition to information about servers, these practices should be considered:
Document Web site
Web site programming should use documented change management
Web site coding should be relative, not absolute, allowing quick reconfiguration if needed
Coordinate contingency solutions with appropriate security policies and controls
Coordinate contingency solutions with incident response procedures
Use results from BIA
Principles of Incident Response and Disaster Recovery

41. Web Sites (continued) Contingency strategies may include:
Document Web site
Code, program, and document Web site properly
Coordinate with security policies and controls
Consider contingencies of supporting infrastructure
Implement load balancing
Coordinate with incident response procedures
Principles of Incident Response and Disaster Recovery

42. Local Area Networks Consider the following practices:
Physical and logical LAN should be well documented
System configuration and vendor information should be well documented
Coordinate with security policies and controls
Use results from BIA
Identify single points of failure that affect critical systems or processes outlined in the BIA
Identify threats to the cabling system such as cable cuts, electromagnetic and radio frequency interference, and damage from fire, water, and other hazards
Principles of Incident Response and Disaster Recovery

43. Local Area Networks (continued)
Contingency strategies may include:
Document the LAN
Coordinate with vendors
Coordinate with security policies and controls
Identify single points of failure
Implement redundancy in critical components
Monitor the LAN
Integrate remote access and wireless area network technology
Principles of Incident Response and Disaster Recovery

44. Wide Area Networks Consider the following practices:
Physical and logical LAN should be well documented
System configuration and vendor information should be well documented
Coordinate with security policies and controls
Use results from BIA
Principles of Incident Response and Disaster Recovery

45. Wide Area Networks (continued)
Contingency strategies may include:
Document the WAN
Coordinate with vendors
Coordinate with security policies and controls
Identify single points of failure
Implement redundancy in critical components
Institute service-level agreements
Principles of Incident Response and Disaster Recovery

46. Distributed Systems Consider the following practices:
Standardize hardware, software, and peripherals
Document system configuration and vendor information
Coordinate with security policies and controls
Use results from the BIA
Principles of Incident Response and Disaster Recovery

47. Distributed Systems (continued)
Contingency strategies may include:
Standardize components
Document system
Coordinate with vendors
Coordinate with security policies and controls
Consider server contingency solutions
Consider LAN contingency solution
Consider WAN contingency solution
Principles of Incident Response and Disaster Recovery

48. Mainframe Systems Consider the following practices:
Store backup media off site
Document system configurations and vendors
Coordinate with network security policies and system security controls
Use results from the BIA
Principles of Incident Response and Disaster Recovery

49. Mainframe Systems (continued)
Contingency strategies may include:
Backup data and store off site
Document system
Coordinate with vendors
Coordinate with security policies and controls
Implement redundancy and fault tolerance in critical system components
Consider hot site or reciprocal agreement
Institute vendor service-level agreements (SLAs)
Replicate data
Implement storage solutions
Use uninterruptible power supplies
Principles of Incident Response and Disaster Recovery

50. Summary of Technical Contingency Planning Considerations
Principles of Incident Response and Disaster Recovery

51. Summary of Technical Contingency Planning Considerations (continued)
Principles of Incident Response and Disaster Recovery

52. Sample Disaster Recovery Plans
Principles of Incident Response and Disaster Recovery

53. Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recovery

54. Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recovery

55. Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recovery

56. Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recovery

57. Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recovery

58. The Combined DR Plan/BC Plan
Many organizations prepare DR and BC plans at the same time and combine them into a single plan
Must be able to support reestablishment of operations at two different locations:
Immediately at an alternate site
Eventually back at the primary site
Execution of a combined plan requires separate execution teams
Principles of Incident Response and Disaster Recovery

59. Final Comments on the DR Plan
Planning process for the DR plan/BC plan should be tied to, but distinct from, the IR plan
These 3 processes should be tightly integrated to allow reaction teams to easily transition from incident response to disaster recovery and business continuity planning
Appendix B contains a sample NIST contingency plan
Remember to keep the plan available but secure
Principles of Incident Response and Disaster Recovery

60. Summary
DR planning is the preparation for and recovery from a disaster
Disasters can be classified by source (natural or man-made) or by speed of development (rapid onset or slow onset)
CPMT assembles the DR team, consisting of representatives from every major organizational unit
Members of the DR team do not serve on IR or BC team because of overlapping duties
DR team may consist of many subteams
Principles of Incident Response and Disaster Recovery

61. Summary (continued)
All members of DR team should have multiple copies of the DR and BC plans available to them at home and office
DR policy is the first deliverable
Effective preventive controls implemented for security also facilitate recovery of information
DR plan should contain detailed procedures for restoring lost or damaged information, in 3 phases:
During the disaster
After the disaster
Before the disaster
Principles of Incident Response and Disaster Recovery

62. Summary (continued)
Training in the use of the DR plan can be used to test the validity and effectiveness of the plan
Testing of the plan is an ongoing activity, with each scenario tested at least semiannually at the walk-through level
Principles of Incident Response and Disaster Recovery