1. Disaster Recovery and Business Continuity Planning in a University Environment Mardecia Bell Ann Harris Copyright Mardecia Bell/Ann Harris 2005. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors.

2. The realization of a single point of failure with one data center for both the central academic and administrative IT environments, prompted NC State University to implement a disaster recovery strategy for communications and critical applications residing on the mainframe & open systems computing environment.

3. History/Timeline 1997 Initiated with the administrative environment Mainframe environment recovery test 1999 Y2K - Business Continuity concept Acquired central repository software (LDRPS) 2001 Scheduled annual Mainframe recovery test Included communications & academic environment 2002Expanded to include Enterprise Business Continuity/Disaster Recovery Planning 2004Successful DR test of ERP systems 2005Co-processing of production services began in Data Center II

4. Implementation Steps Gain Sponsorship Establish Steering Committees Develop University Policy/Regulation Create DR Structure/Establish Staffing Market Program Establish Central Repository Review & Test Plans Regularly

5. Gain Sponsorship Office of the President – University System Chancellor Executive Management –Present your Business Case –Identify the roles involved –Provide Executive Summary of BC/DR Program –Present Statement of Work and Project Plan Add responsibilities to staff work plans

6. Establish Steering Committees IT Steering Committee Business/Service Steering Committee Both committees are comprised of –Vice Chancellor/Vice Provost Level –Representatives from Critical Areas of the Campus –Ex Officio members from IT areas Mission of IT Steering Committee –Provide guidance and oversight for the combined academic and administrative Disaster Recovery Plan.

7. Policy/Regulations/Rule Develop a Policy or Regulation to affirm the mandate and promote cooperation

8. Divide Campus Into Groupings Space/Facilities Teaching and Academic Programs Academic IT Administrative IT Environmental Health and Public Safety Business Administration Research Programs Student Affairs Extension and Engagement

9. Resource Projections Hire Full-Time Business Continuity and Disaster Recovery Personnel –Director of Business Continuity (plus 1 Business Analyst) –Admin IT DR Coordinator (plus 1 Business Analyst) –Academic DR Coordinator (part-time) Add BC/DR responsibilities to work plan of existing staff Identify Coordinators for each business unit

10. Marketing Present at campus departmental meetings Create a Website Utilize listserves Campus Newspaper Network with peer institutions Remain abreast of industry standards Attend conferences, workshops and seminars

11. Establish Central Information Repository Continuous Implementation

12. Accomplishments Disaster Recovery and Business Continuity Plan Risk Assessments for Critical Business Units Successful Mainframe Recovery Tests Designed and implemented infrastructure for central computing environment (academic & administrative) in secondary data center. Implementation of recovery strategies in secondary data center Creation of Administrative IT Disaster Recovery Unit

13. Illustration of Various DR Deployments  Fault-tolerant cluster (file and print services) A Production B Configuration B Production A Configuration B Production A Production  Distributed deployment (hosted systems) A Production A Development A Production  Co-processing and load-balancing (ERP) A Production  Data replication (mainframe) Server Data Server Data Server Data

14. Enterprise Resource Planning (ERP) Deployment DC II  Financial System  Human Resources (Version 8.8)  Student Information System (under construction) DC I Web Server DB Server Application Server Batch Server Campus Users Web Server Application Server Batch Server Web Server Application Server Web Server Application Server Batch Server DB Server Batch Server Data Storage Area Network

15. Summary and Future Steps DC II Hosted systems Infrastructure Data Storage Area Network Active Directory / Windows Novell Directory Services / Novell Citrix ERP Web ERP Batch ERP Application Data Backup/vaulting ERP DB Server DC I Hosted systems Infrastructure Data Storage Area Network Back up/vaulting Active Directory / Windows Novell Directory Services / Novell Citrix ERP Web ERP Batch ERP DB Server ERP Application Development Server Mainframe Server Email/Calendar Anti-SPAM File/Print, User Home Web Server Database Server Development Server Mainframe Server Web Server Database Server Data Storage Area Network Data Email/Calendar Anti-SPAM File/Print, User Home

16. Administrative IT Disaster Recovery Unit Mission Ensure minimal risk of major disruptions to critical University systems and processes in the event that all or part of its computer operations are rendered inoperable. Ensure timely recovery of infrastructure and services in the event of a disruption. Ensure that business continuity plans are available and viable relative to its scenario.

18. Risk Management Identify Mitigate Process Mapping

19. Risk Management Risk Mitigation Prioritize Actions Evaluate recommended Control Options Conduct Cost-Benefit Analysis Select Controls Assign Responsibility Develop Safeguard Implementation Plan Implement Selected Controls Risk Assessment System Characterization Threat Identification Vulnerability Identification Control Analysis Likelihood Determination Impact Analysis Risk Determination Control Recommendations Results Documentation NIST SP 800-30

20. Process Mapping

21. Infrastructure Total DR through distributed high availability Client Recovery Solutions Application Restoration Establish collaborative partnerships with other Universities

22. Client Recovery Solution(s)

23. Application Restoration Event Time Scope of Impact –Infrastructure –Software –Hardware

24. Collaborative Partnerships

25. Readily accessible Secure Onsite Offsite Vaulting

26. Critical Business Units Advancement Services All Campus Network Budget Office College of Agriculture and Life Sciences - Personnel Office ComTech - Data Networking ComTech - Telecommunications Contracts and Grants Controller's Office Enterprise Application and Database Services EH&S - Business Continuity EH&S - Campus Police EH&S - Emergency Response EH&S - Environmental Affairs EH&S - Health and Safety EH&S - Industrial Hygiene EH&S - Insurance and Risk Management EH&S - Radiation Safety EH&S - Transportation EH&S - Waste Management Enrollment Management - Admissions Enrollment Management - Office of Scholarships & Financial Aid Enrollment Management - Registration and Records Enterprise Technology Services and Support Facilities - Construction Management Facilities - Design and Construction Services Facilities - Operations Facilities - University Architect Fire Protection Foundations Accounting & Investments HR - Benefits HR - Employment & Compensation HR - Human Resource Information Management HR - Payroll ITD - Business Services ITD - Computer Operations ITD - Computer Services ITD - Systems Libraries - Administration Materials Management - Materials Support Materials Management - Purchasing Materials Management - University Graphics Real Estate Student Health Services University Cashier's Office University Dining University Housing

27. Business Continuity Planning

28. Communication Consistency in plan updating Training Partnering Emergency Communication standardization –Call Trees –Mobile Devices –Website –Incident Command System Call Center –Incident Report Plan

29. IT Disaster Categorization Category 1: A single person or group in a Critical Business Unit (CBU) is unable to perform their critical functions Category 2: An entire CBU is unable to perform its critical functions Category 3: Multiple CBUs are unable to perform their critical functions Category 4: Non CBUs are not able to perform their critical functions Category 5: A wide spread event that impacts the entire University

30. Goals Total DR through distributed high availability Standardized Emergency Communications Immediate Client Recovery Solutions Improved RTO

31. Ann Harris Asst Dir, Administrative IT Disaster Recovery 919-515-9228 ann_harris@ncsu.edu http://www.fis.ncsu.edu/dr