Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Dealing with Crises: CSDs’ Recent Experiences Singapore Exchange Case Studies SARS in Singapore 9/11/2001 in the U.S. …and a few words about Earthquakes,

Similar presentations


Presentation on theme: "1 Dealing with Crises: CSDs’ Recent Experiences Singapore Exchange Case Studies SARS in Singapore 9/11/2001 in the U.S. …and a few words about Earthquakes,"— Presentation transcript:

1 1 Dealing with Crises: CSDs’ Recent Experiences Singapore Exchange Case Studies SARS in Singapore 9/11/2001 in the U.S. …and a few words about Earthquakes, Typhoons, Ice Storms,Blackouts Mary Ann Callahan, Managing Director, DTCC ACSDA / STRATE Seminar, November 20-21, 2003

2 2 1.Double Wake Up Calls 2.SARS & Terrorism 3.SARS Hit Singapore 4.SARS Contingency Measures in SGX 5.Lessons Learned 6.BCP Best Practices Guidelines 7.Conclusion Business Continuity in the Wake of SARS* & 9/11 *borrowing from Daniel Tan of Singapore Exchange at ISSA…

3 3 Multiple Wake Up Calls - Before, After & Since… Prior to 9/11 and SARS (Severe Acute Respiratory Syndrome)  Disaster scenarios evolved around man-made accidents e.g.,fire, chemical leakage and power outage  BCP was positioned mainly in terms of Disaster Recovery for technology assets, networks and data In the wake of SARS & Terrorism  Many BCP assumptions were proved wrong and inadequate  BCP readiness became our only protection

4 4 SARS & Terrorism – The Similarities  Crippling blow to business operations  Invisible and totally unpredictable  Real or potential staggering loss of life of key executives (2,752 people died in 9/11) and disrupting crisis management plan  Need alternate site to resume business operations  Severe impact on the travel industry and major economic losses

5 5 SARS & Terrorism – The Subtle Differences  SARS is not linked to terrorism  Terrorism usually leads to massive loss of technology and physical assets. SARS requires segregation of critical staff to operate from separate locations  A terrorist attack is not only unpredictable but also less controllable. SARS can be positively controlled with adequate continuity response measures.

6 6 SARS Hit Singapore – A Chronology of Events  Three Singaporeans admitted to hospital  SARS notified as infectious disease under Infectious Disease Act. Healthcare workers were particularly affected 1-3 Mar 17 Mar  Suspects and probable SARS cases centralized at a designated hospital 22 Mar  Childcares, preschools, primary and secondary schools closed 27 Mar

7 7 SARS Hit Singapore – A Chronology of Events  Home quarantine orders implemented. 10 Apr  Hospital visitors restricted to 1 per patient to reduce possible virus spread 18 Apr  Thermal imaging scanners installed at airport and main entry points 23 Apr  Vegetable wholesale center closed for 10 days – fear of SARS spreading to the community 20 Apr  Singapore removed from WHO’s list of SARS affected areas 31 May  In the end, 162 people were infected, and 13 people died.

8 8 SARS – Disaster Scenario  Severe temporary or permanent loss of critical key staff  Absenteeism due to illness, serving Quarantine Orders  Higher incidence of sick leave  Threat of forced closure of key business operation facilities by health authority e.g. Trading floor (Pictures)Pictures

9 9 SGX Open Outcry Trading Floor

10 10 SARS – Contingency Measures in SGX  Staffs with critical operational responsibilities segregated to work from the Emergency Operation Center (EOC) far away from the main office.  Health declaration and temperature checks made compulsory for all visitors. Particulars (names, NRIC, body temperature, etc) recorded for contact tracing.  Business travel ban to SARS affected areas imposed.  Staff advised against social travel; Self-imposed home quarantine for those who did.  Isolation rooms identified for handling suspected SARS.

11 11 SARS – Contingency Measures on Trading Floor Measures to prevent spreading to trading floor  Daily health declaration by all trading floor personnel. (No fever + No coughs/breathing difficulty symptoms + No travel to SARS areas)  Body temperature check using thermal scanner (Picture)Picture  No visitors allowed into trading floor  Disinfectant cleaning done twice a week  Alternative trading room set up; Canadian financial industry had “clean teams”

12 12 Thermal Scanner installed in SGX

13 13 Lessons Learned  Shared recovery solution inadequate (multiple activations, containment)  Segregating personnel in different location impeded by lack of qualified staff and unavailability of IT infrastructure to support primary system and DR site. Need for cross- training, decentralized-operations?  Unmanned data centre with remote operation capability  Potential for increased focus on telecommuting.  Prevention is a BCP strategy

14 14 The Impact of 9/11 DTCC Source:TowerGroup

15 15 9/11 damage to securities industry IT infrastructure estimated at $3.2 billion Estimated workstations lost (‘000) Estimated servers lost Total Damage (in billions) Soft ware ServicesHardware Source:TowerGroup

16 16 DTCC Before 9/11/2001  DTCC staff concentrated at headquarters location (a well- known address) in Manhattan, with modest staff complement in Brooklyn, outside NYC  Principal processing facility and depository primary data center in Manhattan  Depository and clearing corporation data centers and business recovery site split between Manhattan and Brooklyn (but very close)

17 17 Significant Issues That DTCC Faced  Impact of an event affecting the greater NY metropolitan area  9/11 changed old BCP assumptions that only a single building outage should be planned for, and assumptions about telecom and fiber routes, when 80% of inbound phone traffic to Manhattan was blocked.  Site of DTCC’s production data center  Location of DTCC’s staff  Notification and instruction procedures during event and recovery period  Client readiness, and their Business Recovery locations

18 18 DTCC’s Response – first 18 months  Immediately moved primary data center outside Manhattan  Implemented an additional data center, adding equipment to all sites  Relocated staff outside Manhattan; currently planning for a new operational site even farther away  Various recovery scenarios and scripts

19 19 DTCC’s Response – first 18 months  Implemented Officer rotation  Utilize Government telecom programs:  Telecommunication Service Priority - TSP  Government Emergency Telephone Service (GETS)  Participant Readiness:  Developed additional communications requirements for largest clients  Require dedicated backup connection  Require annual recovery testing, from client primary and backup sites, to all DTCC processing sites

20 20 DTCC Remote Data Center (RDC)  Acquired site and completed major modifications.  Implemented offsite tape storage program in 2002; implemented data replication in 2003  Support client traffic daily across all data centers  Distance requires Multi-Hop process part of daily operation  Systems ready and available.  First chance to prove: DTCC relied on RDC staff and components during NY’s blackout in August 2003.

21 21 RDC Management and Command Center  6 DTCC NY staff formed initial team, then local hires  Management staff on site  Staff today covers 3 shifts, representing Network, DP Operations, Command Center, Facilities and System Support  DP Operations calls distributed among all sites  Established Command Center at RDC  Ability to command all data centers from RDC

22 22 Status – Data Centers  Supporting live customer traffic on a daily basis through all data centers; each customer’s traffic is routed though each data center at least once per year.  Successful start-ups of NY Production systems using Multi- Hop data connections  3 rd (Remote)Data Center certified May 2003 for backup of all DTC critical systems.

23 23 Status - Network  Existing circuits at client locations provide connectivity to RDC  Routing to RDC did not require customer involvement  Additional functionality soon:  Network engineers have completed design and implementation of interconnected Stock Exchanges/CCP and Depository networks, to enable all DTCC businesses to reach RDC.

24 24 Our Participants’ Experience Post 9/11  Took 12 months to build out operations space and 12 to 18 months to build out trading floors. Firms directly impacted by the climate control breakdown saw how difficult it is to simultaneously execute people recovery and system recovery.  Firms relocated from disaster sites into temp space (1-3 months). Firms looked for more permanent space (3-9 months)  Lost 10% of Manhattan office space  Not enough lower Manhattan new space  9/11 showed people support is also needed, including an HR helpline and ability to locate staff 

25 25 Tokyo BCP: Focus on Natural Disasters  Earthquake vs. building failure, fire, loss of power  People plan  JASDEC’s back-up center in Osaka

26 26 Other Natural Disasters  Hong Kong Exchanges & Clearing’s procedures for typhoons and black rainstorm warnings.   The Canadian Depository’s ice storm experience in the late 1990s.

27 27 Blackouts – since August 2003! United States/Canada: 14 & 15 August Republic of Georgia: 19 August London, UK: 28 August Yucatan Peninsula: 2 September Sweden & portions of Denmark: 2 September Italy & Switzerland: 28 September Israel: 4 October Czech Republic: 8 October Australia: 9 October Island of Guam: 13 October Argentina: 16 October United Kingdom: 17 October Brazil: 29 October Bangladesh: 30 October

28 28 Regulatory Intervention  Monetary Authority of Singapore issued guidelines to Financial Institutions  In the U.S., Federal Reserve-Securities and Exchange Commission Interagency White Paper  Others regulators, e.g., Financial Services Authority(s), Bank of Canada

29 29  Avoid the ‘War and Peace’ syndrome - don’t confuse crisis management with business continuity planning  Think effect, not cause  Build a solid platform from which you will initiate your response that involves your best and brightest people with specific roles during a disaster  Empower staff with the appropriate decision making authority  If you involve the right people with the right authority and a basic foundation plan, you will be successful Industry Insights: Good Crisis Management is Flexible

30 30 Business Continuity Planning: A Process, Not A Project  Accept the fact that some percentage of what you plan for today will change two weeks from now and will continue to deteriorate over time  It is better to have a plan that is substantially complete/accurate than no plan at all

31 31 The Right Balance  Pre 9/11: Without an obvious and immediate threat, complacency sometimes infiltrated the BC planning process  Post 9/11: In response to a catastrophic event, people can overreact in BC planning  In summary, you should strive to develop BC plans that consider the critical, time sensitive aspects of your business, and are adaptable to the various situations that may arise

32 32 From “Build and React” to “Integrate, Mitigate”  BCP has traditionally been a reactive process  Due to the heightened threat environment, business recovery issues should be considered as a primary operational risk  Strategies such as secondary and tertiary data centers and diversified office space/staff locations are getting increased focus

33 33 Conclusions  Challenging, costly to anticipate all of these events  SGX found shared BCP recovery solutions within the its own market inadequate  Increasing STP and process automation have expensive implications to BCP/DR  Expect the unexpected, plan for the unplanned – network attack?  BCP should be part of IT project life cycle like DR  Readiness and prevention are the only protection  Consider recovery from beyond the shore:  Is there any potential for regional CSDs to work on common solutions?


Download ppt "1 Dealing with Crises: CSDs’ Recent Experiences Singapore Exchange Case Studies SARS in Singapore 9/11/2001 in the U.S. …and a few words about Earthquakes,"

Similar presentations


Ads by Google