Presentation on theme: "Business Continuity /Disaster Recovery Panel"— Presentation transcript:
1 Business Continuity /Disaster Recovery Panel STA Annual Conference 2006
2 A High Level Summary of the Lessons Learned from Hurricane Wilma byFranklin Templeton InvestmentsWayne BehrensDirectorBusiness Continuity PlanningPrepared for theSecurities Transfer AssocationAmelia Island Meeting on Oct 20, 2006
3 Wilma Agenda Franklin Templeton Investments Who was Wilma Impact of WilmaWhat Went RightCrisis Management LessonsBusiness Continuity LessonsTechnology LessonsFacility Lessons
4 Franklin Templeton Investments Parent: Franklin Resources Inc.Approx 500 Billion in Assets Under ManagementMajor BrandsFranklinTempletonMutual SeriesFiduciary TrustDarby OverseasBisset (Canada)Employees: Approx 8,000 in 29 countries. In the Florida Area:St. Petersburg (1,200+)Ft. Lauderdale (466)Nassau (49)Miami (16)
5 Who the Heck was Wilma?Hurricane Wilma is the lesser known cousin of Katrina. However, Wilma was the most intense hurricane ever recorded in the Atlantic basin. It devastated parts of the Yucatán Peninsula as well as southern Florida.There were 62 deaths attributed to Wilma and damage is estimated at $12.2 billion in the U.S., making Wilma the sixth costliest storm in U.S history.When Wilma reached Ft. Lauderdale, she was a category 2 hurricane with sustained winds of 110 mph to 130 MPH.
6 Impact of Wilma on Franklin Wilma reached Ft. Lauderdale early Monday morning, October 24th. By Monday afternoon, we had received preliminary damage assessments indicating that hundreds of windows primarily on the north west section of the building were blown out.We were unable to occupy the building from October 24 through November 18.No loss of life and no injuries to employees. Some damage to a few employee’s homes. Many employee’s homes were without power for days or weeks.500 employees were impacted, with over 230 being relocated to other sites: Toronto, St. Petersburg, Miami, New York, Short Hills, San Mateo, and Rancho Cordova.
7 What Went RightIn general, our Crisis Management, Business Continuity and Technology Disaster Recovery plans worked well.Wilma caused virtually no disruption to our customers.The Ft. Lauderdale Emergency Management Team took charge of the situation in Ft. Lauderdale.All business units were able to follow their business continuity plans and recover their operations.Technology operations were restored to include two critical applications.
8 Crisis Management Lessons When the local Emergency Management Teams (EMT’s) are in the midst of the incident and working literally in the dark, they do not always have the ability to fully coordinate the recovery of business operations away from the site.Each of the major roles in our crisis management teams need to have a specific checklist.We need to put in place a pre-plan to track and deal immediately with the relocation of employees to other sites.You cannot over communicate. Despite the fact that we tried very hard to be proactive on communicating to our employees, we still heard a number of complaints in this area.
9 Business Continuity Lessons Over 40 laptops were left in the FTL office when the hurricane hit. There had been so many hurricane threats during the year that many employees did not feel that Wilma would really hit or cause this much damage if it did.The standard for Business Continuity Plans needs to be expanded to address a month long outage:Shifting work to alternate sites for the first 3-5 days without moving employeesPlan for an incident to last over a month to include a month endA number of issues arose in regards to employees:Pay during the outageChildcare and school closuresBonus for extraordinary effortsRelocation of children & elders
10 Technology Lessons Learned We were surprised by the number of business units that still relied upon hard copies of faxed documents to stand alone fax machine. We have worked on a better process and documentation of FAX rerouting requirements.Better written procedures for forwarding 800 numbers.There is a desire from the business for a better disaster recovery solution for and Blackberry servers.Environmental monitoring of server rooms needs to be tied into a central control point to insure it is remotely accessible.
11 Facilities Lessons Learned We should have drilled more rigorously on damage assessments. For the first couple of days, the damage assessments were verbal and led us to believe the damage was much more extensive than it was. It turned out only 15% to 20% of the work areas were damaged.Conversely, our repair and re-occupancy time estimates were wildly optimistic.Based on initial reports, we planned for a week long disruption. We were out for almost a month.Need to have working knowledge prior to the incident of what local agencies will require to re-occupy a building.Keep in mind the fire marshal and the building inspectors are not always in sync.
12 Questions? Wilma Closing In the end it was the knowledge, flexibility and perseverance of our people who really carried the day and made the recovery a success.Questions?
14 Pandemic Agenda Goal of this presentation Likelihood of a Pandemic Basic Business Continuity StrategyWhy planHow will a pandemic differBasic elements of our planCrisis ManagementBusiness Continuity PlanningTechnologyGeneral ServicesHuman RelationsCorporate CommunicationsWhat our plan does not cover
15 Pandemic Background1. Goal: To provide an overview of Franklin’s current approach and thinking in regards to planning for a possible Pandemic.2. Likelihood of a pandemic occurring: The question is more like earthquakes in California. It is not a question of if, but rather when and how bad. Some data points such as the 1918, 1957 & 1968 pandemics and SAR’s. But, no good data on frequency or severity.3. Strategy: Our basic disaster planning strategy is to shift our operations to other sites for 3 to 5 days, after which we will then need to start to shift people to alternate sites. However, this will not work in a pandemic situation.
16 Pandemic Background (Continued) 4. Why plan if civilization is going to collapse:We used a “reasonable worst case scenario”. A scenario which we think is likely to occur. This is not a worst case scenario which anticipates the general breakdown of society and services.5. How will a pandemic differ from our “normal incidents:- Many sites are likely to be impactedat approximately the same time- Will not be able to shift peoplebetween sitesSite might be impacted, but notincapacitated- Sites affected for months not hoursAffects people directly not facilities or IT.Employees may choose not to come to work.No clear beginning or end.
17 Pandemic Plan Outline Our pandemic plan is broken down into: 1. Crisis Management:- Framework to address a pandemic- Tabletop exercises2. Business Continuity Planning:- Guidance to business units on how to review their business continuity strategies and workflows against apandemic type scenario3. TechnologySteps to reduce the impact of apandemic on our data centers- Increased remote work capabilities- Reviewing other strategies
18 Pandemic Plan Outline (Continued) 4. General ServicesBest practices for employee hygiene program and procedures for facilitymanagers to follow in the event of a pandemic5. Human RelationsGlobal HR policy framework to provide recommendations to local HRgroups to address issues that are likely to arise in a pandemic6. Corporate CommunicationsIntegrated communication plan7. Plan Does not currently include:PPE such as Masks gloves, Etc.Antiviral Drugs such as TamifluVaccines
23 Federal Financial Partnership Financial and Banking Information Infrastructure Committee (FBIIC)(formed January 2002)Financial Services Sector Coordinating Council (FSSCC)(formed June 2002)
24 FBIIC FSSCC PUBLIC SECTOR PRIVATE SECTOR President’s Working Group on Financial MarketsTreasury - Lead Agency(PDD 63)US TreasuryAssistant Secretary forFinancial InstitutionsFBIIC CHAIRAssistant Secretary forFinancial InstitutionsSECTOR LIAISONRhonda MacLeanSECTOR COORDINATORFinancial and Banking InformationInfrastructure Committee(FBIIC)US Treasury DepartmentCommodity Futures Trading CommissionConference of State Bank SupervisorsFederal Deposit Insurance CorporationFederal Housing Finance BoardFederal Reserve Board of GovernorsHomeland Security CouncilNational Association of InsuranceCommissionersNational Credit Union AdministrationNew York Federal Reserve BankOffice of the Comptroller of the CurrencyOffice of Federal Housing Enterprise OversightOffice of Thrift SupervisionSecurities and Exchange CommissionFinancial Services Sector CoordinatingCouncilfor CIP/HLSFinancial Services Trade Associations& InstitutesNew York Stock ExchangeThe ClearinghouseFS/ISACSecurities Industry Automation CorporationThe Options Clearing CorporationChicagoFIRSTNASDAQAMEXASIS
25 Public/Private Partnerships The Role for RegionalPublic/Private Partnerships
26 The Missing PieceTo increase the resilience of financial services in the event of a regional disaster in collaboration with the city, state, and federal agencies.
27 All Disasters are Local How will that jurisdiction prevent, prepare for, and respond to incidents?Do your business continuity plans incorporate government response plans?How can coordination be fostered among jurisdictions?Regional partnerships can strengthen the business continuity plans of participating firms
28 Regional Partnerships: Formed and Forming Miami (FloridaFIRST)San Francisco (BARC FIRST; Bay Area Response Coalition)Los Angeles (SoCal FIRC; Financial Institutions Recovery Coalition)Minneapolis (MN-ISAC; MN Security Board)Tampa Bay Region of FloridaFIRSTHoustonFIRSTPhiladelphiaFIRSTColumbusFIRSTAlabama Recovery Coalition for the Financial SectorChicago (ChicagoFIRST)Washington, DCDetroitAlaskaSeattleJacksonvilleLas VegasNew Orleans (still thinking about it)
29 RPC FIRST Fostering Collaboration among Partnerships ChicagoFIRST leading the formation of a CouncilCouncil would share best practicesCouncil would help one another with administrative questionsCouncil can plug into FSSCCRPC = Regional Partnership CouncilFIRST = Financial Industry Resilience, Security, and TeamworkFormed in early 2006
31 ChicagoFIRST’s Primary Objectives Obtain a seat at Chicago's 911 Center in the event of a crisis that affects Chicago's financial communityLaSalle Bank/ABN AMROCreate permits/passes for essential personnel to safely access business facilities in the event of a general evacuation of the city (credentialing)Northern Trust BankDevelop and communicate standard evacuation procedures for industry personnel to exit city limits in the event of a disasterJP Morgan Chase
32 Members ABN AMRO / LaSalle Bank Allstate Insurance Company Aon ArchipelagoAriel Capital ManagementBank of AmericaChicago Board Options ExchangeChicago Board of TradeChicago Federal Home Loan BankChicago Mercantile ExchangeChicago Stock ExchangeFidelity National FinancialGlobal Electronic Trading CompanyHarris BankJP Morgan ChaseMan FinancialMesirow FinancialMizuho Securities USANorthern TrustThe Options Clearing CorporationPrivateBank and TrustUBSWashington MutualWilliam Blair & Company
33 Strategic Partners (pg. 1 of 2) Chicago Office of Emergency Management and CommunicationsChicago Police DepartmentCommodity Futures Trading CommissionFBI / InfraGardFederal Deposit Insurance CorporationFederal Emergency Management AgencyFederal Reserve Bank of ChicagoFinancial and Banking Information Infrastructure CommitteeFinancial Services Information Sharing and Analysis CenterFinancial Services Roundtable / BITSFinancial Services Sector Coordinating CouncilFutures Industry Association
34 Strategic Partners (pg. 2 of 2) Great Lakes PartnershipIllinois Department of Financial and Professional RegulationIllinois Emergency Management AgencyIllinois State PoliceIllinois Terrorism Task ForceNational Futures AssociationOffice of the Comptroller of the CurrencySecurities and Exchange CommissionSecurities Industry AssociationUnited States Attorney’s Office for the Northern District of IllinoisUnited States Department of Homeland SecurityUnited States Department of the TreasuryUnited States Secret Service
36 Formal 911 Center Seat Obtained seat at 911 Center in fall 2003 Primarily for government agenciesMay use seat when Center is activatedEnhancements to seat at emergency operations centerSet of individuals to staff the seat (with Chicago Fed help)Handbook with protocols for using the seat, activating our crisis communicator, and contact informationPrivate component of web site created and configured to provide a message board for posting and recording critical informationInformation about the membership, including critical locations and essential employees, on the computer at the seat
37 Informal Information Sharing Seat at 911 Center will be used rarelyBut the relationships with the city and state are invaluableSpring 2004 information about leaning transmission towerAugust 1, 2004 threats against financial institutionsLaSalle Bank fire, December 2004
38 Credentialing and Evacuations Discovered city and state each seeking credentialing systems, but not coordinatingCity adopted credentialing pilot in which ChicagoFIRST participatesEvacuationsIllinois Department of Transportation tabletops in 2004, 2005, and 2006September 7, 2006 evacuation drill in the Loop
40 Working Groups Security Working Group Coordinating training needs and opportunitiesCoordinating physical security and optionsPiloted NC4 Situation Awareness ServicePower Working GroupUnderstanding electricity in multi-tenant buildingsSharing ComEd information among members
41 Working Groups Telecommunications Working Group Educating membership GETSTSPSBC call forwardingSurviving a central office failureTeleContinuityLEMKOSprint IP network
42 Working Groups Pandemic Planning Working Group Free exchange of HR, legal, & BCP information, without NDAsCoordinating with state and local health departmentsCoordinating with sector-wide effortsEvaluating hiring a public health advisor for ChicagoFIRSTTabletop scheduled for November 2, 2006
43 Working Groups Public Relations Working Group Single point of contact for the mediaFirms leverage membership with pressChicagoFIRST increases media understandingCrisis Communications Working GroupQuarterly tests of the 911 Center proceduresQuarterly tests of Dialogic (notification data)Quarterly tests of TeleContinuity and GETS
44 2004 MilestonesTestified before House Financial Services Committee on ChicagoFIRST as a partnership9/11 Commission legislation identifies ChicagoFIRST as a modelGAO Report on Financial Market Preparedness praises ChicagoFIRSTTreasury handbook identifies ChicagoFIRST as modelTabletop on city’s response to Chicago financial community
45 2005 Milestones Tabletop focused on futures and options markets Public television features ChicagoFIRSTFund Illinois Terrorism Task Force (ITTF) video for the citizens of IllinoisCo-chair Private Sector Committee of the ITTF
46 2006 Activities Mutual aid among the members Credentialing critical supplies like cashEvacuation drillCity of Chicago camera programProvided testimony on pandemic preparedness to the House Financial Services Committee
47 ChicagoFIRST Model Works The model is the partnership approach, not the goals or organization of ChicagoFIRSTFloridaFIRST covers the entire state, with several regionsBARC FIRST and SoCal FIRC split CaliforniaMN-ISAC has Target, Best Buy, 3M as membersLeverage partnership to encourage public sector information sharing and improvementsSeats in EOCsCredentialingAccess protocols for critical supplies
48 The Value Proposition LaSalle Bank fire Mizuho futures Cooperation vs. competition on employee safety and business continuity (mutual aid established after the fire)Government appreciates single point of contactNC4 and TeleContinuity