1. Creating a complete recovery solution Adam Backman adam@wss.com Partner, White Star Software

2. Why backup?  No reason to backup – Our stuff never fails  Just takes up resources  We already have redundancy  I hate changing the tapes  I’m tired  I’m hungry  I don’t feel good  Are we there yet?

3. Cost of Lost Data  Loss of data is expensive −Cost to rekey −Cost of lost business  Can you recover the data −Not everyone has paper trail  Focus on recovery

4. Building a Good Recovery Strategy  Know your business −Components of business how people do business with you −Components of systems Tools (applications and physical)  Know your risks (fire, flood, hurricane, …)  Be inclusive −Technical people (network, phones, facilities, …) −Business people (people who own the application functionality)  Build an execution plan with contingencies

5. Why be inclusive?  Business people know what they need −What they use in the application −What they need to do their job (phone, fax, internet, …)  Stops the finger pointing when a problem occurs  Business people generally control the budget  Need decision makers −Authorize spending −Data loss vs. recovery time decisions

6. Focus on Recovery  Loss of data Rogue program, hacked, poor testing  Loss of disk Disk failure  Loss of machine Machine failure  Loss of machine room Fire, flood,…  Loss of building Natural disaster

7. Creating a plan  Goals (Event-based goals) −Data corruption (Data) −If we lose a disk (DB) −If we have a fire (Machine) −If we have a natural disaster (Facility)  Hardware  Software  Other stuff

8. Main Goal The main goal for any recovery plan is to minimize or eliminate the impact to the customer

9. Creating a plan - Goals  Acceptable downtime (Generally cost based) Everyone wants zero but it is generally cost prohibitive  Planned outages −Hardware install and maintenance −Software upgrade −O/S upgrade or patch  Notifications (Both before and during outage) −Who −When −What do they do?

10. Creating a Plan – Other Considerations  What makes your business run? −Phones −Faxes −Business to Business (EDI, XML Feed, …)  Can people work from home?  Do you have/need another location?  Contact lists in case of major catastrophe −Kept up-to-date −Kept online and printed in an accessible location

11. Cover All Sides  Everyone should be running after image journaling  Need removable backup periodically (daily) −Wide scale events (fire, flood, …) −To recover from after image journals  Replication is the new normal −OE Replication −Log-based replication −Hardware replication

12. Portions of a OpenEdge Database  Database table of contents (.db)  The data files (.d*)  Before image journals (.b*)  After image journals (.a)  Modified buffers in memory

13. How probkup -online works 1.Lock the DB 2.Do a pseudo-checkpoint 3.Switch AI files (if necessary) 4.Backup the BI file 5.Unlock the DB 6.Backup the database

14. Other Items to Consider  Application files  Data outside the database (GIS, photos, …)  User files (reports, output, …)  External systems (EDI, Warehouse, …)

15. Backup’s Impact to Production  Backing up production −Pause during backup of before image journal −Uses I/O capacity of production  Split mirror backup −Use of quiet point keeps pause to a minimum −Pause is non-zero  After Image file backup −Still needs a backup to begin the process −Little impact for backup process −Potentially long recovery time

16. After Image Management  Use OpenEdge after image management −Automated −Supported −Nice features  Use script-based after image management −Custom −Requires support and testing

17. Replication options  OpenEdge Replication −Supported by Progress −Real time replication −Allows for reporting against target database −Additional license cost  Log-based replication −Free −Requires self-support −Not real time (greater potential data loss) −No reporting option  Hardware replication

18. OpenEdge Replication – Single Target Production (source)Reporting (target) Shared Memory Replication Server Replication Agent Source DB Target DB

19. OpenEdge Replication Production (source) Target 1 Shared Memory Replication Server Rep Agent Source DB Target DB Target 2 Shared Memory Rep Agent Target DB Multiple Targets

20. Log-Based Replication  Log-based replication has been used for years Log-based replication provides a vehicle for replication without the licensing costs of OE Replication  Not real-time  Code for this type of replication must be maintained by the user and there is no official support from the vendor

21. Hardware-Based Replication  Hardware-based replication is a function of the hardware vendors and thus supported directly by them  This method is NOT supported by Progress  ALL write operations must be guarantied across the source and target disk systems

22. Archiving – Why is it important  Needs to be accessible  Needs to be secure  Needs to be in a different location  Need to have policies −Physical access −Number of saved archives −Security

23. Archiving  Who does your archiving (Iron Mountain, third- party, someone’s house, …)  What do you keep −Two weeks of dailies −5 weeks of weeklies −1 year of monthlies  How to label you backups −Who did the backup −Command to restore −Date and Time

24. Pre-archiving  After image files −Move to remote system  Database backups −Move to remote system before moving to removable media  Support files  User data

25. Archiving (continued)  Data Archiving – What data archiving? −Archive/Delete? −Archive/Save historical −Archive/Save aggregates  After Image file archiving −At least 2 backups worth −I recommend a week or more if possible

26. How about if I am a SAAS user  Who is your provider  Verify their recovery plan  Run dry run of at least one recovery scenario  Have specific service level agreements −Time to recover −Maximum loss of data −Penalties for missing times (need to cover your costs)

27. How about if I am a SAAS provider  Build regular recovery plan  Unique concerns −Security −Compliance (HIPAA, SOX, …)  Build achievable SLAs for your users that at a minimum include: −Data loss −Uptime −Maximum recovery time −Maintenance time

28. Implementing Your Plan  First implementation should be a totally manual process to insure the steps work and allow for documentation  Document the process as you go −Who are you logged in as? −Exactly what you typed −Where you were (console, remote, …) −Can things be done in parallel or sequentially −Where are the logs −What to look for in the logs (good and bad)

29. Documentation  All recovery documentation should be VERY specific as the person doing the recovery will not be familiar with you or your environment  Create documents for normal maintenance −Backups −Database growth −Modification of OS, Application, printers, …  Create scenario based recovery plans −Lose a disk (or disk pair) −Fire −Flood

30. Testing Your Plan  Who does the test? −Not the person who wrote it −The backup person for the implementation −Someone who is “always” there regardless of technical ability  How often to test? −Material data change −Any change in database configuration  New directories  New areas −Do you have a second site or redundant hardware? −Do you have enough disk capacity (space and throughput)

31. How to test your plan  Fail over to your backup system −You may lose the use of the backup system but that is a good trade in the vast majority of cases −Maybe not live but at least a restore and test  Fail back to your primary system −After a live failover  Contingency planning for: −Personnel −Physical plant −Equipment −Lead time for resources (hours or days)

32. Summary: Recovery Planning  You still need to backup  Backup your backup not production if possible  Be inclusive when building your team  Create and maintain a comprehensive plan Include everything needed to use the application:  Hardware  Applications  Data  Create and maintain physical and online contact lists and store it with your archived backups

33. Conclusion: Recover planning  Always backup what you have now, however little, before starting to recover  Test your plan periodically (At least annually)

34. Still have questions? Please feel free to contact me directly. Adam Backman White Star Software (603)897-1010 adam@wss.com

35. THANK YOU Thank you for your time