Presentation is loading. Please wait.

Presentation is loading. Please wait.

B1: After Imaging New features. Greater flexibility. Richard Banville Fellow.

Similar presentations


Presentation on theme: "B1: After Imaging New features. Greater flexibility. Richard Banville Fellow."— Presentation transcript:

1 B1: After Imaging New features. Greater flexibility. Richard Banville Fellow

2 © 2008 Progress Software Corporation 2 Agenda Basics New features Online implementation Archival and Maintenance

3 © 2008 Progress Software Corporation 3 Agenda Basics What is after imaging? Why use after imaging? How does it work? What is the AI archiver?

4 © 2008 Progress Software Corporation 4 What is after imaging? A journal of transaction data that can replay changes to a database Referred to as a redo log Granularity of recovery To last completed transaction To a point in time To a specific transaction number. Space is not reused without interaction (as it is in the before image journal files)

5 © 2008 Progress Software Corporation 5 Why use after imaging? Protection Removes single point of failure Media loss Human error –Table drop –File deletion No data loss High availability & Business Continuity For replication Manual or OpenEdge Replication

6 © 2008 Progress Software Corporation 6 How does it work? Transaction data synchronously written to an after image (AI) journal AI journal replayed in recovery scenarios Not automatically enabled Automation Some automatic maintenance features Recovery not automated

7 © 2008 Progress Software Corporation 7 How the AI File Works FOR EACH customer: UPDATE customer. END. BI Note Written AI Note Written Transaction Begin BI and AI Transaction End Notes Written *** Data updated sometime later

8 © 2008 Progress Software Corporation 8 Forward Processing - Physical View DatabaseBefore Image Shared Memory After Image t Full AI files must be managed

9 © 2008 Progress Software Corporation 9 Forward Processing - Physical View DatabaseBefore Image Shared Memory After Image t

10 © 2008 Progress Software Corporation 10 The Recovery Process DatabaseBefore Image After Image Backup Remote copy 1a 1b 3 thru n roll forward 2 Redo

11 © 2008 Progress Software Corporation 11 Agenda Online AI functionality Automatic AI file management AI archiver New Features

12 © 2008 Progress Software Corporation 12 New after imaging features (10.1A thru 10.1C) Online AI functionality Add extents Start/Stop (enable/disable) Automatic archiving –Archive extents –Change archive directory –Change archive interval Manage AI buffers Guarded recovery process Online features are a huge advance in AIs flexibility

13 © 2008 Progress Software Corporation 13 What is the AI archiver? Daemon (AIMD) to manage/move full AI files Timer mode (every seconds) On-demand mode (when full) Archives & marks AI extents for reuse Enablement Offline via rfutil Online via the probkup utility Additional online features Switch archive locations Change archive interval No support for archiving to a second host No support for single user activity Introduced in 10.1A

14 © 2008 Progress Software Corporation 14 Empty AI Archiver Shared Memory DB BI A1 AI Archive Daemon Archive 1 Archive 2 Archive 3 Knows when and where to archive A2 Busy FullEmpty

15 © 2008 Progress Software Corporation 15 Agenda Creating AI files Enabling AI Starting and managing the AI archiver Implementing After Imaging Online

16 © 2008 Progress Software Corporation 16 Creating AI files New Syntax Notes: Ensure structure file correct before adding Defaults to 8K AI block size Ensure default user permissions are correct prior to adding extents prostrct addonline prostrct addonline -validate Adding extents online

17 © 2008 Progress Software Corporation 17 Creating AI files Q: How many AI files to create? A: More than one A2: Everyone has a different opinion 10 is a good starting point Extent type: variable, fixed or vixed? Depends on archive mechanism Variable – easier to maintain Fixed for ultimate performance –Makes some sense with fixed amount of work

18 © 2008 Progress Software Corporation 18 Need to have: AI files must reside on a different disk/LUN than any other aspect of the database (db, dn, bn) Nice to have: AI archived to yet another disk/physical locale –not always possible/plausible Avoid I/O bottlenecks Add more spindles if need be Location, location, location After Image File and Archive Locations

19 © 2008 Progress Software Corporation 19 Enabling After Imaging Online New option to the probkup utility Notes: Make sure your AI files are in place Common uses: –Oops, forgot to enable after imaging –AI off Recoverable maintenance AI on probkup online enableai

20 © 2008 Progress Software Corporation 20 Managing AI buffers online (10.1C) New option Notes: For performance, set aibufs >= bi buf pool size –Online default same as -bibufs (8K AI buffers) Permissions for new shared memory proutil increase -aibufs

21 © 2008 Progress Software Corporation 21 Offline: AI must already be enabled ;( Online: AI can be enabled in conjunction ;) Enabling the AI archiver rfutil -C aiarchiver enable probkup online [ enableai ] enableaiarchiver -aiarcdir -aiarcinterval [ -aiarcdircreate ]

22 © 2008 Progress Software Corporation 22 New Syntax: Can optionally create directories on the fly Again, watch permissions Changing the Archive Directory Online rfutil -C aiarchiver setarcdir rfutil -C aiarchiver setarcdir / -aiarcdircreate

23 © 2008 Progress Software Corporation 23 Changing Archive Interval Online New Syntax: Valid range 120 to 86,400 (in seconds) 2 minutes to 1 day Suggest 15 to 60 minutes based on activity Common uses Correct mistake of minutes vs seconds Change in business constraints (lost data) Archiver awakes every 5 seconds rfutil -C aiarchiver setinterval Forcing an extent switch automatically

24 © 2008 Progress Software Corporation 24 Changing Archive Mode Online On-demand Mode No interval or interval 0 Use with fixed length extents or manual switch Timed Mode Interval set 120 to 86,400 (in seconds) Archives regardless of activity Notes: Change time and/or on-demand during off hours Timed must be set at each invocation Archiver still awakes every 5 seconds rfutil -C aiarchiver setinterval On-demand vs timed archival

25 © 2008 Progress Software Corporation 25 Agenda Monitoring your AI files Archive log naming Backing up AI archive files Cleaning up AI archive files Archiving and Maintenance

26 © 2008 Progress Software Corporation 26 Manual AI Archival Manually archive AI files while db is online Check for full extents Force extent switch Find next full extent to archive Rinse, lather, repeat Example code: No archiver? No problem. rfutil -C aimage new rfutil -C aimage full rfutil -C aimage list

27 © 2008 Progress Software Corporation 27 Not user definable (it should be) Naming convention Recovery easily managed Files relate to a particular backup Date and time Extent sequence number in the name Directory separator / replaced with ~ Example: Archive Log Naming. date. time. backup-seq. ai-seq. ext-name usr1~richb~x x.a1

28 © 2008 Progress Software Corporation 28 Backup of Archives Increase types of recoverable disasters How often Throughout the day Where to Separate media for media failure protection Secondary machine as quickly as possible Best if archival system is very far away NOT included in the OpenEdge database backup

29 © 2008 Progress Software Corporation 29 Cleaning Up Archives Eventual disk space issue Retention dependant on business needs Clean up Create an automated process to clean up archives after a period of time (days) Keep at least 7 days of archives online How often do you backup? –Archives without a backup is useless Where are they stored? Maintaining a hot standby? Archiver retains archives forever!

30 © 2008 Progress Software Corporation 30 Cleaning Up Archives - Unix/Linux The find command is very useful for this task # Archive Cleaner # Should be run from cron daily # Test yourself with echo before implementing ARCDIR= KEEPDAYS=+30 find $ARCDIR -mtime $KEEPDAYS -exec \rm {} \ 2>&1 > /dev/null Beware of changes in $ARCDIR Consider rather than /dev/null Incorporate remote copy Add automatic validation

31 © 2008 Progress Software Corporation 31 Cleaning Up Archives - Windows Uses file from the free windows resource kit File name: %WINDOWS%\system32\forfiles.exe REM ** Archive cleanup batch file REM ** Have this run as a scheduled task REM ** Test yourself with echo before implementing Set ARCDIR=your-archive-directory Set KEEP=-30 forfiles /P %ARCDIR% /m *.a* /D %KEEP% /C cmd /C del REM OR forfiles -p%ARCDIR% -m*.a* -d%KEEP% -ccmd /c del //technet.microsoft.com/en-us/library/cc aspx

32 © 2008 Progress Software Corporation 32 Disaster Scenarios Floods Fire Bad stuff happens

33 © 2008 Progress Software Corporation 33 Using auto AI archives to recover In the AI archiver log.archival.log Find most recent backup to restore –Look for BACKUP_SET Identify AI extent to start roll forward with –Match aiseq value to archived AI extent Ignore AI extents backup sequence value –Roll forward it and all later archives Log file layout documented in DB Admin Guide Example: … It is quite easy

34 © 2008 Progress Software Corporation 34 AI Example # OpenEdge RDBMS After-image Extent Manager Archival Log File # Do not edit this file with an editor as it is automatically # maintained by the OpenEdge RDBMS after-image extent manager. 0032,/usr1/db/x, ,100553,1,2, ,100551,1,1,x.bak,BACKUP_SET_ _ # # After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56 # 0255, ,100556, ,/usr1/db/x, ,100853, ,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x x.a1 0001,/usr1/dbx, ,100908, ,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2 0032,/usr1/db/x, ,100947,1,3, ,100946,4,1,../x.bak2,BACKUP_SET_ _ ,/usr1/db/x, ,100949, ,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3 0001,/usr1/db/x, ,101029, ,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4 0001,/usr1/db/x, ,101355, ,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a1 0001,/usr1/db/x, ,101441, ,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2 0001,/usr1/db/x, ,101831, ,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3 0001,/usr1/db/x, ,102037, ,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4

35 © 2008 Progress Software Corporation 35 A Eye Exam(ple) # OpenEdge RDBMS After-image Extent Manager Archival Log File # Do not edit this file with an editor as it is automatically # maintained by the OpenEdge RDBMS after-image extent manager. 0032,/usr1/db/x, ,100553,1,2, ,100551,1,1,x.bak,BACKUP_SET_ _ # # After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56 # 0255, ,100556, ,/usr1/db/x, ,100853, ,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x x.a1 0001,/usr1/dbx, ,100908, ,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2 0032,/usr1/db/x, ,100947,1,3, ,100946,4,1,../x.bak2,BACKUP_SET_ _ ,/usr1/db/x, ,100949, ,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3 0001,/usr1/db/x, ,101029, ,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4 0001,/usr1/db/x, ,101355, ,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a1 0001,/usr1/db/x, ,101441, ,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2 0001,/usr1/db/x, ,101831, ,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3 0001,/usr1/db/x, ,102037, ,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4

36 © 2008 Progress Software Corporation 36 A Eye Exam(ple) # OpenEdge RDBMS After-image Extent Manager Archival Log File # Do not edit this file with an editor as it is automatically # maintained by the OpenEdge RDBMS after-image extent manager. 0032,/usr1/db/x, ,100553,1,2, ,100551,1,1,x.bak,BACKUP_SET_ _ # # After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56 # 0255, ,100556, ,/usr1/db/x, ,100853, ,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x x.a1 0001,/usr1/dbx, ,100908, ,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2 0032,/usr1/db/x, ,100947,1,3, ,100946,4,1,../x.bak2,BACKUP_SET_ _ ,/usr1/db/x, ,100949, ,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3 0001,/usr1/db/x, ,101029, ,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4 0001,/usr1/db/x, ,101355, ,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a1 0001,/usr1/db/x, ,101441, ,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2 0001,/usr1/db/x, ,101831, ,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3 0001,/usr1/db/x, ,102037, ,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4 Backup file-name, BACKUP_SET AI Seq #4 AI file-name to start rollforward AI Seq #4

37 © 2008 Progress Software Corporation 37 Using auto AI archives to recover Documentation issue: buseq aiseq Wrong AI file? No problem. Error message reported Retry with the correct one Roll forward last active/not archived AI extent Always use AI sequence value Archive location within.log file is relatively meaningless w/respect to its BACKUP_SET WHY? … Some notes on the AI archive log

38 © 2008 Progress Software Corporation 38 Why AI Log is NOT Sequential Backup starts w/an AI file switch Current busy extent is marked full Subsequent DB changes recorded in new AI extent –which may also fill up and be auto archived Archival of these extents may occur before or after backup finishes (and is logged) …with respect to the BACKUP_SET

39 © 2008 Progress Software Corporation 39 Why AI Log is NOT Sequential A1 DB BI AI Archive Daemon Archive 1 Archive 2 Archive 3 A2 Users Backup DB.bak Archive Log Extent Switch …with respect to the BACKUP_SET No sync on write order BusyFullEmpty Busy

40 © 2008 Progress Software Corporation 40 Recovery Process Overview Get everyone off the system (crons too!) Backup what you have (if anything) Database (even just parts) Recovery files (AI, BI,TL) AI Archives Delete broken database Restore most recent backup Apply after image archives rfutil -C roll forward oplock –a (10.1c)

41 © 2008 Progress Software Corporation 41 Extending Archival Usefulness Log-based replication

42 © 2008 Progress Software Corporation 42 Log-Based Replication Hot standby database Good to use for backups –Eliminates backup I/O on production –Use -norecover option with probkup Less recovery downtime Cannot be updated (subsequent rollf. will fail) Hot standby copy (#2) Provide for reporting (semi-stale) Midnight copy of hot standby #1 Extending archival usefulness

43 © 2008 Progress Software Corporation 43 Log-Based Replication Auto archived files Deal w/awkward naming convention Create sub-directory under archive directory Process looks for new archives Applies new archives to hot-standby database Moves applied archives Best if on a remote host Less contention w/production Improved disaster recovery strategy Extending archival usefulness

44 © 2008 Progress Software Corporation 44 Summary All OpenEdge systems should use AI Many new online AI capabilities AI management tool provided Use archives to maintain a hot-standby Improved high availability

45 © 2008 Progress Software Corporation 45 Questions ?

46 © 2008 Progress Software Corporation 46 Outstanding Questions Answered Does ai archiver use ai extract or simply copy? It uses Ai Extract. Is ai archiver available for Workgroup licenses? Yes. It is functionality and not performance related. You can specify several archive directories. Is there a way to tell which is the current archive directory? There is a message in the log when a dir switch is performed. There is no way to programmatically determine this. I will add an enhancement request. When ai is locked due to replication, is the file archived and later marked empty or does the archive process stall waiting for the locked extent to become unlocked? The extent is archived. Once it is unlocked it is emptied by the watch dog. The aimgt deamon also attempts to empty locked extents. Is there any documentation on using ai archiver with replication? Yes, there is a callout in the ai mgt doc when using it with replication. The doc explains what happens to an extent with ai mgt and repl running. Is it safe to change permissions of a file after add online completes? No. Although add online has completed, all users connected to the database have not opened the new file. They open it the next time they access it. Changing its permissions while the database is still running may cause the database to shutdown when a user attempts to open a file it does not have permission to.

47 © 2008 Progress Software Corporation 47 Thank You

48 © 2008 Progress Software Corporation 48


Download ppt "B1: After Imaging New features. Greater flexibility. Richard Banville Fellow."

Similar presentations


Ads by Google