New features. Greater flexibility. B1: After Imaging New features. Greater flexibility. Richard Banville Fellow

Agenda Basics New features Online implementation Archival and Maintenance 2

Agenda Basics What is after imaging? Why use after imaging? How does it work? What is the AI archiver? Notes: Loss of database and/or before image will not cause loss of data as after image files can be applied to backup to get current Loss of database backup can be rectified by applying multiple days of after image files Deletion of files, programming errors and the like can be recovered using AI files. The point-in-time feature allows you to roll forward transactions to a particular point-in-time or a particular transaction number. 3

What is after imaging? A journal of transaction data that can replay changes to a database Referred to as a redo log Granularity of recovery To last completed transaction To a point in time To a specific transaction number. Space is not reused without interaction (as it is in the before image journal files) Notes: Loss of database and/or before image will not cause loss of data as after image files can be applied to backup to get current Loss of database backup can be rectified by applying multiple days of after image files Deletion of files, programming errors and the like can be recovered using AI files. The point-in-time feature allows you to roll forward transactions to a particular point-in-time or a particular transaction number. 4

Why use after imaging? Protection No data loss Removes single point of failure Media loss Human “error” Table drop File deletion No data loss High availability & Business Continuity For replication Manual or OpenEdge Replication Notes: Loss of database and/or before image will not cause loss of data as after image files can be applied to backup to get current Loss of database backup can be rectified by applying multiple days of after image files Deletion of files, programming errors and the like can be recovered using AI files. The point-in-time feature allows you to roll forward transactions to a particular point-in-time or a particular transaction number. 5

How does it work? Transaction data synchronously written to an after image (AI) journal AI journal “replayed” in recovery scenarios Not automatically enabled Automation Some automatic maintenance features Recovery not automated

How the AI File Works FOR EACH customer: UPDATE customer. END. Transaction Begin BI Note Written AI Note Written BI and AI Transaction End Notes Written Notes: Just prior to the update a before image note is written to capture the record state prior to update and the before image file and the after image file are updated at the end of the update and then a transaction end note is written at the end of the transaction. *** Data updated “sometime” later 7

Forward Processing - Physical View Shared Memory 3 + t 1 2 Before Image After Image Database Notes: An active database consists of very current data (shared memory), current data (before image file) and archival data (the database). The after image file(s) are associated with the backup not the database Full AI files must be managed 8

Forward Processing - Physical View Shared Memory 3 + t 1 2 Before Image After Image Database Notes: An active database consists of very current data (shared memory), current data (before image file) and archival data (the database). The after image file(s) are associated with the backup not the database 9

The Recovery Process Remote copy 1a 1b 3 thru n roll forward 2 Backup After Image Database Before Image 2 Redo 10

Agenda New Features Online AI functionality Automatic AI file management AI archiver

New after imaging features (10.1A thru 10.1C) Online features are a huge advance in AI’s flexibility Online AI functionality Add extents Start/Stop (enable/disable) Automatic archiving Archive extents Change archive directory Change archive interval Manage AI buffers Guarded recovery process 12

What is the AI archiver? Introduced in 10.1A Daemon (AIMD) to manage/move “full” AI files Timer mode (every <n> seconds) On-demand mode (when full) Archives & marks AI extents for reuse Enablement Offline via rfutil Online via the probkup utility Additional online features Switch archive locations Change archive interval No support for archiving to a second host No support for single user activity Daemon does connect to the database and therefore uses 1 -n 13

AI Archiver Shared Memory DB BI Archive 1 A1 Busy Empty Full Archive 2 Daemon Knows when and where to archive DB BI Archive 1 A1 Busy Empty Full Archive 2 Busy Empty A2 Archive 3

Implementing After Imaging Online Agenda Implementing After Imaging Online Creating AI files Enabling AI Starting and managing the AI archiver 15

Creating AI files New Syntax Notes: Adding extents online New Syntax Notes: Ensure structure file correct before adding Defaults to 8K AI block size Ensure default user permissions are correct prior to adding extents prostrct addonline <db> <st-file-name> prostrct addonline <db> <st-file-name> -validate 16

Creating AI files Q: How many AI files to create? A: More than one A2: Everyone has a different opinion 10 is a good starting point Extent type: variable, fixed or vixed? Depends on archive mechanism Variable – “easier” to maintain Fixed for “ultimate” performance Makes some sense with fixed amount of work

After Image File and Archive Locations Location, location, location Need to have: AI files must reside on a different disk/LUN than any other aspect of the database (db, dn, bn) Nice to have: AI archived to yet another disk/physical locale not always possible/plausible Avoid I/O bottlenecks Add more spindles if need be

Enabling After Imaging Online New option to the probkup utility Notes: Make sure your AI files are in place Common uses: Oops, “forgot” to enable after imaging AI off  Recoverable maintenance  AI on probkup online <db> <backup-file> enableai

Managing AI buffers online (10.1C) New option Notes: For performance, set aibufs >= bi buf pool size Online default same as -bibufs (8K AI buffers) Permissions for new shared memory proutil <db> increase -aibufs <n>

Enabling the AI archiver Offline: AI must already be enabled ;( Online: AI can be enabled in conjunction ;) rfutil <db> -C aiarchiver enable probkup online <db> <backup> [ enableai ] enableaiarchiver -aiarcdir <dir> -aiarcinterval <n-seconds> [ -aiarcdircreate ]

Changing the Archive Directory Online New Syntax: Can optionally create directories on the fly Again, watch permissions rfutil <db> -C aiarchiver setarcdir <dir-list> rfutil <db> -C aiarchiver setarcdir <dir-list>/ -aiarcdircreate

Changing Archive Interval Online Forcing an extent switch automatically New Syntax: Valid range 120 to 86,400 (in seconds) 2 minutes to 1 day Suggest 15 to 60 minutes based on activity Common uses Correct mistake of minutes vs seconds Change in business constraints (lost data) Archiver awakes every 5 seconds rfutil <db> -C aiarchiver setinterval <#> Archiving regardless of activity means that even during periods of slowness, lots of extents will be archived even if not full

Changing Archive Mode Online On-demand vs timed archival rfutil <db> -C aiarchiver setinterval <#> On-demand Mode No interval or interval 0 Use with fixed length extents or manual switch Timed Mode Interval set 120 to 86,400 (in seconds) Archives regardless of activity Notes: Change time and/or on-demand during off hours Timed must be set at each invocation Archiver still awakes every 5 seconds Archiver awakes every 5 seconds You will “rarely” have more than one full extent if the AI archiver is active It would be nice to not archive ai extents which have not been used at all.

Archiving and Maintenance Agenda Archiving and Maintenance Monitoring your AI files Archive log naming Backing up AI archive files Cleaning up AI archive files 25

Manual AI Archival Manually archive AI files while db is online No archiver? No problem. Manually archive AI files while db is online Check for full extents Force extent switch Find next full extent to archive Rinse, lather, repeat Example code: http://wss.com/products/downloads.html rfutil <db> -C aimage list rfutil <db> -C aimage new rfutil <db> -C aimage full

Archive Log Naming Not user definable (it should be) Naming convention Recovery easily managed Files relate to a particular backup Date and time Extent sequence number in the name Directory separator “/” replaced with “~” Example: <db> . date . time . backup-seq . ai-seq . ext-name Recovery is NOT automatic but is easily managed because of naming convention usr1~richb~x.20080806.151638.00003.00000001.x.a1

Backup of Archives Increase types of recoverable disasters How often NOT included in the OpenEdge database backup Increase types of recoverable disasters How often Throughout the day Where to Separate media for media failure protection Secondary machine as quickly as possible Best if archival system is “very” far away

Cleaning Up Archives Eventual disk space issue Archiver retains archives forever! Eventual disk space issue Retention dependant on business needs Clean up Create an automated process to clean up archives after a period of time (days) Keep at least 7 days of archives online How often do you backup? Archives without a backup is useless Where are they stored? Maintaining a hot standby?

Cleaning Up Archives - Unix/Linux The find command is very useful for this task # Archive Cleaner # Should be run from cron daily # Test yourself with echo before implementing ARCDIR=<your-archive-directory> KEEPDAYS=+30 find $ARCDIR -mtime $KEEPDAYS -exec \rm {} \ 2>&1 > /dev/null Beware of changes in $ARCDIR Consider e-mail rather than /dev/null Incorporate remote copy Add automatic validation

Cleaning Up Archives - Windows Uses file from the free windows resource kit File name: %WINDOWS%\system32\forfiles.exe REM ** Archive cleanup batch file REM ** Have this run as a scheduled task REM ** Test yourself with echo before implementing Set ARCDIR=your-archive-directory Set KEEP=-30 forfiles /P %ARCDIR% /m *.a* /D %KEEP% /C “cmd /C del /q @FILE” REM OR forfiles -p%ARCDIR% -m*.a* -d%KEEP% -c“cmd /c del /q @FILE" //technet.microsoft.com/en-us/library/cc755872.aspx

Disaster Scenarios Bad stuff happens Floods Fire

Using auto AI archives to recover It is quite easy In the AI archiver log “<db>.archival.log” Find most recent backup to restore Look for “BACKUP_SET” Identify AI extent to start roll forward with Match “aiseq” value to archived AI extent Ignore AI extent’s backup sequence value Roll forward it and all later archives Log file layout documented in DB Admin Guide Example: …

AI Example # OpenEdge RDBMS After-image Extent Manager Archival Log File # Do not edit this file with an editor as it is automatically # maintained by the OpenEdge RDBMS after-image extent manager. 0032,/usr1/db/x,20080807,100553,1,2,20080807,100551,1,1,x.bak,BACKUP_SET_20080807_100551 # # After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56 0255,20080807,100556,100100 0001,/usr1/db/x,20080807,100853,20080807,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x.20080807.100552.00002.00000001.x.a1 0001,/usr1/dbx,20080807,100908,20080807,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00002.00000002.x.a2 0032,/usr1/db/x,20080807,100947,1,3,20080807,100946,4,1,../x.bak2,BACKUP_SET_20080807_100946 0001,/usr1/db/x,20080807,100949,20080807,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000003.x.a3 0001,/usr1/db/x,20080807,101029,20080807,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000004.x.a4 0001,/usr1/db/x,20080807,101355,20080807,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000005.x.a1 0001,/usr1/db/x,20080807,101441,20080807,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000006.x.a2 0001,/usr1/db/x,20080807,101831,20080807,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000007.x.a3 0001,/usr1/db/x,20080807,102037,20080807,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000008.x.a4

A Eye Exam(ple) # OpenEdge RDBMS After-image Extent Manager Archival Log File # Do not edit this file with an editor as it is automatically # maintained by the OpenEdge RDBMS after-image extent manager. 0032,/usr1/db/x,20080807,100553,1,2,20080807,100551,1,1,x.bak,BACKUP_SET_20080807_100551 # # After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56 0255,20080807,100556,100100 0001,/usr1/db/x,20080807,100853,20080807,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x.20080807.100552.00002.00000001.x.a1 0001,/usr1/dbx,20080807,100908,20080807,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00002.00000002.x.a2 0032,/usr1/db/x,20080807,100947,1,3,20080807,100946,4,1,../x.bak2,BACKUP_SET_20080807_100946 0001,/usr1/db/x,20080807,100949,20080807,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000003.x.a3 0001,/usr1/db/x,20080807,101029,20080807,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000004.x.a4 0001,/usr1/db/x,20080807,101355,20080807,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000005.x.a1 0001,/usr1/db/x,20080807,101441,20080807,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000006.x.a2 0001,/usr1/db/x,20080807,101831,20080807,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000007.x.a3 0001,/usr1/db/x,20080807,102037,20080807,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000008.x.a4

Backup file-name, BACKUP_SET AI file-name to start rollforward A Eye Exam(ple) # OpenEdge RDBMS After-image Extent Manager Archival Log File # Do not edit this file with an editor as it is automatically # maintained by the OpenEdge RDBMS after-image extent manager. 0032,/usr1/db/x,20080807,100553,1,2,20080807,100551,1,1,x.bak,BACKUP_SET_20080807_100551 # # After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56 0255,20080807,100556,100100 0001,/usr1/db/x,20080807,100853,20080807,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x.20080807.100552.00002.00000001.x.a1 0001,/usr1/dbx,20080807,100908,20080807,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00002.00000002.x.a2 0032,/usr1/db/x,20080807,100947,1,3,20080807,100946,4,1,../x.bak2,BACKUP_SET_20080807_100946 0001,/usr1/db/x,20080807,100949,20080807,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000003.x.a3 0001,/usr1/db/x,20080807,101029,20080807,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000004.x.a4 0001,/usr1/db/x,20080807,101355,20080807,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000005.x.a1 0001,/usr1/db/x,20080807,101441,20080807,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000006.x.a2 0001,/usr1/db/x,20080807,101831,20080807,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000007.x.a3 0001,/usr1/db/x,20080807,102037,20080807,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000008.x.a4 Backup file-name, BACKUP_SET AI Seq #4 AI Seq #4 AI file-name to start rollforward AI Seq #4

Using auto AI archives to recover Some notes on the AI archive log Documentation issue: buseq  aiseq Wrong AI file? No problem. Error message reported Retry with the correct one Roll forward last active/not archived AI extent Always use AI sequence value Archive location within .log file is “relatively” meaningless w/respect to its BACKUP_SET WHY? … Backup causes an AI file switch Current “busy” extent is marked “full” Archive of this full extent post dates backup time It’s data is already in the backup

Why AI Log is NOT Sequential …with respect to the BACKUP_SET Backup starts w/an AI file switch Current “busy” extent is marked “full” Subsequent DB changes recorded in new AI extent which may also fill up and be auto archived Archival of these extents may occur before or after backup finishes (and is logged)

Why AI Log is NOT Sequential …with respect to the BACKUP_SET Archive Log DB.bak No sync on write order Backup AI Archive Daemon Extent Switch DB BI A1 Archive 1 Full Busy Empty Archive 2 Archive 3 A2 Users Busy Empty

Recovery Process Overview Get everyone off the system (crons too!) Backup what you have (if anything) Database (even just parts) Recovery files (AI, BI,TL) AI Archives Delete broken database Restore most recent backup Apply after image archives rfutil <db> -C roll forward oplock –a <aifile> (10.1c)

Extending Archival Usefulness Log-based replication 41

Log-Based Replication Extending archival usefulness Hot standby database Good to use for backups Eliminates backup I/O on production Use -norecover option with probkup Less recovery downtime Cannot be updated (subsequent rollf. will fail) Hot standby copy (#2) Provide for reporting (semi-stale) “Midnight” copy of hot standby #1

Log-Based Replication Extending archival usefulness Auto archived files Deal w/awkward naming convention Create sub-directory under archive directory Process looks for new archives Applies new archives to hot-standby database Moves applied archives Best if on a remote host Less contention w/production Improved disaster recovery strategy

Summary All OpenEdge systems should use AI Many new online AI capabilities AI management tool provided Use archives to maintain a hot-standby Improved high availability 44

? Questions

Outstanding Questions Answered Does ai archiver use ai extract or simply copy? It uses Ai Extract. Is ai archiver available for Workgroup licenses? Yes.  It is functionality and not performance related. You can specify several archive directories.  Is there a way to tell which is the current archive directory? There is a message in the log when a dir switch is performed. There is no way to programmatically determine this. I will add an enhancement request. When ai is locked due to replication, is the file archived and later marked empty or does the archive process stall waiting for the locked extent to become unlocked? The extent is archived.  Once it is “unlocked” it is emptied by the watch dog.  The aimgt deamon also attempts to empty locked extents. Is there any documentation on using ai archiver with replication? Yes, there is a callout in the ai mgt doc when using it with replication.  The doc explains what happens to an extent with ai mgt and repl running. Is it safe to change permissions of a file after add online completes? No. Although add online has completed, all users connected to the database have not opened the new file. They open it the next time they access it. Changing its permissions while the database is still running may cause the database to shutdown when a user attempts to open a file it does not have permission to.

Thank You Thank you in many languages