1. New features. Greater flexibility.
B1: After Imaging
New features. Greater flexibility.
Richard Banville
Fellow

2. Agenda Basics New features Online implementation
Archival and Maintenance 2

3. Agenda Basics What is after imaging? Why use after imaging?
How does it work?
What is the AI archiver?
Notes:
Loss of database and/or before image will not cause loss of data as after image files can be applied to backup to get current
Loss of database backup can be rectified by applying multiple days of after image files
Deletion of files, programming errors and the like can be recovered using AI files. The point-in-time feature allows you to roll forward transactions to a particular point-in-time or a particular transaction number. 3

4. What is after imaging?
A journal of transaction data that can replay changes to a database
Referred to as a redo log
Granularity of recovery
To last completed transaction
To a point in time
To a specific transaction number.
Space is not reused without interaction
(as it is in the before image journal files)
Notes:
Loss of database and/or before image will not cause loss of data as after image files can be applied to backup to get current
Loss of database backup can be rectified by applying multiple days of after image files
Deletion of files, programming errors and the like can be recovered using AI files. The point-in-time feature allows you to roll forward transactions to a particular point-in-time or a particular transaction number. 4

5. Why use after imaging? Protection No data loss
Removes single point of failure
Media loss
Human “error”
Table drop
File deletion
No data loss
High availability & Business Continuity
For replication
Manual or OpenEdge Replication
Notes:
Loss of database and/or before image will not cause loss of data as after image files can be applied to backup to get current
Loss of database backup can be rectified by applying multiple days of after image files
Deletion of files, programming errors and the like can be recovered using AI files. The point-in-time feature allows you to roll forward transactions to a particular point-in-time or a particular transaction number. 5

6. How does it work?
Transaction data synchronously written to an after image (AI) journal
AI journal “replayed” in recovery scenarios
Not automatically enabled
Automation
Some automatic maintenance features
Recovery not automated

7. How the AI File Works FOR EACH customer: UPDATE customer. END.
Transaction Begin
BI Note Written
AI Note Written
BI and AI Transaction End Notes Written
Notes:
Just prior to the update a before image note is written to capture the record state prior to update and the before image file and the after image file are updated at the end of the update and then a transaction end note is written at the end of the transaction.
*** Data updated “sometime” later 7

8. Forward Processing - Physical View
Shared Memory
3 + t 1 2
Before Image
After Image
Database
Notes:
An active database consists of very current data (shared memory), current data (before image file) and archival data (the database).
The after image file(s) are associated with the backup not the database
Full AI files
must be managed 8

9. Forward Processing - Physical View
Shared Memory
3 + t 1 2
Before Image
After Image
Database
Notes:
An active database consists of very current data (shared memory), current data (before image file) and archival data (the database).
The after image file(s) are associated with the backup not the database 9

10. The Recovery Process Remote copy 1a 1b 3 thru n roll forward 2 Backup
After Image
Database
Before Image 2
Redo 10

11. Agenda New Features Online AI functionality
Automatic AI file management
AI archiver

12. New after imaging features (10.1A thru 10.1C)
Online features are a huge advance in AI’s flexibility
Online AI functionality
Add extents
Start/Stop (enable/disable)
Automatic archiving
Archive extents
Change archive directory
Change archive interval
Manage AI buffers
Guarded recovery process 12

13. What is the AI archiver? Introduced in 10.1A
Daemon (AIMD) to manage/move “full” AI files
Timer mode (every <n> seconds)
On-demand mode (when full)
Archives & marks AI extents for reuse
Enablement
Offline via rfutil
Online via the probkup utility
Additional online features
Switch archive locations
Change archive interval
No support for archiving to a second host
No support for single user activity
Daemon does connect to the database and therefore uses 1 -n 13

14. AI Archiver Shared Memory DB BI Archive 1 A1 Busy Empty Full Archive 2
Daemon
Knows when and
where to archive DB BI
Archive 1 A1
Busy
Empty
Full
Archive 2
Busy
Empty A2
Archive 3

15. Implementing After Imaging Online
Agenda
Implementing After Imaging Online
Creating AI files
Enabling AI
Starting and managing the AI archiver 15

16. Creating AI files New Syntax Notes:
Adding extents online
New Syntax
Notes:
Ensure structure file correct before adding
Defaults to 8K AI block size
Ensure default user permissions are correct prior to adding extents
prostrct addonline <db> <st-file-name>
prostrct addonline <db> <st-file-name> -validate 16

17. Creating AI files Q: How many AI files to create?
A: More than one
A2: Everyone has a different opinion
10 is a good starting point
Extent type: variable, fixed or vixed?
Depends on archive mechanism
Variable – “easier” to maintain
Fixed for “ultimate” performance
Makes some sense with fixed amount of work

18. After Image File and Archive Locations
Location, location, location
Need to have:
AI files must reside on a different disk/LUN than any other aspect of the database (db, dn, bn)
Nice to have:
AI archived to yet another disk/physical locale
not always possible/plausible
Avoid I/O bottlenecks
Add more spindles if need be

19. Enabling After Imaging Online
New option to the probkup utility
Notes:
Make sure your AI files are in place
Common uses:
Oops, “forgot” to enable after imaging
AI off  Recoverable maintenance  AI on
probkup online <db> <backup-file> enableai

20. Managing AI buffers online (10.1C)
New option
Notes:
For performance, set aibufs >= bi buf pool size
Online default same as -bibufs (8K AI buffers)
Permissions for new shared memory
proutil <db> increase -aibufs <n>

21. Enabling the AI archiver
Offline:
AI must already be enabled ;(
Online:
AI can be enabled in conjunction ;)
rfutil <db> -C aiarchiver enable
probkup online <db> <backup> [ enableai ]
enableaiarchiver
-aiarcdir <dir>
-aiarcinterval <n-seconds>
[ -aiarcdircreate ]

22. Changing the Archive Directory Online
New Syntax:
Can optionally create directories on the fly
Again, watch permissions
rfutil <db> -C aiarchiver setarcdir <dir-list>
rfutil <db> -C aiarchiver setarcdir <dir-list>/
-aiarcdircreate

23. Changing Archive Interval Online
Forcing an extent switch automatically
New Syntax:
Valid range 120 to 86,400 (in seconds)
2 minutes to 1 day
Suggest 15 to 60 minutes based on activity
Common uses
Correct mistake of minutes vs seconds
Change in business constraints (lost data)
Archiver awakes every 5 seconds
rfutil <db> -C aiarchiver setinterval <#>
Archiving regardless of activity means that even during periods of slowness, lots of extents will be archived even if not full

24. Changing Archive Mode Online
On-demand vs timed archival
rfutil <db> -C aiarchiver setinterval <#>
On-demand Mode
No interval or interval 0
Use with fixed length extents or manual switch
Timed Mode
Interval set 120 to 86,400 (in seconds)
Archives regardless of activity
Notes:
Change time and/or on-demand during off hours
Timed must be set at each invocation
Archiver still awakes every 5 seconds
Archiver awakes every 5 seconds
You will “rarely” have more than one full extent if the AI archiver is active
It would be nice to not archive ai extents which have not been used at all.

25. Archiving and Maintenance
Agenda
Archiving and Maintenance
Monitoring your AI files
Archive log naming
Backing up AI archive files
Cleaning up AI archive files 25

26. Manual AI Archival Manually archive AI files while db is online
No archiver? No problem.
Manually archive AI files while db is online
Check for full extents
Force extent switch
Find next full extent to archive
Rinse, lather, repeat
Example code:
rfutil <db> -C aimage list
rfutil <db> -C aimage new
rfutil <db> -C aimage full

27. Archive Log Naming Not user definable (it should be) Naming convention
Recovery easily managed
Files relate to a particular backup
Date and time
Extent sequence number in the name
Directory separator “/” replaced with “~”
Example:
<db> . date . time . backup-seq . ai-seq . ext-name
Recovery is NOT automatic but is easily managed because of naming convention
usr1~richb~x x.a1

28. Backup of Archives Increase types of recoverable disasters How often
NOT included in the OpenEdge database backup
Increase types of recoverable disasters
How often
Throughout the day
Where to
Separate media for media failure protection
Secondary machine as quickly as possible
Best if archival system is “very” far away

29. Cleaning Up Archives Eventual disk space issue
Archiver retains archives forever!
Eventual disk space issue
Retention dependant on business needs
Clean up
Create an automated process to clean up archives after a period of time (days)
Keep at least 7 days of archives online
How often do you backup?
Archives without a backup is useless
Where are they stored?
Maintaining a hot standby?

30. Cleaning Up Archives - Unix/Linux
The find command is very useful for this task
# Archive Cleaner
# Should be run from cron daily
# Test yourself with echo before implementing
ARCDIR=<your-archive-directory>
KEEPDAYS=+30
find $ARCDIR -mtime $KEEPDAYS -exec \rm {} \ 2>&1 > /dev/null
Beware of changes in $ARCDIR
Consider rather than /dev/null
Incorporate remote copy
Add automatic validation

31. Cleaning Up Archives - Windows
Uses file from the free windows resource kit
File name: %WINDOWS%\system32\forfiles.exe
REM ** Archive cleanup batch file
REM ** Have this run as a scheduled task
REM ** Test yourself with echo before implementing
Set ARCDIR=your-archive-directory
Set KEEP=-30
forfiles /P %ARCDIR% /m *.a* /D %KEEP% /C “cmd /C del
REM OR
forfiles -p%ARCDIR% -m*.a* -d%KEEP% -c“cmd /c del
//technet.microsoft.com/en-us/library/cc aspx

32. Disaster Scenarios
Bad stuff happens
Floods
Fire

33. Using auto AI archives to recover
It is quite easy
In the AI archiver log “<db>.archival.log”
Find most recent backup to restore
Look for “BACKUP_SET”
Identify AI extent to start roll forward with
Match “aiseq” value to archived AI extent
Ignore AI extent’s backup sequence value
Roll forward it and all later archives
Log file layout documented in DB Admin Guide
Example: …

34. AI Example
# OpenEdge RDBMS After-image Extent Manager Archival Log File
# Do not edit this file with an editor as it is automatically
# maintained by the OpenEdge RDBMS after-image extent manager.
0032,/usr1/db/x, ,100553,1,2, ,100551,1,1,x.bak,BACKUP_SET_ _100551 #
# After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56
0255, ,100556,100100
0001,/usr1/db/x, ,100853, ,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x x.a1
0001,/usr1/dbx, ,100908, ,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2
0032,/usr1/db/x, ,100947,1,3, ,100946,4,1,../x.bak2,BACKUP_SET_ _100946
0001,/usr1/db/x, ,100949, ,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3
0001,/usr1/db/x, ,101029, ,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4
0001,/usr1/db/x, ,101355, ,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a1
0001,/usr1/db/x, ,101441, ,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2
0001,/usr1/db/x, ,101831, ,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3
0001,/usr1/db/x, ,102037, ,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4

35. A Eye Exam(ple)
# OpenEdge RDBMS After-image Extent Manager Archival Log File
# Do not edit this file with an editor as it is automatically
# maintained by the OpenEdge RDBMS after-image extent manager.
0032,/usr1/db/x, ,100553,1,2, ,100551,1,1,x.bak,BACKUP_SET_ _100551 #
# After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56
0255, ,100556,100100
0001,/usr1/db/x, ,100853, ,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x x.a1
0001,/usr1/dbx, ,100908, ,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2
0032,/usr1/db/x, ,100947,1,3, ,100946,4,1,../x.bak2,BACKUP_SET_ _100946
0001,/usr1/db/x, ,100949, ,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3
0001,/usr1/db/x, ,101029, ,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4
0001,/usr1/db/x, ,101355, ,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a1
0001,/usr1/db/x, ,101441, ,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2
0001,/usr1/db/x, ,101831, ,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3
0001,/usr1/db/x, ,102037, ,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4

36. Backup file-name, BACKUP_SET AI file-name to start rollforward
A Eye Exam(ple)
# OpenEdge RDBMS After-image Extent Manager Archival Log File
# Do not edit this file with an editor as it is automatically
# maintained by the OpenEdge RDBMS after-image extent manager.
0032,/usr1/db/x, ,100553,1,2, ,100551,1,1,x.bak,BACKUP_SET_ _100551 #
# After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56
0255, ,100556,100100
0001,/usr1/db/x, ,100853, ,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x x.a1
0001,/usr1/dbx, ,100908, ,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2
0032,/usr1/db/x, ,100947,1,3, ,100946,4,1,../x.bak2,BACKUP_SET_ _100946
0001,/usr1/db/x, ,100949, ,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3
0001,/usr1/db/x, ,101029, ,100552,3,4,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4
0001,/usr1/db/x, ,101355, ,100552,3,5,/usr1/db/x.a1,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a1
0001,/usr1/db/x, ,101441, ,100552,3,6,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a2
0001,/usr1/db/x, ,101831, ,100552,3,7,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a3
0001,/usr1/db/x, ,102037, ,100552,3,8,/usr1/db/x.a4,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x x.a4
Backup file-name, BACKUP_SET
AI Seq #4
AI Seq #4
AI file-name to start rollforward
AI Seq #4

37. Using auto AI archives to recover
Some notes on the AI archive log
Documentation issue: buseq  aiseq
Wrong AI file? No problem.
Error message reported
Retry with the correct one
Roll forward last active/not archived AI extent
Always use AI sequence value
Archive location within .log file is “relatively” meaningless w/respect to its BACKUP_SET
WHY? …
Backup causes an AI file switch
Current “busy” extent is marked “full”
Archive of this full extent post dates backup time
It’s data is already in the backup

38. Why AI Log is NOT Sequential
…with respect to the BACKUP_SET
Backup starts w/an AI file switch
Current “busy” extent is marked “full”
Subsequent DB changes recorded in new AI extent
which may also fill up and be auto archived
Archival of these extents may occur before or after backup finishes (and is logged)

39. Why AI Log is NOT Sequential
…with respect to the BACKUP_SET
Archive Log
DB.bak
No sync on
write order
Backup AI
Archive
Daemon
Extent Switch DB BI A1
Archive 1
Full
Busy
Empty
Archive 2
Archive 3 A2
Users
Busy
Empty

40. Recovery Process Overview
Get everyone off the system (crons too!)
Backup what you have (if anything)
Database (even just parts)
Recovery files (AI, BI,TL)
AI Archives
Delete broken database
Restore most recent backup
Apply after image archives
rfutil <db> -C roll forward oplock –a <aifile>
(10.1c)

41. Extending Archival Usefulness
Log-based replication 41

42. Log-Based Replication
Extending archival usefulness
Hot standby database
Good to use for backups
Eliminates backup I/O on production
Use -norecover option with probkup
Less recovery downtime
Cannot be updated (subsequent rollf. will fail)
Hot standby copy (#2)
Provide for reporting (semi-stale)
“Midnight” copy of hot standby #1

43. Log-Based Replication
Extending archival usefulness
Auto archived files
Deal w/awkward naming convention
Create sub-directory under archive directory
Process looks for new archives
Applies new archives to hot-standby database
Moves applied archives
Best if on a remote host
Less contention w/production
Improved disaster recovery strategy

44. Summary All OpenEdge systems should use AI
Many new online AI capabilities
AI management tool provided
Use archives to maintain a hot-standby
Improved high availability 44

45. ?
Questions

46. Outstanding Questions Answered
Does ai archiver use ai extract or simply copy?
It uses Ai Extract.
Is ai archiver available for Workgroup licenses?
Yes.  It is functionality and not performance related.
You can specify several archive directories.  Is there a way to tell which is the current archive directory?
There is a message in the log when a dir switch is performed. There is no way to programmatically determine this. I will add an enhancement request.
When ai is locked due to replication, is the file archived and later marked empty or does the archive process stall waiting for the locked extent to become unlocked?
The extent is archived.  Once it is “unlocked” it is emptied by the watch dog.  The aimgt deamon also attempts to empty locked extents.
Is there any documentation on using ai archiver with replication?
Yes, there is a callout in the ai mgt doc when using it with replication.  The doc explains what happens to an extent with ai mgt and repl running.
Is it safe to change permissions of a file after add online completes?
No. Although add online has completed, all users connected to the database have not opened the new file. They open it the next time they access it. Changing its permissions while the database is still running may cause the database to shutdown when a user attempts to open a file it does not have permission to.

47. Thank You
Thank you in many languages