1. B9: Success with OpenEdge® Replication
What you don’t know can hurt you
David Eddy
Senior Solution Consultant

2. Obligatory (gratuitious) Quotation
An ounce of prevention is worth a pound of cure.
28.35 grams of prevention is worth .45 kilograms of cure

3. Success With Replication
Understand the architecture
Know your requirements
Analyze your system
Plan, configure, deploy,monitor
Succeed 3

4. Agenda Terminology and architecture – Visual
Replication performance hotspots
Replication availability hotspots
Source and target management 4

5. Replication Terminology
Primary – original production machine
Secondary – original recovery machine
Source – production database
Target – recovery database
Replication server – rpserver (source)
Replication agent – rpagent (target)
DBService queue – stores IPC messages
Network pipe – TCP/IP rpserver -> rpagent

6. Architecture of Replication – Primary/Source
Primary Machine
R/Write
Clients
AI Extents
DB Service Q
Repl Server
Source Database
Database Brk/Server Processes ! 6

7. Architecture of Replication – Secondary/Target
Secondary Machine
Repl Agent
Database Brk/ServerProcesses
Target Database
R/O
Clients 7

8. Architecture of Replication
Primary Machine
Secondary Machine
R/Write
Clients
AI Extents
DB Service Q
Repl Server
Source Database
Database Brk/Server Processes
Repl Agent
Database Brk/ServerProcesses
Target Database
R/O
Clients 8

9. Agenda Terminology and architecture - Visual
Replication performance hotspots
Replication availability hotspots
Source and target management 9

10. Database Brk/Server Processes Database Brk/ServerProcesses
Performance Hotspots
Primary Machine
Secondary Machine
R/Write
Clients
AI Extents
DB Service Q
Repl Server
Source Database
Database Brk/Server Processes
Repl Agent
Database Brk/ServerProcesses
Target Database
R/O
Clients 10

11. Source DB Performance Considerations
Speed/power of machine
Replication Plus – offload read only clients to target database
AI files – fixed extents
AI/BI blocksizes should be the same
16 K is the most efficient
Need to truncate ai and bi

12. Target DB Performance Considerations
System should not be underpowered by comparison to the source system
No need for after-imaging
Read only clients

13. Network Performance WAN vs. LAN The bigger the pipe the better
Determine size of pipe
Whitepaper available on PDSN

14. Sample Network Bandwidth Calculation
Hourly After Image Blocks from 5 production databases = 713mb
By calculation:
Replication size = 1.5 * AI size
713mb*1.5 = mb
Add replication overhead (1.1)
1069.5mb*1.1 = mb
Throughput per second:
mb/3600=.32679mb/b
Change to kilobytes /s:
.32679mb/s * 1000 = kb/b needed to sustain transfer

15. Replication Performance – DBService Buffer
DBService Buffer Queue must be sized appropriately – Solution P121969
Failure to do so may result in source slowdown and target falling behind
Analyze AI activity prior to implementing
Use PROMON -> R&D -> Status Display (#1)-> DB Service Manager (#16).

16. Determining Optimal -Pica
The larger the –pica, the further behind the target may get
Monitor AI writes during busiest period for one hour
-pica = (TAIW/Blockcount) * 1.25
Blockcount depends on version
OpenEdge 10 – blockcount is 9.16
Progress® 9.1x – blockcount is 18.2

17. Sample –Pica Calculation
Values up to 8192k for 10.1B01 and later
34560 TAIW over one hour
Formula for OE 10:
(34560/9.16) * 1.25 =
Why not just use 8192???

18. “Houston, we have a problem” - Promon
03/03/ Status: Database Service Manager
17:15:02
Communication Area Size : KB
Total Message Entries :
Free Message Entries :
Used Message Entries :
Registered Database Service Objects
Name Rdy Status Messages Locked by
OpenEdge Replication Server Y RUN
OpenEdge RDBMS Y REG
OpenEdge DB Agent Y RUN

19. Agenda Terminology and Architecture - visual
Replication performance hotspots
Replication availability hotspots
Source and target management 19

20. Availability Hotspots
Primary Machine
Secondary Machine
R/Write
Clients
AI Extents
DB Service Q
Repl Server
Source Database
Database Brk/Server Processes
Repl Agent
Database Brk/ServerProcesses
Target Database
R/O
Clients 20

21. Availability Hotspots
Loss of Database
TCP or Process Failure
Primary Machine
Secondary Machine
R/Write
Clients
AI Extents
DB Service Q
Repl Server
Source Database
Database Brk/Server Processes
DOWN!
Locked
Repl Agent
Database Brk/ServerProcesses
Target Database
R/O
Clients
Severed 21

22. Availability Hotspots – AI Management
Replication DOES NOT manage AI files
AI Files must be emptied and backed up
10.1A AI archiver became available.
Database crashes when no empty AI extents.

23. Availability Hotspots – Locked AI Files
AI “locked” when AI notes not replicated to the target db
Common issue – often caused by simple maintenance routines and failure to monitor status of replication
Configure the rpserver and rpagent to detect and handle outages
All about the *.repl.properties file!!
Configure, test well and sleep easier at night

24. Availability – Many Locked AI Files
Use dsrutil monitor against source and target db. Check for replication shared memory
If rpagent is running, restart the replication server on the source
If rpagent is not running, restart target database and restart replication server

25. “Houston, we have a problem” – part 2
Extent: 1 Extent: 3
Status: Busy Status: Locked
Type: Variable Length Type: Variable Length
Path: C:\wrk101c\repl\source.a1 Path: C:\wrk101c\repl\source.a3
Size: Size:
Used: Used:
Start: Wed May 14 14:09: Start: Wed May 14 14:08:
Seqno: 5 Seqno: 3
Extent: 2 Extent: 4
Status: Locked Status: Locked
Path: C:\wrk101c\repl\source.a2 Path: C:\wrk101c\repl\source.a4
Size: Size:
Used: Used:
Start: Wed May 14 13:45: Start: Wed May 14 14:08:
Seqno: 2 Seqno: 4

26. Availability – All AI Files Locked
Source database activity comes to a halt
Can you afford to bring the db down?
If not - disablesitereplication
If yes
stop database
add new ai files
prostrct reorder
restart.

27. Prostrct add sourcedb addai.st
Extent: Extent: 4
Status: Busy Status: Locked
Type: Variable Length Type: Variable Length
Path: C:\wrk101c\repl\source.a1 Path: C:\wrk101c\repl\source.a4
Size: Size:
Used: Used:
Start: Wed May 14 14:09: Start: Wed May 14 14:08:
Seqno: Seqno: 4
Extent: Extent: 5
Status: Locked Status: Empty
Type: Variable Length Type: Variable Length
Path: C:\wrk101c\repl\source.a2 Path: C:\wrk101c\repl\source.a5
Size: Size: 120
Used: Used: 0
Start: Wed May 14 13:45: Start: N/A
Seqno: Seqno: 0
Extent: Extent: 6
Path: C:\wrk101c\repl\source.a3 Path: C:\wrk101c\repl\source.a6
Size: Size: 120
Used: Used: 0
Start: Wed May 14 14:08: Start: N/A
Seqno: Seqno: 0

28. Prostrct reorder ai sourcedb
Extent: Extent: 4
Status: Busy Status: Locked
Type: Variable Length Type: Variable Length
Path: C:\wrk101c\repl\source.a1 Path: C:\wrk101c\repl\source.a4
Size: Size:
Used: Used:
Start: Wed May 14 14:09: Start: Wed May 14 13:45:
Seqno: Seqno: 2
Extent: Extent: 5
Status: Empty Status: Locked
Type: Variable Length Type: Variable Length
Path: C:\wrk101c\repl\source.a2 Path: C:\wrk101c\repl\source.a5
Size: Size:
Used: Used:
Start: N/A Start: Wed May 14 14:08:
Seqno: Seqno: 3
Extent: Extent: 6
Path: C:\wrk101c\repl\source.a3 Path: C:\wrk101c\repl\source.a6
Size: Size:
Used: Used:
Start: N/A Start: Wed May 14 14:08:
Seqno: Seqno: 4

29. Source and Target DBs Won’t Synchronize
Try restarting a few times.
-Ma, -Mn, -n match?
Changes to .properties files?
If it continues to fail, contact support
Source/target db log files
AI files
pmmgr.properties file
arguments=-logging 2

30. Agenda Terminology and architecture - visual
Replication performance hotspots
Replication availability hotspots
Source and target management 30

31. Managing Replication – DSRUTIL Monitor and Status
Not immediately obvious when a failure occurs
Proactively verify replication performance and status

32. DSRUtil Monitor DSRUTIL source/target –C monitor
Attaches to replication shared memory.
Cannot connect to replication shared memory. Status = -1
Checks status of Server and Agent
Server status
Agent status

33. DSRUtil source –C monitor
OpenEdge Replication Monitor Page 1
Database: C:\wrk101c\repl\source
Database is enabled as OpenEdge Replication: Source
Server is: In Normal Processing
Number of configured agents:
Delay Interval (current / min / max): / 5 / 500
Recovery information:
State: No recovery being performed Agents needing recovery:
Agents connected:
Agents in synchronization:
Transition information:
Type: Manual

34. DSRUtil target –C monitor (page 1)
Database: C:\wrk101c\repl\target
Database is enabled as OpenEdge Replication: Target
Agent:
Name: agent1
ID:
Host name:
State: Normal Processing
Ready: Yes
Critical: No
Method: Asynchronous
Agent is waiting for: Nothing
Maximum bytes in TCP/IP message:
Server/Agent connection time: Wed May 14 13:48:
Delay Interval (current / min / max): / 5 / 500
Transition information:
Type: Manual
The last block received at: Wed May 14 13:51:
Activity information:
Blocks received:
Blocks processed:

35. DSRUtil target –C monitor (page 2)
Blocks acknowledged:
Notes processed:
Transactions started:
Transactions ended:
Synchronization points:
AI Block Information:
Source RDBMS Block (Seq / Block): / 2516
Last Processed Block (Seq / Block): / 2494
Latency Information:
Repl Server behind Source DB by: second(s)
Current Source Database Transaction:
Last Transaction Applied to Target:
Target Current as of (Target, Source): Wed May 14 13:51: , Wed May 14 13:51: with delta of 000:00:00

36. DSRUTIL Status DSRUTIL source/target –C status
Return code indicates current state of replication server or agent.
Good for automated scripts
Grep for return code
Replication User Guide provides details

37. Managing Replication – OE Management
OpenEdge Management 10.1B02/3.1B02+ remotely monitor log files
10.1B02 AdminServer running on the remote system
Attach OE_DB_Replication Log File Ruleset to the log file monitor
Alert/ thrown when message violating rule set appears.

38. OE Management – Replication Log File Rule Set

39. Summary Terminology and Architecture Replication performance hotspots
Replication availability hotspots
Source and target management

40. Success With Replication
Understand the architecture
Know your requirements
Analyze your system
Plan, Configure, Monitor
Succeed 40

41. For More Information, go to…
PSDN
Knowledge Centrum (esupport.progress.com)
3.1B01/10.1B01 upgrade: P122926, P123418, P123420, P123424, P123426, P123427, P123676
sizing –pica: P121969
Documentation
OpenEdge Replication 10.1C docs (
OpenEdge Management 3.1C docs (

42. ?
Questions

43. Thank You