1. Oracle® Streams for Near Real Time Asynchronous Replication Nimar S. Arora Oracle USA

2. 2 Overview  Information Sharing  Replication Uses  Desirable Characteristics of Replications  Misconceptions in Research  Streams Replication  Performance Assumptions  Parallel Apply

3. 3 Information Sharing  Get information from those who have it and give to those who need it  Capture and Consumption of Information  Explicit Capture and Consumption is Messaging  Implicit Capture and Consumption is Replication  Cross Product  Cross Platform

4. 4 Replication Uses  High Availability  Disaster Recovery  Data Warehouse Loading  Online Upgrade  Etc..

5. 5 Desirable Characteristics of Replication  Zero or low latency – Easy to detect and resolve inconsistencies  Zero or low impact on OLTP performance – Online instantiation  Keep up with any workload – Large or small transactions – DDLs and DMLs  Flexible – Arbitrary topology over LAN or WAN – Arbitrary data filtering and transformations

6. 6 Misconceptions in Research  Inconsistencies are hard  Asynchronous replication replicates a transaction after it commits on the source  Synchronous replication doesn’t hurt source performance

7. 7 Streams Replication EMP Apply Propagation Empno | job |.. 7901 | sales |… 7902 | coding |... Queue Capture Update scott.emp set job=‘coding’ where empno=7902 Redo Log ack Logical Change Record (LCR)

8. 8 Performance Assumptions  LCRs can be processed in memory faster than they can be written to storage  Network bandwidth is comparable to storage bandwidth  Network latencies are irrelevant when streaming data without acknowledgement  Applying LCRs is slower than storage bandwidth (hence parallel apply is a must)

9. 9 Parallel Apply  Respect all database constraints – Primary key – Unique key – Foreign key  Respect commit order for non-database constraints  For DDLs, respect table locks

10. 10 Parallel Apply (sample schema) SCOTT.EMP: NameType ----------------------------------------------------------- EMPNO NUMBER(4) ENAME VARCHAR2(10) JOB VARCHAR2(9) MGR NUMBER(4) HIREDATE DATE SAL NUMBER(7,2) COMM NUMBER(7,2) DEPTNO NUMBER(2) OFFICE NUMBER(3) Primary Key - Foreign Key - Unique Key -

11. 11 Parallel Apply (example) Source Replica T1: Insert emp.. (7902,.. T1: Commit T2: update emp.. Empno=7901 T2: update emp.. Empno=7902 T2: Commit Row dependency Commit order dependency T1: insert emp.. (7902,.. T1: commit T2: update emp.. 7901 T2: update emp.. 7902 T2: commit T3: insert emp.. 7904 T3: commit Apply T3: Insert emp.. (7904,.. T3: Commit

12. 12 Parallel Apply (example) FOREIGN KEY: T4: Insert into emp (empno,..) values (7905,..); Commit; T5: Insert into emp(empno,mgr,..) values (7906, 7905..) Commit; UNIQUE CONSTRAINT: T6: update emp set office=‘102’ where office=‘101’ Commit; T7: update emp set office=‘101’ where office=‘100’ Commit; Supplemental logging at source for multi-column constraints

13. 13 Parallel Apply (Dependency Hash Table) For each row change, each constraint, compute hash value Hash value = hash of Overwrite transaction id in each of the hash slots Depend upon prior transaction id in each of the hash slots Example T2 depends upon T1: xid 7901: 7902: …. T1 T2

14. 14 Q & Q U E S T I O N S A N S W E R S A