1. Introduction to Heterogeneous Data Replication Spring COMMON 1999 Richard Sinn IBM Santa Teresa Lab

2. Agenda n What is and Why Replication ? n Introduction to IBM Data Propagator Relational (DPropR, DPR) –DB2 to DB2 family including DB2/400 n Introduction to DataJoiner –DB2 to non-DB2 –non-DB2 to DB2 –non-DB2 to non-DB2 n Putting Them Together

3. What is Replication ? n Moving Data from one place to another n Replication vs Copying n From Source to Target n Maintain changes automatically

4. Why Replication ? n Data is everywhere n Use Information Intelligently and improve decision making n Stay competitive with both Data access and Data movement n Speedup application deployment n Reengineer business processes n Increase online throughput n Improve system availability n Support audit requirements n Support Data Warehousing

5. IBM Data Propagator Solution n Multiple platforms (AS/400, NT, AIX, etc) n Supported by IBM n Simple Administration n Proven Technology n Speed Implementation with IBM services organization n Built on standard SQL

6. DPropR Components Capture Apply Admin Tools Source Server Target Server Control Server (Flexible location) Windows NT, Windows 95, OS/2 Platform Web Browser

7. Replication Admin Tools n Define replication sources –Also called Registration –This is what Capture works with logically n Defines subscriptions –This is what Apply works with logically n Create replication control tables –AS/400 DPR install program will create all the necessary replication control tables

8. Admin Tool: DJRA

9. Define Registration n Register a source table as a replication source n Process data using vertical fragmentation –Replicate only selected columns n Select before and after image of a column

10. Registration Capture Registered Source Tables At the Source Server

11. Registration Using DRJA

12. Define Subscription n Subscribe to registered table n Defines source and target relationship –For example, many target tables can subscribe to one source table n Horizontal fragmentation –Columns can be further sub-selected

13. Subscription Apply Target Tables Subscription Definitions Registered Source Tables

14. Subscription Using DJRA

15. Replication Flow n Source data changes are captured by Capture component according to registration n Change Data are then put into Staging Tables n According to subscription, Apply component apply changes to target(s) using the data in Staging Tables

16. The Capture Component Journals, Logs Capture Registered Source Tables Staging Tables Unit of Work Table At the Source Server Register Table

17. Journal/Log based Capture n Capture reads the DB2/400 Journal –Non-AS/400 DB2 will be reading Log n Inserts rows into the change data table –One of the staging tables n Time stamps or relative byte address is used as a log sequence number n Prunes the change data table n Communicates with Apply via ASN/IBMSNAP_PRUNCNTL table

18. The Apply Component Apply Register Table Staging Tables Unit of Work Table Target Tables Subscription Definitions

19. Capture and Apply

20. DPropR Functions n Subscription Set n Time based Replication n Event based Replication n Multiple Target Types n Blocking Factor n Update Anywhere n Programming Capability –SQL Statement / Call Procedure –ASNDONE –ASNLOAD n Support Transaction Consistency

21. Different Target Types n User copy tables –Represent source data at a particular point in time n Point-in-time tables –Represent source data at a particular point in time plus some overhead columns n Replica tables –Updates to a Replica table are automatically applied to the original source table of the Replica, Auto conflicts detection

22. Consistent Change Data n Consistent Change Data (CCD) n Maintain complete histories of data changes (Audit) n Condense hot spot updates before transmitting data n Loading dock n Used by NotesPump or other applications

23. Agenda n What is and Why Replication ? n Introduction to IBM Data Propagator Relational (DPropR, DPR) –DB2 to DB2 family including DB2/400

24. What if I have non-DB2 data? What Happens to my: n Oracle Database n Microsoft SQL Server n Sybase SQL Server n Informix Database n And all the applications that built with them ?

25. Introduction to DataJoiner n Transparent Access to Heterogeneous Data Sources n Transparent DDL Support n Heterogeneous Database Joins n Global Optimization n Two-Phase Commit Support n Integrated Data Replication n Spatial Data Management/Access

26. DataJoiner Overview Setup n NT Server (on IPCS) n Install DJ n Install other DBs protocols –SQL*Net for Oracle –Informix-net for Informix –ODBC drivers or Sybase open client for MS SQL Server or Sybase SQL Server Basic Data Access n On NT with DJ: –Create database –Connect to DB –Create Server Mapping (Create Nickname) n Setup client to go to NT with DJ n All access will map to real target DB

27. DataJoiner Overview NT Server IPCS DataJoiner DB Protocol Client System Non-DB2 (e.g. Oracle) Non-DB2 (e.g. Oracle)

28. Total Access with DataJoiner

29. DB2/Non-DB2 Replication AS/400 with DPR/400 (Capture/400 Running; Could be any DB2 family) AS/400 with DPR/400 (Capture/400 Running; Could be any DB2 family) Source Server DataJoiner on NT IPCS Apply Running DataJoiner on NT IPCS Apply Running Sybase, Informix, Oracle Microsoft SQLServer, DB2 Family Sybase, Informix, Oracle Microsoft SQLServer, DB2 Family Target Server Admin (DJRA)

30. Non-DB2/Non-DB2 Replication DataJoiner on NT IPCS Apply Running DataJoiner on NT IPCS Apply Running Sybase, Informix, Oracle Microsoft SQLServer, DB2 Family Sybase, Informix, Oracle Microsoft SQLServer, DB2 Family Target Server Capture Triggers Using CCD Sybase, Microsoft Oracle, Informix SQL Anywhere Source Server Admin (DJRA)

31. Trigger Based Capture I n Trigger: Wake up and do actions based on event (such as insert, delete, update) n Insert, Update, and Delete triggers created on the source table n Committed changes inserted into the CCD table n Capture both before and after image of the column

32. Trigger Based Capture II n Prunes CCD tables via trigger on IBMSNAP_PRUNCNTL table –Clean up CCD n Update Synchpoint via Trigger on IBMSNAP_REG_SYNCH Table –Communication with Capture

33. Trigger Based Capture III n Log sequence numbers generated dependent on OEMs n Pre-existing Triggers –Run-time error –Check by DJRA and inform users –Depends on OEM DB

34. Heterogeneous Replication

35. n DJRA tolerates the pre-existing target table n DataJoiner does the data type mapping n Read only copies (one direction with User Copy, PIT) n No Support for Replica yet n Target table created in data source n Nickname for target table created in DJ Database

36. Putting Them Together Replication n DPR on AS/400 and other platforms n Integrated support with DPNR, Data Refresher, NotesPump n Expert Info and Support from IBM n Proven Technology Heterogeneous Data Access n DataJoiner on IPCS n Server consolidation advantages with IPCS

37. Putting Them Together n Data from Any to Any n Replicate Changes n Time & Event based replication n Update Anywhere n Data Transformation for Datawarehouse n Build Powerful Applications

40. Reference Information n http://www.software.ibm.com/data/dpropr n http://www.software.ibm.com/data/datajoiner n http://www.as400.ibm.com n http://as400bks.rochester.ibm.com/

41. Trademark Information n AIX, AS/400, DataHub, DataJoiner, DataPropagator, DataRefresher, DB2, DProp, DRDA, IBM, IMS, MVS, OS/2, OS/400, RS/6000, VM/ESA and VSE/ESA are trademarks or registered trademarks of the IBM Corpopration in the United States or other countries or both. n Microsoft, Windows, and the Windows 95 and the Windows NT logos are trademarks or registered trademarks of Microsoft Corporation. n UNIX is a registered trademark in the United States and other countries licensed exclusively through X/Open Company Limited. n Lotus Notes is a trademark of the Lotus Development Corporation. n Other company, product, and service names may be trademarks or service marks of others.