1 DPSync Overview

2 Agenda The Problem and the Options Concepts of ADABAS-to-RDBMS Replication A Brief History of ADABAS-to-RDBMS Replication DPSync: How It Works Points of Distinction Customer Benefits Summary: Ask the Difficult Questions

3 The Problem ADABAS Web Services Web Services SQL Access SQL Access Remote Access Remote Access Data Warehouse Data Warehouse Sophisticated Analysis Sophisticated Analysis Graphics Graphics SOAP SOAP Other Software Packages Other Software Packages Etc. Etc. This Is What I Have…This is What I Need… Legacy COBOL Programs Legacy COBOL Programs NATURAL Programs NATURAL Programs Batch Jobs Batch Jobs RDBMS

4 ADABAS Web Services Web Services SQL Access SQL Access Remote Access Remote Access Data Warehouse Data Warehouse Sophisticated Analysis Sophisticated Analysis Graphics Graphics SOAP SOAP Other Software Packages Other Software Packages Etc. Etc. What I Have What I Need Legacy COBOL Programs Legacy COBOL Programs NATURAL Programs NATURAL Programs Batch Jobs Batch Jobs RDBMS The Options Rewrite All the Legacy & NATURAL Programs Replication via PLOG-Based Periodic Updates Replication via Extract-Based Periodic Updates Real-Time ADABAS-to-RDBMS Replication

5 The Replication Requirements Continuum Timeliness of Replication Complexity of Replication Management

6 Concepts Of Real-Time ADABAS-to-RDBMS Replication

7 Analyze the ADABAS data content Design the RDBMS Produce all the metadata for the RDBMS and the transformation Design, develop and test application programs* Extract the ADABAS Data, Load the Initial RDBMS Activate the Real-Time Process For - Intercepting the ADABAS Updates - Transforming to Relational - Transmitting of the Updates to the RDBMS Server - Applying the updates to the RDBMS On-going operations, including recovery from exceptions, resynchronization of the databases and recovery from RDBMS errors On-going maintenance of customer-written programs* On-going maintenance of metadata * Not applicable for DPSync Whats Involved In Implementing A Real-Time ADABAS-to-RDBMS Replication Application Concepts

8 1.Analysis and RDBMS Design Tools 2.Extraction, Transformation, and Load (ETL) Facilities 3.Real-Time Interception and Delivery of Changes 4.Sophisticated and Complex Transformations (from ADABAS to RDBMS structures and data), Including Transforming PEs and MUs to Child and Grandchild Tables 5.Application of the Updates, RDBMS Error Notification, and Recovery 6.Special Functions (e.g., Synchronized Backups and Restores) 7.Console Facilities for Statistics and Operations The Components of an ADABAS Replication Software Product Concepts

9 1.Robustness of features 2.Ease of implementation, including no requirement for custom- developed software 3.Ease of operation (i.e., procedures for running synchronized backups/restores, handling errors, etc.) 4.Ease of maintenance (software and metadata) 5.Performance, including any impact on the production ADABAS system and use of ADASAV and high-speed RDBMS loaders 6.Reliability -- software maturity 7.Data integrity, including guarantees that all the changes will be replicated, in their original sequence, without duplicates, and without any lost data 8.Scalability and Deployment Options 9.An End-to-End Solution ADABAS Replication Software Products: Attributes To Look For Concepts

10 A Brief History of ADABAS-to-RDBMS Replication Software

11 History of ADABAS-to-RDBMS Replication Software A innovator in ADABAS replication software for over since 1982 Since 1995, our products have been used in the analysis, design, implementation and maintenance of ADABAS-to-RDBMS replication processes We have listened to the needs of ADABAS customers, and for the last decade we have supplied software for ADABAS replication for data warehouses and other applications Treehouse Software Its Not a New Idea

12 History of ADABAS-to-RDBMS Replication Software An understanding of the needs of real customers An awareness of the technical pitfalls Robust features, as a result of 6 incremental releases of the product Broad Array of Target Platforms: Oracle, DB2, Sybase, SQL Server, UNIX, z/OS, Windows A mature, stable and reliable software product DPSync: 1995 to 2009 Treehouse: What Does 14 Years Give You?

13 History of ADABAS-to-RDBMS Replication Software A Complete Solution No Programs To Develop Guarantee No Lost Data ETL via ADASAV & High-Speed RDBMS Loaders Robust Features Sophisticated Transformations Reliability Data Integrity Ease of Use Ease of Maintenance RDBMS Vendor Independence Stability Maturity DPSync ( ) What Does 14 Years Give You?

14 DPSync How It Works System Overview Analysis and Design Materialization (ETL) Real-Time Replication

15 DPSync – How It Works System Overview

16 Analyze actual ADABAS data content Generate, view and modify system-recommended RDBMS schemata Select an RDBMS schema Modify the schema Define conversion processes Map ADABAS files/fields to RDBMS tables/columns Result generates the DPSync Metadata and RDBMS DDL statements Intuitive interactive user interfaces, including GUI Analysis and Design DPSync – How It Works

17 Statistics of MUs and PEs for sizing of child tables and potential de-normalization of tables to individual column(s). ADABAS File Analysis

18 Statistics of alphanumeric fields for candidate variable character text columns. ADABAS File Analysis

19 ADABAS File Analysis This screen shows descriptor/superdescriptor usage statistics to determine candidate Primary Keys and indexed columns.

20 RDBMS Modeling and Mapping Auto Generation Example: Four tables with Primary Key and Foreign Key constraints, and the added DPS PE Sequencer (PE occurrence).

21 Standard utilities – no programs to write or maintain Automatic DDL generation High speed extraction via ADASAV Utility High speed RDBMS loaders Complex data transformations and conversions No Problems with concurrent ADABAS updates Materialization (ETL) DPSync – How It Works

22 Real-Time Replication - Overview DPSync – How It Works

23 Real-Time Replication: Interception of the Changes Standard ADABAS Replication Exit Volume Spikes Handled Gaps Are Managed No Impact on ADABAS Performance DPSync – How It Works

24 Real-Time Replication: Transformation & Delivery DPSync – How It Works Automated Analysis and Design Sophisticated Transformations, including Child and Grandchild Tables from PEs and MUs Paced SQL TCP/IP Transmissions Only ETed Transactions Shipped

25 Real-Time Replication: Applying The Updates No Programs To Write Sophisticated RDBMS Structures, Including Child & Grandchild Tables from MUs and PEs Adheres To RDBMS Foreign-Key/Referential Integrity Constraints Paced SQL TCP/IP Transmissions and Queue Journal and Error Log Automatic RDBMS Error Notification via Built-In Error Recovery via Scripts Multi-Threaded DPSync – How It Works

26 Real-Time Replication: DPSync Console Statistics System States Error Recovery Scripts Run-Time Parameters Journal and Error Log Display Error Retry Security DPSync – How It Works

27 Real-Time Replication – DPSync Console : Statistics and System States Latency Times Queue Times RDBMS Times Number ADABAS Transactions Number SQL Statements Number of Errors System Paused System Halted DPS/R-RSU Connection States Overruns (Volume Spikes) Etc. DPSync – How It Works

28 Real-Time Replication – DPSync Console: Maintaining Error Recovery Scripts Conditions and Groups Definitions Groups Actions by Error Type: Pause Retry Halt the System Write to Error Log Notify via Ignore Options By Error Type Text to Send Etc. DPSync – How It Works

29 Real-Time Replication – DPSync Console: Run-Time Parameters DPSync – How It Works

30 Real-Time Replication – DPSync Console: Journal and Error Log Display DPSync – How It Works

31 Real-Time Replication – DPSync Console: Security DPSync – How It Works

32 Real-Time Replication – DPSync Console: Other Console Functions DPSync – How It Works

33 Points of Distinction Loading the Initial RDBMS (ETL) Loading the Initial RDBMS (ETL) Lost (Replication) Data Lost (Replication) Data Detection and Notification of RDBMS Errors Detection and Notification of RDBMS Errors Ease of Implementation Ease of Implementation Ease of Maintenance Ease of Maintenance Transformation Facilities Transformation Facilities The Total Solution The Total Solution DPSync vs. Other Software Products

34 Two Ways to Load the Initial RDBMS 1.Full Live Extracts 2.Use High-Speed ADASAV and High-Speed RDBMS Loaders Points of Distinction

35 ADABASReplicatorMemoryQueuesMsgSoftware ExampleTransfor-mation JDBC App. ExampleJDBCApp. RDBMS Customer-Written Application Program Complex Transformations Extracts For Extracts For RDBMS Load ADABAS Replicated Changes Live ExtractLive ExtractUtility Before/After Images Simplex Transformations Flawed Method For Loading the Initial RDBMS No Use of High-Speed Loaders Concurrent ADABAS Updates Can Cause RDBMS Errors Time Constraints Requires Special Programming Flaws: Use Live Data Extracts Points of Distinction

36 ScenarioFlaw 1. Concurrent ADABAS DELETE occurs before the loader has extracted the base record. RDBMS replication of the DELETE fails due to row not found. 2. Concurrent ADABAS INSERT occurs before the loader has extracted the base record. A duplicate row is both replicated and then loaded to the RDBMS. 3. Concurrent ADABAS UPDATE occurs before the loader has extracted the base record. RDBMS replication of the UPDATE fails due to row not found. Etc. Flaws In Live Data Extracts Loading the Initial RDBMS Lock-out ADABAS Updates during the entire RDBMS Load Custom Programming (Customer-Written Loader Program) Alternatives for Circumvention Points of Distinction

37 DPSync: Loading the Initial RDBMS Standard Utilities – No Programs to Write Replications Are Accepted and Queued Until the RDBMS Load completes Built-In Sophisticated Transformations High-Speed Points of Distinction

38 Two Ways to Handle Lost Data 1.Lose Data, But Recover It Using Live Data Extracts 2.Replication Software Guarantees No Lost Data Points of Distinction

39 Replication Data is Lost ADABASReplicatorMemoryQueuesMsgSoftware ExampleTransfor-mation JDBC App. ExampleJDBCApp. RDBMS Customer-Written Application Program Complex Transformations Lost Data Recovery Lost Data Recovery ADABAS Replicated Changes Live ExtractLive ExtractUtility After Images Only Simplex Transformations Flawed Process for Handling Lost Data What Should Be Refreshed? Missing Before Image: Was the Record Inserted? DELETEs Are Not Handled Replicates Unmodified Records Referential Integrity RDBMS Constraints May Be Violated PE/MU Child/Grandchild Tables Handled? Requires Special Programming Flaws: Use Live Data Extracts to Recover It Points of Distinction

40 DPSync: Guarantee No Lost Data Points of Distinction

41 No Programs To Write Automatic RDBMS Error Notification via Built-In Error Recovery via Scripts Console Display of RDBMS Errors Console May Retry Erroneous Transactions DPSync: Notification of and Recovery From RDBMS Errors DPSync Points of Distinction

42 DPSync: Ease of Implementation Built-In Analysis and Design Tools – Produces All the Metadata and RDBMS DDL 100% Off the Shelf – No Programs to Write Requires No Third-Part Messaging Software Includes ETL Facility No Modifications to ADABAS Applications Uses Existing TCP/IP Network Points of Distinction

43 DPSync: Ease of Maintenance & Operation No Programs to MaintainNo Programs to Maintain Off-the-Shelf Software Ensures Future CompatibilityOff-the-Shelf Software Ensures Future Compatibility No Data Recovery Operational Procedures Needed Due to Lost DataNo Data Recovery Operational Procedures Needed Due to Lost Data Facilities for Synchronized Backups and RestoresFacilities for Synchronized Backups and Restores Tools For Maintaining the MetadataTools For Maintaining the Metadata Built-In Statistics and Console FunctionsBuilt-In Statistics and Console Functions Treehouse 24x7 Technical Support is Renowned WorldwideTreehouse 24x7 Technical Support is Renowned Worldwide Points of Distinction

44 Map ADABAS data to multiple, independent RDBMS schemata Value-based mapping and handling of ADABAS record types Transform PEs and MUs to child and grandchild tables Adheres to RDBMS foreign-key/referential integrity constraints for child and grandchild tables Natural dates and binary fields Pseudo data, such as ISN, PE/MU occurrence numbers, DBID, FNR, etc. Ability to normalize PEs and MUs into single rows Concatenate ADABAS fields into a single RDBMS column Extract ADABAS substrings (offset, length) Insert Constants (literals) Transformations are Built-In (Not a Customer-Written Program) Etc., on and on… DPSync: Sophisticated Transformation Facilities Points of Distinction

45 Operations Maintenance Analysis Design ADABAS RDBMS Extraction, Trans- formation & Load Real-Time Replication No Programs to Write Guarantees No Lost Data High-Speed ADASAV and RDBMS Loaders PEs and MUs Transformed To Child Tables All the Tools Needed All scenarios served: ETL,Choose Your Time Replication, Real-Time Replication Built-InSophisticatedTransformationFacilities Legacy COBOL Programs NATURAL Programs Batch Jobs DPSync: The Total Solution Points of Distinction Windows Console

46 Customer Benefits A product, not a consulting engagement No impact on the production ADABAS system Cost-effective implementation Time-proven vendor commitment to ADABAS- to-RDBMS Replication Software Over 13 years of product evolution, from a company with over 30 years of ADABAS experience

47 ADABAS-to-RDBMS Replication Ask The Difficult Questions Real-Time Summary

48 Summary: Ask The Difficult Questions End-to-End Total Solution: Analysis, Design, ETL, Real-Time Replication, Operations Maintenance Guaranteed No Loss of Data Analysis and Design Tools Built-In Sophisticated & Complex Transformations PEs and MUs Transformed to Child and Grandchild Tables, Adherence to RDBMS Foreign-Key/Referential Integrity Constraints Automatic Generation of RDBMS DDL Uses ADASAV and High-Speed RDBMS Loaders No Need for Full Live Extract Programming for ETL Special Functions, Such as Synchronized Backups Exception Processing Notification ( ) and Recovery Facilities for RDBMS Errors Windows-Based Console for Statistics and Operations Easy to Implement, Use and Maintain Reliable, Mature Software – 11 Years Evolution No Programs to Write DPSync – A Mature Product That Meets the Challenge Are These Issues Important To Me?

49