IBM Mainframe-Integration Mainframe Change Data Capture Presented by

Workstation/ Client-Server The current situation: Mainframe Applications + Databases Workstation/ Client-Server Applications + Databases Internet/Web Applications + Databases Mobile Applications + Databases

Mainframe Applications + Databases The current situation: Mainframe Applications + Databases • ADABAS • IDMS/DB • DB/2 • IMS and DL/I • VSAM • DATACOM/DB • other…

Workstation Client/Server Applications + Databases The current situation: Workstation Client/Server Applications + Databases • SQL Server • Oracle • DB/2 UDB • MS Access • other…

Internet/Web- Applications The current situation: Internet/Web- Applications + Databases • Sybase • MySQL • DB/2 UDB • ORACLE • other…

Mobile Applications + Databases The current situation: Mobile Applications + Databases • MS SQL • MS Access • DB/2 UDB • ORACLE • other…

The current situation – Data exchange: Data exchange problems • Different data formats • Different data models • Large data volumes • Limited batch window • Requirement for up-to-dateness of information

The Solution Moving data... → as much as needed → as little... tcVISION – Mainframe Change Data Capture The Solution Moving data... → as much as needed → as little... → as transparent... → as flexible... → as secure... AS POSSIBLE

tcVISION – Mainframe Change Data Capture The new data exchange generation

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Basic questions: 1. How current must the information be? Data exchange between source- and target-system should be ... • continuous / in realtime • cyclic / interval based • event based

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Basic questions: 2. What are the attributes of the datasource? The datasource attributes influence the choice for the data capture method ... • Size of datasource • Format of datasource (Structure, DBMS) • Change frequency • Amount of changes and • available resources (CPU-power, network bandwidth)

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Usage of the DBMS logging capabilities IMS/DB, VSAM, DB/2, DL/I, ADABAS, IDMS, DATACOM Transfer of changed data in scheduled time frame Best for batch window Best for processing right after logfile creation

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Efficient transfer of entire databases Analyzis for data consistancy Best for „Initial Load“ prior to log processing Best for periodic mass data transfer One step data transfer

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Comparison of data snapshots Efficient transfer of changed data since last processing IMS/DB, DL/I, VSAM, DB/2, ADABAS, IDMS, DATACOM, Sequential files Flexible processing options (SORT etc.) Automatic creation of deltas by tcVISION

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Realtime capture of changed data Changes directly obtained from DBMS IMS/DB, VSAM, DB/2, ADABAS, IDMS, DATACOM Secure data storage even across DBMS restart Flexible propagation methods

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Up-To-Dateness of changed data

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Data volume

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Steps: Extraction of database logs Process meta information Process changes of selected tables Propagate changes Implementation into target database

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Steps: Read datasource Process meta information Selective and structured transfer Implementation into target databse

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Steps: Read datasource Read corresponding snapshot Determine changes Process meta information Propagate changed data Implementation into target database

tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Steps: Capture changes in DBMS Transfer changes into „Pool storage“ Parallel processing of changed data from the pool Propagation of data to the target system Implementation into target database

Summary tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Summary Every method has been designed to meet special needs and requirements. An efficient and secure data synchronization is guaranteed with all methods.

tcVISION – Mainframe Change Data Capture Transfer and Propagation Propagation to other mainframe systems Propagation to Internet-databases Propagation to mobile systems Propagation to client server databases

tcVISION – Mainframe Change Data Capture Transfer and Propagation Propagation independent of data origin Propagation to different target systems (even for the same data)

tcVISION – Mainframe Change Data Capture Examples Case 1: DB/2 Log processing and implementation into DB/2 UDB

tcVISION – Mainframe Change Data Capture Examples Case 2: DL/I Real-Time Capturing and implementation into a DB/2 UDB and VSAM

tcVISION – Mainframe Change Data Capture Examples Case 3: IDMS- Realtime Capturing and bi-directional synchronization with ORACLE, DB2/UDB and MS-SQL

tcVISION – Mainframe Change Data Capture Examples DL/I IDMS SQLServer All cases are Customer implementations, that are currently in production.

tcVISION – Mainframe Change Data Capture Examples Steps: Import of the metadata Assignment of target tables Definition of processing script Test and implementation

tcVISION – Mainframe Change Data Capture Examples Import of meta data The tcVISION Manager publishes the layout of the log-record for every table  for later use by the processing scripts

tcVISION – Mainframe Change Data Capture Examples Assignment of target tables Creation of target tables or Assignment of existing tables in the target system.

tcVISION – Mainframe Change Data Capture Examples Definition of a processing script Use the tcVISION script wizard to create the script Definition of all required parameter: Input and output, LUW processing etc.

tcVISION – Mainframe Change Data Capture EXAMPLES Test and implementation Start and test the script Implement script execution into scheduler and automatic processing

tcVISION – Mainframe Change Data Capture Example Technical aspects: Standard data exchange was not possible due to extremely large amount of data. • 760 tables • 200.000 changes per day are processed • Propagation to DB2 UDB under SUN Solaris • Processing time with tcVISION per day less than 30 minutes

tcVISION – Mainframe Change Data Capture Examples Summary Just a few steps to implement a data synchronization process Solid monitoring and logging High stability Changes applied to structures of source databases are automatically processed

tcVISION – Mainframe Change Data Capture Examples Summary Just a few steps to implement a data synchronization process Solid monitoring and logging High stability Changes applied to structures of source databases are automatically processed

tcVISION – Mainframe Change Data Capture Examples Steps: Define a collector and pool storage + Definition of a processing script Creation of target table Test and implementation DL/I

tcVISION – Mainframe Change Data Capture Examples Creation of a collector/pool combination for the DL/I database Definition of a processing script

tcVISION – Mainframe Change Data Capture Examples Define Collector as DBMS Extension Define Pool to buffer the changes Define processing script to implement into the target database Use project wizard to specify the work-flow

tcVISION – Mainframe Change Data Capture Examples Test and implementation Start the project using the tcVISION Control Board

tcVISION – Mainframe Change Data Capture Examples Technical aspects: Standard data exchange was not possible due to extremely large amount of data. • 40 DLI segments • 200.000.000+ records • 150.000+ changes per day • Implementation: Realtime using DBMS Extension Capturing of changes performed by Online- and Batch-programs.

tcVISION – Mainframe Change Data Capture Examples Additional Implementation: If the target DB/2 is not available, changes are automatically saved into a VSAM KSDS  Highest possible recovery Customer performs a mainframe to mainframe synchronization

tcVISION – Mainframe Change Data Capture Examples Steps: Step 1: Create a collector/pool combination + Definition of a processing script Creation of target table Test and implementation SQLServer IDMS

tcVISION – Mainframe Change Data Capture Examples Import of the SCHEMA definitions for the IDMS database SQLServer

tcVISION – Mainframe Change Data Capture Examples SQLServer Define Collector as DBMS Extension Define Pool to buffer the changes Define processing scripts to implement the changes Use the project wizard to define the necessary steps

tcVISION – Mainframe Change Data Capture Examples Test and implementation Start the project using the tcVISION Control Board

tcVISION – Mainframe Change Data Capture Examples Technical aspects: SQLServer Conventional synchronization not possible • Bi-directional updates must be supported • Updates in DB2/UDB, ORACLE and MS-SQL are captured by TRIGGERS • TRIGGERS invoke tcACCESS/ODBC to perform IDMS updates • tcVISION recognizes updates from tcACCESS and prevents a loop-back

Workstation/ Client-Server tcVISION – Mainframe Change Data Capture Mainframe Applications + Databases Internet/Web Applications + Databases Workstation/ Client-Server Mobile Applications + Databases

Workstation/ Client-Server tcVISION – Mainframe Change Data Capture Mainframe Applications + Databases Internet/Web Applications + Databases Workstation/ Client-Server Mobile Applications + Databases

tcVISION – Mainframe Change Data Capture Data exchange challenges: • Different data formats • Different data models • Large data volumes • Limited batch window • Requirement for up-to-dateness of information

tcVISION – Mainframe Change Data Capture Data exchange challenges: • Automatic data conversion into the correct target format • Different data models • Large data volumes • Limited batch window • Requirement for up-to-dateness of information

tcVISION – Mainframe Change Data Capture Data exchange challenges: • Automatic data conversion into the correct target format • Adaptation of the data structure during transfer • Large data volumes • Limited batch window • Requirement for up-to-dateness of information

tcVISION – Mainframe Change Data Capture Data exchange challenges: • Automatic data conversion into the correct target format • Adaptation of the data structure during transfer • Only transfer changed data • Limited batch window • Requirement for up-to-dateness of information

tcVISION – Mainframe Change Data Capture Data exchange challenges: • Automatic data conversion into the correct target format • Adaptation of the data structure during transfer • Only transfer changed data • Batch-Compare with operational database • Requirement for up-to-dateness of information

tcVISION – Mainframe Change Data Capture Data exchange challenges: • Automatic data conversion into the correct target format • Adaptation of the data structure during transfer • Only transfer changed data • Batch-Compare with operational database • Real-Time Capturing meets maximum requirements for up-to-date information

tcVISION – Mainframe Change Data Capture The state-of-the-art demand for Mainframe data exchange: • Automatic data conversion into the correct target format • Adaptation of the data structure during transfer • Only transfer changed data • Batch-Compare with operational database • Real-Time Capturing meets maximum requirements for up-to-date information

YOU will profit from: • Efficiency • Transparency • Auditability tcVISION – Mainframe Change Data Capture YOU will profit from: • Efficiency • Transparency • Auditability • Up-to-dateness

www.treehouse.com