CERN IT Department CH-1211 Genève 23 Switzerland t Streams new features in 11g Zbigniew Baranowski

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 2 Outline Combined capture and apply Automatic split and merge procedures Compare and converge Streams performance advisor Other features

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 3 Combined capture and apply Optimization of data flow management Uses direct connections between capture and apply process It is applied automatically if required conditions are satisfied

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 4 Combined capture and apply CC&A requirements – The capture process's queue must have a single publisher, and it must be the capture process – A propagation must be configured between the capture process's queue and the apply process's queue. There can be no intermediate queues – Each apply process that applies changes from the same source database must use a different queue

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 5 Combined capture and apply Improved flow control (new capture process behavior) –No subscribers => capturing paused –Apply or propagation disabled => target removed from subscribers list => no LCRs accumulation in source queue –Target re-enabled => target added to the subscribers list => automatic re-capturing of missing changes –Controlling the oldest applied SCN of subscribers –Constant max queues capacity messages for source queue 1000 messages for destination queue –No spilling from queues Max throughput increased about x10

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 6 Combined capture and apply Point in time recovery automatic handling –No additional steps are required to restart replication –Capture needs an access to the old archive logs before recovery –Recovery steps Stop capture, apply and propagation Perform point in time recovery Start apply, propagation and capture Start SCN is determinated automatically

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 7 Combined capture and apply

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 8 Automatic split & merge Are split & merge procedures needed with combined capture and apply? Split of broken stream procedure is required: –to keep replicas in archive log window –to reduce number of LCRs to be re-captured by capture process Start SCN = min {SCN app1, SCN app2, …} All recaptured LCRs will be propagated to all destinations, but applied only by one.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 9 Automatic split & merge Replacement of our custom scripts Three ways of performing split and merge operation –SQL script generation for manual execution –Splitting and merging procedures DBMS_STREAMS_ADM.SPLIT_STREAMS DBMS_STREAMS_ADM.MERGE_STREAMS –Automatic replication management (new in 11.2) New capture parameters –split_threshold –merge_theshold Monitor job running periodically

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 10 Automatic split & merge DBA_STREAMS_SPLIT_MERGE DBA_STREAMS_SPLIT_MERGE_HIST

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 11 Compare and converge DBMS_COMPARSION –Shared object synchronization package Compare rows in an object at 2 database Converge objects in case of differences –Supported database object types Tables Single-tables views Materialized views Synonyms for tables, single-tables views and materialized views –Version supporting: Local >= 11.1 Remote >=10.1.0

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 12 Compare and converge –Comparison against different object types e.g. table vs. materialized view Column level synchronization –Scan options Identifying row differences (in separate table) Random portion of data –Percentage of data to be scanned –Cyclical way Custom portion of data (rowid range) –No support for: LONG, LONG RAW, ROWID, CLOB, NCLOB, BLOB, BFILE User defined types

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 13 Compare and converge DBA_COMPARISONGeneral information about comparsions in the local databases DBA_COMPARISON_COLUMNSInformation about columns compared by each comparsion in local database DBA_COMPARISON_SCANInformation about each scan run result DBA_COMPARISON_SCAN_VALUESInformation about range of rows compared during scan DBA_COMPARISON_ROW_DIFDetailed information about row differences found in the each scan run. Populated when compare run with perform_row_dif parameter.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 14 Compare and converge Compare & converge steps –Create an comparison on object DBMS_COMPARISON.CREATE_COMPARISON –Perform comparison DBMS_COMPARISON.COMPARE –If necessary do the conversion DBMS_COMPARISON.CONVERGE –Local wins –Remote wins

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 15 Compare and converge

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 16 Compare and converge

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 17 Compare and converge Test result for artificial data with data set of 20 x varchar2 columns 17B each – Comparisons of consistent data– 18 MB/s inconsistent data – 1,7 MB/s purged table – 6 MB/s – Convergence of inconsistent data – 4 MB/s purged table – 3,5 MB/s

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 18 Streams performance advisor DBMS_STREAMS_ADVISOR_ADM – Analyzes local and remote (via dblink) database configuration – Provides Oracle streams topology representation –By assignment of to each stream path Streams performance statistics –Component level –Session level – Gathers streams related metrics with snapshots and compares them DBMS_STREAMS_ADVISOR_ADM.ANALYZE_CURRENT_PERFORMANCE; – Stores statistics in set of dictionary views

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 19 Streams performance advisor dictionary views DBA_STREAMS_TP_COMPONENT contains information about each Oracle Streams component at each database DBA_STREAMS_TP_COMPONENT_LINK contains information about how messages flow between Oracle Streams components DBA_STREAMS_TP_COMPONENT_STAT contains temporary performance statistics and session statistics about each Oracle Streams component. DBA_STREAMS_TP_DATABASE contains basic information about each database that contains Oracle Streams components (name, version, last advisor run etc.) DBA_STREAMS_TP_PATH_BOTTLENECK contains temporary information about Oracle Streams components that might be slowing down the flow of messages in a stream path

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 20 Streams performance advisor UTL_SPADV Package –Replacement of strmmon command line tool –Monitoring of whole stream path –Live streams data collector –Implemented with pl/sql procedures Stores measurements in db tables –Monitoring management UTL_SPADV.START_MONITORING UTL_SPADV.STOP_MONITORING UTL_SPADV.SHOW_STATS

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 21 Streams performance advisor UTL_SPADV Package –Pre run grant execute on SYS.DBMS_LOCK to … –Example output PATH 1 RUN_ID 3 RUN_TIME 2008-MAR-20 10:52:22 CCA Y | CAPTURE_HNS=>HUB.EXAMPLE.COM LMR 95% 0% 3.3% "" LMP (1) 86.7% 0% 11.7% "" LMB 86.7% 0% 11.7% "" CAP+PS E % 71.7% 11.7% "" | SPOKE1.EXAMPLE.COM=> 98.5% 0% 0.6% "" | "STRMADMIN"."DESTINATION_SPOKE1" | APPLY_SPOKE APR 93.3% 0% 6.7% "" APC 98.1% 0% 1.8% "" APS (1) % "CPU + Wait for CPU" | "NO BOTTLENECK IDENTIFIED"

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 22 Other features LCRs tracking – By adding a tag to each captured LCR – V$STREAMS_MESSAGE_TRACKING Synchronous Capture – Uses internal mechanism for capturing changes (DML only) instead of logminer

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 23 Other features Oracle Streams Jobs Use Oracle Scheduler instead of DBMS_JOB Automated Alerts for Oracle Streams Clients and Thresholds in EM Support for –Transparent Data Encryption –XMLType Columns –Virtual Columns –Compressed table (in 11.2)

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 24 Bugs discovered Propagation view population –After recreation of processes all propagation metrics in v$propagation_sender –Assigned to the Oracle development Unable to split a stream as strmadmin user – ORA-02289: sequence does not exist –Fixed by: grant select on STREAMS$_SM_ID to strmadmin

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Zbigniew Baranowski 25 Conclusions Real time replication – About 10 times faster than 10g – Reduced resource consumption Easier streams recovery – Split & merge – Point in time recovery – Compare & converge synchronization More diagnostic and monitoring tools – Enterprise Manager – Streams Performance Advisor – LCRs tracking