We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byLamont Whitus
Modified over 2 years ago
10/10/2014 DB2 for z/OS Distributed Processing Fundamentals Bill Arledge Consulting DB2 Product Manager BMC Software, Inc.
© Copyright 10/10/2014 BMC Software, Inc 2 Overview Distributed Processing Fundamentals Distributed threads – What can I see? WLM, enclaves and SRBs DDF and DB2 system considerations Reporting on DB2 DDF activity
© Copyright 10/10/2014 BMC Software, Inc 3 Basic Concepts – Intro What is “Distributed” If not further qualified: › Could mean a distributed platform or database –UNIX, AIX, Windows –Oracle, DB2 LUW, and others › Could mean distributed applications –Portions of data for a single application implemented across multiple DB2s, possibly geographically dispersed –Data distributed vertically or horizontally › DB2 on z/OS as a backend database server –Application UI and business logic implemented on other platform –Requesters coming from multiple front-ends using different technologies – This is our focus for today TABLE DB2 PRD4 DALLAS TABLE DB2 PRD3 HOUSTON TABLE DB2 PRD1 AUSTIN Global Inventory Table
© Copyright 10/10/2014 BMC Software, Inc 4 Distributed Processing Basic Concepts – Units of Work TABLE DB2 PRD1 AUSTIN TABLE DB2 PRD5 SAN JOSE Remote Request One SQL Statement Single DB2 instance S1 TABLE DB2 PRD4 DALLAS Remote Unit of Work Multiple statements Multiple tables Single DB2 instance S1 S2 TABLE DB2 PRD3 HOUSTON Distributed Unit of Work Multiple statements Multiple tables Multiple DB2s Single DB2 per statement TABLE DB2 PRD3 ATLANTA S1 S2 TABLE DB2 PRD2 PHOENIX Distributed Request Multiple statements Multiple tables Multiple DB2s Multiple DB2s/statement TABLE DB2 PRD5 SAN JOSE S1
© Copyright 10/10/2014 BMC Software, Inc 5 Distributed Fundamentals SQL Access System Directed Access - Three Part Names - Alias - Supported by DRDA and PP SELECT * FROM PRD1.RNDWDA.ORDER WHERE QUANTITY BETWEEN 1 and 100 or... CREATE ALIAS RNDWDA.AUSTIN_ORDERS FOR PRD1.RNDWDA.ORDER SELECT * FROM RNDWDA.AUSTIN_ORDERS › Application Directed Access –Explicit CONNECT by application –Supported by DRDA only –Remote BIND required EXEC SQL CONNECT TO PRD1 EXEC SQL CALL PROCONE › Remote Stored Procedure Call –Explicit CONNECT by application –DRDA only PROCEDURE PROCONE EXEC SQL SELECT..... EXEC SQL UPDATE..... EXEC SQL CONNECT TO PRD1 SELECT * FROM RNDWDA.ORDER
© Copyright 10/10/2014 BMC Software, Inc 6 Distributed Fundamentals Connectivity Options DB2 PRD5 SAN JOSE TABLE DB2 PRD1 TABLE DRDA or PP DB2 Connect ESE IBM Data Server Drivers DRDA Data Server Drivers include: IBM Data Server Client IBM Data Server Runtime Client IBM Data Server Driver for JDBC and SQLJ IBM Data Server Driver for ODBC and CLI **** Other vendors provide distributed data access drivers
© Copyright 10/10/2014 BMC Software, Inc 7 Basic Terminology Definitions Location (DB2 for z/OS term) - Or: RDB-Name, VTAM nodes, TCP/IP partners Connection – between a requester and a server - TCP/IP ports, or VTAM LUNAMEs Network protocol – TCP/IP or SNA (VTAM) Conversation – handle traffic on a connection - Also referred to as a session DRDA – one per requester to handle SQL & open cursors Private protocol – may have more, one per open cursor DB2 PRD5 AUSTIN TABLE Application Requester Application Server DB2 PRD1
© Copyright 10/10/2014 BMC Software, Inc 8 Distributed Fundamentals A Tale of Two Protocols Private Protocol (PP) - First delivered in DB Uses 3 part names Select C1,C2,C3 from DB2G.APPL1.TABLE1 - Dynamic SQL only No remote bind - DB2 on z/OS to DB2 on z/OS only - No stored procedure support - Functionally stabilized and on the way out DRDA (Distributed Relational Database Architecture) - Introduced in DB Supports 3-part names and explicit CONNECT statements - Dynamic and Static SQL Remote bind capabilities - Supports stored procedures - Supports all RDBMS implemented using DRDA protocol - Supports SNA and TCP/IP - Is the strategic architecture for distributed Specifying the protocol Default can be set at subsystem level using DBPROTCL DSNZPARM value (DRDA or PRIVATE) With DB2 9.1 PRIVATE can no longer be specified Can also be specified in the DBPROTOCOL parm of the BIND statement (D or P) Only option in DB2 9
© Copyright 10/10/2014 BMC Software, Inc 9 Distributed Fundamentals DB2 on z/OS Distributed Implementation In the beginning - 3 DB2 operational address spaces - Plus all the allied agent address spaces CICS, IMS, TSO Attach TSO Batch, Call Attach Facility Distributed Data Facility (DDF) in DB2 V2R2 - Access using 3 part names or aliases - DIST address space introduced DRDA (Distributed Relational Database Architecture) first implemented in DB2 V2R3 Major enhancements delivered in DB2 V4 - DRDA support of stored procedures - DBAT user priority More in DB2 V5 - TCP/IP, ODBC, CLI, JDBC - Much more... Web-based access comes of age - Java, JDBC Universal Driver, Websphere... DIST Along came distributed Sign-0n More stuff ALLIED AGENTS ALLIED AGENTS ALLIED AGENTS DBM1MSTRIRLM In the beginning ALLIED AGENTS Sign on and such LockingMost of the Action
© Copyright 10/10/2014 BMC Software, Inc 10 Database Access Threads (DBATs) Service distributed workloads Implemented as an MVS WLM enclave running in preemptive SRBs originating in the DIST address space (more coming) DBAT Types - DBAT (Server) - DBAT (Dist) Allied Threads Service local attachment facilities Run at the dispatching priority of the requesting application Can become distributed requesters Allied Agent Types - Allied - Allied Dist (requester) DB2 on z/OS Distributed Implementation A Word About Threads Allied Threads DBM1 IMS/TM DIST Distributed Connections Distributed Traffic Allied Agents CICS Batch Database Access Threads
© Copyright 10/10/2014 BMC Software, Inc 11 DDF Setup Considerations Related ZPARM Review CMTSTAT – DDF Threads IDTHTOIN – Idle Thread Timeout TCPKPALV – TCP/IP Keepalive POOLINAC – Pool Thread Timeout ACCUMACC and ACCUMUID MAXTYPE1 (PP) – Max Inactive DBATs KEEPDYNAMIC(YES) / MAXKEEPD EXTRAREQ / SRV – Extra Blocks REQ / SRV And of course: - MAXDBAT – Max Remote Active - CONDBAT – Max Remote Connected
© Copyright 10/10/2014 BMC Software, Inc 12 DDF Setup Considerations Related ZPARM Review CMTSTAT – DDF Threads IDTHTOIN – Idle Thread Timeout TCPKPALV – TCP/IP Keepalive POOLINAC – Pool Thread Timeout ACCUMACC and ACCUMUID MAXTYPE1 (PP) – Max Inactive DBATs KEEPDYNAMIC(YES) / MAXKEEPD EXTRAREQ / SRV – Extra Blocks REQ / SRV And of course: - MAXDBAT – Max Remote Active - CONDBAT – Max Remote Connected
© Copyright 10/10/2014 BMC Software, Inc 13 DDF ZPARM View
© Copyright 10/10/2014 BMC Software, Inc 14 DBAT Processing Modes Mode is defined with the ZPARM CMTSTAT - Very critical option for DDF that defines the flow of DBAT processing - “DDF Threads” on panel DSNTIPR Two choices: - INACTIVE – highly recommended Provides DBAT pooling for DRDA access More effective WLM classification per UOW Reduced Resource usage - ACTIVE DBAT created for each new client application DBAT held through commits Use this only if the applications require it ZPARM Considerations CMTSTAT INACTIVE / ACTIVE POOLINAC Used / not applicable IDTHTOIN Used / not recommended CONDBAT:MAXDBAT > / =
© Copyright 10/10/2014 BMC Software, Inc 15 DB2 Thread Pooling Driven by CMTSTAT INACTIVE in DSNZPARM As active DBATs are freed they remain in the pool and can be immediately reused Often called Type 2 Inactive Thread support - Use the Thread Pooling terminology as it’s more accurate DB2 Connect ESE and JDBC Type 4 Driver also provide thread pooling functions that can (and often) should be used with DDF Thread Pooling Distributed Connections Benefits Reduces CPU required to create and destroy DBATs Reduced number of DBATs decreases real and virtual storage requirements Better scalability for distributed connections Active DBATs Pooled DBATs Active Connections Inactive Connections
© Copyright 10/10/2014 BMC Software, Inc 16 Processing Diagram CMTSTAT INACTIVE New Connection > CONDBAT? Reject Yes Pooled DBAT Avail? Yes MAXDBAT Reached? No Queue Yes Create DBAT No Reply “ready” to client First SQL / UOW? Process SQL Enclave / Class 1 Reuse DBAT Commit / Rollback? End enclave / Write Acctg Pool DBAT / Inactv. Conn. Resumed Connection YesNo Yes No CHARACTERISTICS No end user “think time” included Enclave is created when the first SQL is received Enclave is deleted at commit/ rollback (thread complete) New enclave for each UOW, reclassified by WLM Use multi-period response time or velocity goals CHARACTERISTICS No end user “think time” included Enclave is created when the first SQL is received Enclave is deleted at commit/ rollback (thread complete) New enclave for each UOW, reclassified by WLM Use multi-period response time or velocity goals
© Copyright 10/10/2014 BMC Software, Inc 17 Processing Diagram CMTSTAT ACTIVE New Connection MAXDBAT Reached? Queue Yes Create DBAT No Enclave / Class 1 Reply “ready” to client Process SQL Commit / Rollback? Yes Term Thread? End enclave / Write Acctg No Term. DBAT & Connection Yes No CHARACTERISTICS End User “think time” is included Enclave is created when DBAT is created Enclave is deleted at thread termination Only one enclave, no reclassification Can only use a single-period velocity goal CHARACTERISTICS End User “think time” is included Enclave is created when DBAT is created Enclave is deleted at thread termination Only one enclave, no reclassification Can only use a single-period velocity goal
© Copyright 10/10/2014 BMC Software, Inc 18 WLM Enclaves WLM Enclaves are independent, dispatchable units-of-work that span multiple address spaces and can include combinations of SRBs and TCBs - Concept of “Business Unit-of-Work” DB2 on z/OS uses enclaves for work coming through DDF - Controlled by WLM - Eligible for zIIP processors Thread priority set by WLM workload classification - Providing good DDF classifications is vital - No DDF workload classification defaults to SYSOTHER which means minimal service
© Copyright 10/10/2014 BMC Software, Inc 19 WLM Workload Classification Identifiers Connection Type ** WLM CT ** - DRDA or Private Protocol Plan name ** WLM PN ** - Always ‘DISTSERV’ for distributed threads Package / Collection ** WLM CN/PK ** First package accessed Stored procedure name ** WLM PR ** If First SQL is a CALL AUTHID of client ** WLM UI ** Often not unique for non-z/OS clients Correlation ID ** WLM CI ** Accounting String ** WLM AI ** DB2 for z/OS server information - Subsystem instance ** WLM SI ** - Subsystem collection name (Data sharing group) ** WLM SSC ** - Sysplex name ** WLM PX **
© Copyright 10/10/2014 BMC Software, Inc 20 More Identifiers from non-z/OS Clients Clients can flow other identifiers to DB2 for z/OS - Values can be set by DDF threads via “Server Connect” and “Set Client” calls ODBC/CLI/VB (SQLSetConnectionAttr) Non-OBDC (sqleseti) JDBC (DB2Connection) DRDA (ACCRDB prddta / sqlstt in EXCSQLSET) Most important IDs supported in V8 with special registers - Workstation Client Accounting - Workstation Userid ** WLM SPM 1-16 ** - Workstation Name ** WLM SPM ** - Workstation Application ** WLM PC 1-32 **
© Copyright 10/10/2014 BMC Software, Inc 21 Setting and Passing Client Identifiers recated)setClientInfo (3.5x.xx) DB2 z Special register setDB2ClientUserClientUser CURRENT CLIENT_USERID setDB2ClientWorkstationClientHostname CURRENT CLIENT_WRKSTNNAME setDB2ClientApplicationInformation ApplicationName CURRENT CLIENT_APPLNAME setDB2ClientAccountingInformationClientAccountingInformation CURRENT CLIENT_ACCTNG java.sql.Connection.setClientInfo - Sets values for client info properties - Previous set methods are deprecated
© Copyright 10/10/2014 BMC Software, Inc 22 Websphere Applications WAS setClientInformation API can be used to pass client registers - API is defined on the WSConnection class - Part of the com.ibm.websphere.rsadapter package - Values that can be passed include WSConnection.CLIENT_ID WSConnection.CLIENT_LOCATION WSConnection.CLIENT_APPLICATION_NAME WSConnection.CLIENT_ACCOUNTING_INFO - Client Information can be reset by calling method with a NULL parameter - Trace Specification WAS.clientinfo=all to pass client information to the backend without tracing of client activities.
© Copyright 10/10/2014 BMC Software, Inc 23 DBATs and Accounting ACTIVE mode - Only cut at thread termination, not at commit INACTIVE mode - DRDA – at “clean” COMMIT or ROLLBACK “Type 2 inactive” - DRDA with KEEPDYNAMIC(YES) At “clean” commit (DB2 V8 and above) - PP DBAT – at commit or termination At commit, if “Type 1 Inactive” (MAXTYPE1) allowed Else only at termination Active thread is idle too long and is canceled - At “Idle Thread Timeout” (IDTHTOIN), if allowed Checked every 2 minutes
© Copyright 10/10/2014 BMC Software, Inc 24 Accounting and DDF Rollup Option in DB2 V8 to reduce accounting volume - Turned on if ZPARM ACCUMACC > 1 - Just going to V8 the ACCUMACC parm defaults to 10 which means it’s on You could see a big drop in SMF 101 records Data accumulated for specified # of threads - For matching IDs, based on ACCUMUID - Combination of the 3 workstation IDs Total of 10 combinations in V8 7 new options in DB2 9 Accounting written when - “Too old” (staleness threshold) - “Too much” (internal storage threshold reached) - “Just enough” (limit threshold reached) One accounting record reflects one or more threads - Currently no DDF statistics (QLAC) or QMDA accounting - Only one “ROLLUP” package Active thread data only shows the current thread counts
© Copyright 10/10/2014 BMC Software, Inc 25 DBAT Thread Status Assigned to a remote client (RA or RX) - Actively processing SQL or waiting for more - Waiting for more work after “clean” commit, if: INACTIVE mode – only: – KEEPDYNAMIC(YES) – all resources & DBAT kept ACTIVE mode – even after commit –All resources & DBAT kept until thread termination Pooled (DA) - DRDA clients only, with INACTIVE mode Freed or newly created DBATs are pooled (also called ‘DBAT slots’) - Available for reuse by any new / resumed request - Still uses resources (esp. DBM1 storage) - Still shown and counted as “active threads” But connection name is “DISCONN” Can be terminated if not used (POOLINAC)
© Copyright 10/10/2014 BMC Software, Inc 26 DISPLAY THREAD Command Output
© Copyright 10/10/2014 BMC Software, Inc 27 Inactive Connections Inactive connections (R2) - Associated with a remote requester - Waiting for more work - Speeds up response to additional SQL Previously called Inactive Type 2 DBATs - Not DBATs at all
© Copyright 10/10/2014 BMC Software, Inc 28 Active DBATs (Data Sharing Members) Hyperlink
© Copyright 10/10/2014 BMC Software, Inc 29 Enclave Views (MAINVIEW for z/OS)
© Copyright 10/10/2014 BMC Software, Inc 30 Analyzing DDF Thread Data The accounting data is the first source Still analyze other application considerations - Elapsed and CPU times, I/O, SQL counts... But in addition: - Elapsed time inside / outside the DB2 server - Number of messages and blocks sent / received Batch reports summarized by - The important DDF identifiers for your workloads
© Copyright 10/10/2014 BMC Software, Inc 31 Thread Accounting – DBAT Specifics Summary Information Distributed Indicators/ Identifiers DBAT Accounting Metrics - “Application” processing is occurring in the workstation client - Class 2 – Class 1 CPU measures time not processing SQL in DB2 Includes parts of thread creation and termination and moving data from and to communication buffers zIIP Metrics - Actual CPU time on the zIIP Not included in “standard” class 1 & 2 CPU times - CPU eligible for the zIIP
© Copyright 10/10/2014 BMC Software, Inc 32 Thread Accounting – DBAT Specifics DDF Summary Client Identifiers Available in DB2 V8 WLM Service Class DBAT Details - Wait on MAX DBATs - PROD-ID = IBM Universal Driver for JDBC and SQLJ - Traffic Information
© Copyright 10/10/2014 BMC Software, Inc 33 Tracing Distributed Workloads Additional focus on one workload - Summary exception trace (accounting) - Detail trace with important event IFCIDs All the usual qualifiers are available For DDF, important to reduce the data: - Filter by requesting location - Filter by Workstation ID(s) In V9, DB2 also allows qualification by these IDs Exception Filters can be used to keep only threads that may need analysis (high In-DB2 elapsed, etc.)
© Copyright 10/10/2014 BMC Software, Inc 34 Detail Traces Detail traces can include selected event groups - Basic thread flow and SQL - Also can choose to add scans, I/O, locks Another group to include specific DDF events - The volume can be high - Use it only when needed - To understand the conversation flow Each event has a pop-up view with the IFCID details
© Copyright 10/10/2014 BMC Software, Inc 35 DDF Statistics The next place to look are the statistics Global statistics - Critical DB2 subsystem tuning information Location statistics - Application impact on DB2 and network DRDA_Remote_Locs (combined) Private Protocol locations (separate) DDF Address Space CPU usage - TCB and SRB
© Copyright 10/10/2014 BMC Software, Inc 36 Global DDF Statistics - STDISTD
© Copyright 10/10/2014 BMC Software, Inc 37 Exception Monitoring Review your current exceptions - Are DDF conditions being monitored? Statistics - DBAT high water mark - Queuing for a DBAT? - DDF still active? - DBM1 storage usage - DDF conversations queued - DDF Excessive Getpages Accounting - Focus on DDF service levels Filter for DBATs / most important work Elapsed time / CPU usage
© Copyright 10/10/2014 BMC Software, Inc 38 Dynamic Cache – STCACHED View
© Copyright 10/10/2014 BMC Software, Inc 39 Extended Reporting Distributed workloads are often volatile - Less insight and control Can be useful to track activity over time - Store and query summary data in DB2 tables When needed, distributed traces and monitoring z/OS reporting on WLM can be helpful - Enclaves – SMF 30 - Workloads by service class – SMF 72 MAINVIEW for z/OS provides online views as well as reports
© Copyright 10/10/2014 BMC Software, Inc 40 Summary SQL is SQL, but: Distributed Workloads are increasing at most DB2 shops DDF processing has a unique set of characteristics that differ from traditional DB2 applications New technologies (for DBAs) like WLM
© Copyright 10/10/2014 BMC Software, Inc 41 Questions?
© Copyright 10/10/2014 BMC Software, Inc 42
Introduction to z/OS Basics © 2006 IBM Corporation Chapter 3: z/OS Overview.
1 Advanced Database Application Development Performance Tuning Performance Benchmarks Standardization E-Commerce Legacy Systems.
The Whirlwind Tour Chapter 1a. 2 Transactions: Where It All Started [Cuneiform] documents now number about half a million, three- quarters of them more.
2011 Gelb Information Systems Corp. Think Faster with Gelb Information Any questions? to: Call: CICS.
Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9 th Edition Chapter 3: Processes.
Database Controls 2012 National State Auditors Association Information Technology Conference September 2012.
© Copyright. This presentation may not be duplicated without a written permission from CSL International Ltd. 1 September 2012 Virtualization Management.
SQL tuning and the Dynamic Statement Cache. What Will We Talk About? Some SQL Tuning Fundamentals Dynamic SQL in More Detail Introduction to DB2 Statement.
1 Process Description and Control Chapter 2. 2 Process A program in execution An instance of a program running on a computer The entity that can be assigned.
Copyright 2004 Bernd Brügge TUM Software Engineering WS TUM System Design II Bernd Brügge Technische Universität München Applied Software Engineering.
©Silberschatz, Korth and Sudarshan8.1Database System Concepts, 5 th Ed, slide version 5.0, August Chapter 8: Application Design and Development.
Silberschatz, Galvin and Gagne ©2010 Operating System Concepts Essentials – 8 th Edition Chapter 16: Windows 7.
2 Agenda l Database Overview l Oracle Audit/Security/Control.
Debbie Sheetz Sr. Staff Consultant BMC Software, Customer Support/Waltham, MA Top Ten Topics in Performance Reporting/Modeling for Virtualization using.
1 Introduction to ASP.NET. 2 Static and Dynamic Web Applications HTML is used to create static content Browser software interprets HTML tags and formats.
HORIZONT 1 XINFO ® The IT Information System HORIZONT Software for Datacenters Garmischer Str. 8 D München Tel ++49(0)89 /
Chapter 4 1 Threads Threads are a subdivision of processes Since there is less information associated with a thread than there is info associated with.
©Silberschatz, Korth and Sudarshan18.1Database System Concepts Chapter 18: Database System Architectures Centralized Systems Client--Server Systems Parallel.
Database System Concepts, 6 th Ed. ©Silberschatz, Korth and Sudarshan See for conditions on re-usewww.db-book.com Chapter 26: Advanced.
1. 2 Layer 2 Switching Switching breaks up large collision domains into smaller ones Collision domain is a network segment with two or more devices sharing.
CSC Proprietary 2/11/2014 3:44:12 AM 008_P2_CSC_white 1 Active Server Pages (ASP)
VMware vCenter Server High Availability Product Support Engineering VMware Confidential.
The Architecture of Transaction Processing Systems Chapter 26.
Understanding I/O Performance with PATROL- Perform and PATROL-Predict Debbie Sheetz Sr. Staff Consultant BMC Software.
1 The Arena modeling system is a flexible and powerful tool that allows analysts to create animated simulation models that accurately represent virtually.
Database System Concepts, 6 th Ed. ©Silberschatz, Korth and Sudarshan See for conditions on re-usewww.db-book.com Chapter 17: Database.
CA IDMS TM SQL Routines: An Overview July 22, 2008 Cal J. Domingue.
© 2016 SlidePlayer.com Inc. All rights reserved.