IMS and WebSphere MQ GSE BENELUX IMS User Group Meeting

IMS and WebSphere MQ GSE BENELUX IMS User Group Meeting
June Antwerp Belgium Steve Nathan -

Disclaimer © Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM, the IBM logo, ibm.com, and IMS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at

AGENDA Introduction WebSphere MQ Messages Using the MQ API
WebSphere MQ IMS Bridge Appendix 1 – Building OTMA User Data for MQ Appendix 2 – WebSphere MQ IMS Bridge Security

IMS Application Access to WebSphere MQ Messages
IMS applications can access WebSphere MQ messages in two ways: 1. The IMS application uses the MQ API to Get and Put messages with syncpoint coordination with IMS IMS BMP MPP IFP (not JMP or JBP until IMS 13) Requires connecting MQ to IMS via ESS Link program with MQ IMS stub (CSQQSTUB) WebSphere MQ messages can be inserted to the IMS Message Queue by an application program (BMP/MPP) Could be a Trigger message (MQ IMS BMP Trigger Monitor) Could be the real Message IMS Batch No ESS interface Syncpoint coordination requires RRS Link program with WebSphere MQ two-phase commit batch stub CSQBRRSI or CSQBRSTB+ATRSCSS

IMS Application Access to WebSphere MQ Messages
IMS applications can access WebSphere MQ messages in two ways: 2. The WebSphere MQ IMS Bridge puts the message on the IMS Message Queue via OTMA The WebSphere MQ IMS Bridge is code in the MQ Queue Manager Does not require connecting WebSphere MQ to IMS via ESS But the ESS connection could also exist for programs using the MQ API Requires OTMA configuration in the MQ CSQZPARM

Connecting WebSphere MQ and IMS via ESS
WebSphere MQ for z/OS attaches to IMS just like DB2 using the external subsystem (ESS) (ESAF) interface OTHER MQ SYSTEMS IMS DBRC IMS DLISAS IMS BMP QUEUE MANAGER MQ CHANNEL INITIATOR CICS TSO BATCH IMS CONTROL REGION IMS MPR ESS Interface RRS

Define WebSphere MQ to IMS by adding ESS information to the IMS PROCLIB (member name IMIDxxxx) FORMAT: SST=,SSN=,LIT=,ESMT=,RTT=,REO=,CRC= SST: Subsystem Type - “MQS” SSN: Subsystem Name - MQ subsystem LIT: Language Interface Token - See CSQQDEFV ESMT: External Subsystem Module Table - “CSQQESMT” RTT: Resource Translation Table - Not Used by MQ REO: Region Error Option - “R”, “Q”, or “A” CRC: Subsystem Recognition Character - Not Used by MQ The /SSR command is not supported

Place the MQ authorized library (HLQ.SCSQAUTH) in the IMS control region and dependent region DFSESL concatenations Copy module CSQQDEFV from HLQ.SCSQASMS to be customized, assembled, and linked into an authorized library in the IMS control region STEPLIB concatenation CSQQDEFV CSECT CSQQDEFX NAME=CSQ1,LIT=LIT1,TYPE=DEFAULT CSQQDEFX NAME=CSQ3,LIT=LIT2 CSQQDEFX TYPE=END

Subsystem Connection /DIS SUBSYS ALL SUBSYS CRC REGID PROGRAM LTERM STATUS CSQ3 ! CONN CSQ1 < CONN DB2R = CONN CONN CONN

Using the WebSphere MQ API with IMS Applications
WebSphere MQ application stubs An application program must be linked with a “stub” module in order to use the MQ API There are three possible “stubs” that can be used in IMS applications CSQQSTUB IMS stub WebSphere MQ knows the application is running in an IMS environment Provides two-phase commit for IMS and MQ API calls Not for IMS Batch (DLI/DBB) CSQBSTUB Batch stub WebSphere MQ does not know the application is running in an IMS environment There is no two-phase commit with IMS Can be used for online processing – more later

WebSphere MQ application stubs An application program must be linked with a “stub” module in order to use the MQ API There are three possible “stubs” that can be used in IMS applications CSQBRSTB Batch two-phase commit stub Can only be used in IMS batch jobs Requires RRS

Calls to MQ, IMS and DB2 can be made within the same unit of work (UOW) MQ API calls IMS IOPCB calls IMS ALTPCB calls IMS database calls DB2 calls

IMS and MQ Units of Work An IMS commit is also an MQ and DB2 commit SYNC, CHKP, GU to IOPCB (MODE=SNGL), normal program termination An IMS backout (ROLB) is also an MQ and DB2 backout Any IMS abend is also an MQ and DB2 backout ROLL, miscellaneous abends

At normal syncpoint.... IMS input message is dequeued IMS NON-EXPRESS output messages are sent IMS EXPRESS output messages have already been sent IMS database updates are committed DB2 updates are committed

At normal syncpoint.... MQ input messages marked with SYNCPOINT, or MARK_SKIP BACKOUT are dequeued MQ input messages marked with NO_SYNCPOINT have already been dequeued MQ output messages marked with SYNCPOINT are sent MQ output messages marked with NO_SYNCPOINT have already been sent If the IMS application is message driven (BMP or MPP) the MQ connection handle is closed for security reasons Connection security is by Userid Each message can be from a different Userid

At abnormal termination or ROLx.... IMS input message is dequeued IMS has Non-Discardable Message Exit IMS NON-EXPRESS messages are discarded IMS EXPRESS output messages have already been sent IMS database updates are backed out DB2 updates are backed out

At abnormal termination or ROLx.... MQ input messages marked with SYNCPOINT are re-queued MQ input messages marked with NO_SYNCPOINT have already been dequeued MQ input messages marked with MARK_SKIP_BACKOUT are not backed out They are passed to a new UOW If the new UOW abends for any reason the message will be re-queued MQ output messages marked with SYNCPOINT are discarded MQ NO_SYNCPOINT output messages have already been sent

Getting the default queue manager name is not straightforward... MQCONN using default name (blank) MQOPEN the Queue Manager MQOD Objectype = MQOD_Q_MGR MQOD Objectname = blanks MQOO_INQUIRE MQINQ for object name

STROBE shows MQ CPU in detail by Module/Section Note the expense of MQCONN #PUP ** PROGRAM USAGE BY PROCEDURE ** .SYSTEM SYSTEM SERVICES MQSRIES MVS/ESA MQSERIES MODULE SECTION FUNCTION % CPU TIME MARGIN OF ERROR 6.86% NAME NAME SOLO TOTAL CSQILPLM MQ DATA MGR SERVICE RTN ** CSQLLPLM MQ LOCK MGR SERVICE RTN *** CSQMLPLM MQ MSG MGR SERVICE RTN *** CSQPLPLM MQ BUFFR MGR SERVICE RT * CSQQCONN CSQQCONN MQSERIES IMS ADAPTER ****************** CSQQDISC MQSERIES IMS ADAPTER *** CSQQNORM MQSERIES IMS ADAPTER * CSQSLD MQ STG MGR GLBL MOD EP * CSQWVCOL MQ IFC RECORD COLLECTIO *** SECTION .MQSRIES TOTALS:

In a message driven environment MQ forces a Close/Disconnect and Connect for each message – not each schedule That is because MQCONN authority is by Userid and each message can be from a different user MQCONN and MQDISC are very expensive and do a lot of I/O to STEPLIB Preloading all of the CSQQxxxx modules in the MQ authorized library eliminated the overhead and STEPLIB access This is an absolute MUST if your MPP transactions issue MQ API calls It is also required for message-driven BMPs Another customer reported that preloading CSQACLST, CSQAMLST, and CSQAVICM to do data conversion was helpful

In a message driven environment MQ forces a Close/Disconnect and Connect for each message – not each schedule This can cause problems in a WFI/PWFI environment with Triggered Queues If there are no more messages on the IMS queue and the IMS application does a GU to the IOPCB IMS does not notify MQ for TERM THREAD until the next message arrives or a QC is returned to the IMS application During that time the MQ Queue may still be open MQ internally closes all open queues when it receives TERM THREAD If there are triggered FIRST queues open new messages arriving in MQ will not generate trigger messages because the queue is open To avoid this problem the IMS application should explicitly MQCLOSE any triggered FIRST queues before issuing the next GU to the IOPCB

There have been reports of IMS application programs ABENDing with 0C1 when issuing MQ API calls The main program is an IMS program (ENTRY DLITCBL) It dynamically calls a sub-program which ONLY issues MQ API calls There were no IMS calls The sub-program was NOT linked with the IMS language interface DFSLI000 This resulted the ABEND0C1 The sub-program must also be linked with DFSLI000 because the MQ API calls are going through the IMS ESS interface

In a message driven environment MQ forces a Disconnect and Connect for each message – not each schedule There is an alternative if your application does not require syncpoint coordination for MQ calls and IMS You can link the application with the MQ batch stub – CSQBSTUB Then a Wait-for-input program can Connect once in the beginning and Disconnect once at the end (but remember previous foil) It can Open queues once in the beginning and Close them once at the end It can issue MQGETs and MQPUTs during IMS transactions It will have to issue MQCMIT calls for any work done “In Syncpoint” from an MQ perspective The first MQCONN in an address space will determine which interface will be used so CSQQSTUB and CSQBSTUB transactions must run in different IMS Message Regions This MPR must also have an SSM member excluding MQ CSQBDEFV can be used to define a default Queue Manager

There are several ways the MQ API can be used to have IMS programs interact with MQ queues WebSphere MQ IMS Trigger Monitor Customer MQ IMS Trigger Monitor Customer MQ IMS Queue Monitor Customer MQ IMS Queue Processor

WebSphere MQ IMS Trigger Monitor
The WebSphere MQ IMS Trigger Monitor is an IBM supplied non-Message Driven BMP job which reads “trigger” messages from an MQ Initiation Queue and inserts them to the IMS Message Queue The IMS application retrieves the trigger message with a GU to the IOPCB The trigger message contains the Queue Manager and Queue Name where the real message resides The IMS application then uses the MQAPI to retrieve the real message The reply message would be done via MQPUT or ISRT to an ALTPCB The reply can not be made to the IOPCB because the message came from a non-message driven BMP

These are the steps for the MQ IMS Trigger Monitor The MQ IMS Trigger Monitor BMP (CSQQTRMN) is started MQCONN to the MQ Queue Manager MQOPEN the Initiation Queue MQGET with Wait on the Initiation Queue An MQ application MQPUT’s a message to the triggered queue MQ generates a trigger message and puts it on the initiation queue MQ IMS Trigger Monitor BMP receives the trigger message

These are the steps for the MQ IMS Trigger Monitor (continued) The MQ IMS Trigger Monitor BMP does CHNG/ISRT/PURG of the trigger message to the IMS Queue The MQ IMS Trigger Monitor BMP issues a SYNC call IMS logs the trigger message IMS puts the message in the IMS Message Queue IMS enqueues the trigger message to the IMS transaction The IMS transaction is scheduled in an MPR The IMS transaction does GU to the IOPCB and retrieves the trigger message

These are the steps for the MQ IMS Trigger Monitor (continued) The IMS Transaction does MQCONN for the Queue Manager The IMS Transaction does MQOPEN for the Input Queue The IMS Transaction does MQGET for the real MQ message The IMS Transaction processes the message including IMS and ESAF calls The IMS Transaction does MQPUT1 for the MQ Reply message The IMS Transaction does MQCLOSE for the MQ Input Queue The IMS Transaction does MQDISC to the Queue Manager The IMS Transaction does GU to the IOPCB to create an IMS syncpoint

The MQ IMS Trigger Monitor reads the MQ Trigger Message with NO_SYNCPOINT The Trigger Message is deleted immediately If the BMP ABENDs before its SYNC call or IMS ABENDs before the message gets to the IMS message queue the Trigger Message is gone but the real message is still on the MQ queue If the triggering option was FIRST and this was the last message on the queue there will be no more Trigger Messages and the real message will not be retrieved until the TriggerInterval is reached If the triggering option is EVERY there will not be another trigger message until the next message arrives on the real queue The real message will not be processed until a new trigger message wakes up the MQ IMS Trigger Monitor

IMS application coding consideration The IMS application must only process ONE real MQ message per GU to the IOPCB to retrieve a trigger message Consider this flow GU IOPCB and get trigger message MQCONN MQOPEN MQGET real message Process including IMS and DB2 updates MQPUT1 the reply Go To MQGET until no more messages What could go wrong???

IMS application coding consideration What could go wrong? There were no IMS syncpoints in this loop MQCMIT is ignored if using the CSQQSTUB MQCMIT will not commit IMS or DB2 resources You can not issue an IMS CHKP or SYNC call in an MPP If there is an ABEND multiple MQ messages worth of updates may be backed out If MQGET in SYNCPOINT all of the MQ messages are re-queued If MQGET NO SYNCPOINT they have all been freed While you are looping processing the messages all of the IMS and DB2 locks for all of the messages processed are still being held and all of the data buffers are still in use If triggering was EVERY there are trigger messages for which there are no real messages This will result in “false schedules” of IMS transactions

What about triggering If triggering is FIRST and the IMS transaction is processing the real queue and more real messages arrive there will be no more trigger messages But when the real queue is closed – explicitly or implicitly – and there are messages on the real queue then a trigger message will be generated If triggering is EVERY there will be a trigger message for every real message even if the IMS application has the queue open In a Shared MQ Queue environment may have MQ IMS Trigger Monitors on multiple MQ Queue Managers each waiting on the same Shared Initiation Queue MQ will generate a Trigger Message for EACH MQ Queue Manager that has an MQ IMS Trigger Monitor waiting One IMS application will get the real message One IMS application will have a “false schedule” Please read this:

Advantages It is provided by IBM Only the small trigger message is logged in IMS Only the small trigger message is in the IMS message queue One customer reported that 90% of their 2.8 millions transactions per day come in through their 4 MQ IMS Trigger Monitors Disadvantages A Trigger Monitor BMP can only wait on one Initiation Queue But one Initiation Queue can be used for multiple Real queues There are many steps for each message WebSphere MQ Triggering There are many considerations

Customer IMS Trigger Monitor
It is possible to write a Customer IMS Trigger Monitor This monitor could be written in assembler and wait on multiple Initiation Queues at the same time The one advantage is that it can wait on multiple queues It has all the disadvantages of the IBM MQ IMS Trigger Monitor It also has the disadvantage of being very difficult to write I did write one and it took over a year to program for all of the idiosyncrasies of waiting on multiple ECBs I mention it here so that you will not do it

Customer IMS Queue Monitor
It is possible to write a Customer IMS Queue Monitor which reads “real” messages from an MQ Queue and inserts them to the IMS Message Queue The IMS application retrieves the real message with a GU to the IOPCB The reply message would be done via MQPUT or ISRT to an ALTPCB The reply can not be made to the IOPCB because the message came from a non-message driven BMP

These are the steps for the Customer IMS Queue Monitor The Customer IMS Queue Monitor BMP is started MQCONN to the MQ Queue Manager MQOPEN the Real Queue MQGET with Wait on the Real Queue The wait time is short enough to avoid ABENDS522 An MQ application MQPUT’s a message to the Real Queue Customer IMS Queue Monitor BMP receives the Real message

These are the steps for the Customer IMS Queue Monitor (continued) The Customer IMS Queue Monitor BMP does CHNG/ISRT/PURG of the Real message to the IMS Queue May be a multi-segment message The Customer IMS Queue Monitor BMP issues a SYNC call IMS logs the Real message IMS puts the Real message in the IMS Message Queue IMS enqueues the Real message to the IMS transaction The IMS transaction is scheduled in an MPR The IMS transaction does GU to the IOPCB and retrieves the Real message

These are the steps for the Customer IMS Queue Monitor (continued) The IMS Transaction processes the message including IMS and ESAF calls The IMS Transaction does MQCONN for the reply message Queue Manager The IMS Transaction does MQPUT1 for the MQ Reply message The IMS Transaction does MQDISC The IMS Transaction does GU to the IOPCB to create an IMS syncpoint

The Customer IMS Queue Monitor can read the MQ Real Message In SYNCPOINT The Real Message is not deleted until the IMS SYNC call If the BMP ABENDs before its SYNC call or IMS ABENDs before the message gets to the IMS message queue the MQ message is re-queued The number of times this happens will be shown in MQMD_BackOutCount

The Customer IMS Queue Monitor may have to pass the Reply-to Queue and Reply-to Queue Manager information to the IMS transaction This can be done by inserting an extra IMS message segment Could pass just the Reply-to information Could pass the entire MQMD

Advantages Less overhead in the IMS MPR No MQ Triggering complications and overhead Disadvantages The Customer IMS Queue Monitor can only wait on one Real Queue But there can be multiple BMP’s reading the same queue The Real MQ message is logged in IMS This could be VERY large The Real MQ message goes in the IMS message queue

Customer IMS Queue Processor
It is possible to write a Customer IMS Queue Processor which reads “real” messages from an MQ Queue and does all of the processing within the BMP itself There is no message switching to an IMS transaction The reply message would be done via MQPUT or ISRT to an ALTPCB This is the most efficient way for IMS applications to process MQ messages using the MQ API

These are the steps for the Customer IMS Queue Processor The Customer IMS Queue Processor BMP is started MQCONN to the MQ Queue Manager MQOPEN the Real Queue MQGET with Wait on the Real Queue The wait time is short enough to avoid ABENDS522 An MQ application MQPUT’s a message to the Real Queue Customer IMS Queue Processor BMP receives the Real message

These are the steps for the Customer IMS Queue Processor (continued) The Customer IMS Queue Processor processes the message including IMS and ESAF calls It may have to call different subroutines for different “transaction codes” The Customer IMS Queue Processor does MQPUT1 for the MQ Reply message The Customer IMS Queue Processor does an IMS SYNC call The Customer IMS Queue Processor loops to do another MQGET with Wait

The Customer IMS Queue Processor can read the MQ Real Message In SYNCPOINT The Real Message is not deleted until the IMS SYNC call If the BMP ABENDs before its SYNC call or IMS ABENDs before the message gets to the IMS message queue the MQ message is re-queued The number of times this happens will be shown in MQMD_BackOutCount

The Customer IMS Queue Processor does not have to pass the Reply-to Queue and Reply-to Queue Manager information to the IMS transaction The input MQMD is available

Advantages No IMS MPR overhead No IMS logging of the MQ messages No IMS message on the IMS Queue No MQ Triggering complications and overhead Disadvantages The Customer IMS Queue Processor can only wait on one Real Queue But there can be multiple BMP’s reading the same queue

WebSphere MQ IMS Bridge
This is code in the MQ Queue Manager The IMS Bridge is an OTMA client For specially defined queues it will MQGET the messages from the queue and send them to IMS using the IMS OTMA interface The IMS bridge also gets output messages from IMS via the OTMA interface IOPCB output The output message is MQPUT to the Reply-to Queue and Reply-to Queue Manager in the original MQ input message MQMD passed and returned in the OTMA Prefix User Data ALTPCB output The output message is MQPUT to the Reply-to Queue and Reply-to Queue Manager in the MQMD created by the OTMA DRU exit and returned in the OTMA Prefix User Data

SERVER z/OS,WIN, AIX, SUN, … SNA z/OS TCP/IP ICON OTMA IMS VTAM IMS BRIDGE MQ QM LU1 LU2 LU 6.1 LU 6.2 XCF Any App RYO Client TN3270 Websphere ITRA PC End User Application SCI OM IMS Control Center

One MQ queue manager can connect to multiple IMS control regions One IMS control region can connect to multiple MQ queue managers MQ and all of the IMS Control Regions it connects to must be in the same XCF group MQ and IMS can be on different LPARs in the same Sysplex WebSphere MQ IMS Bridge start and stop events are sent to the SYSTEM.ADMIN.CHANNEL.EVENT.QUEUE

When the message arrives in MQ it will be sent via XCF to the IMS OTMA interface Message may be: an IMS transaction an IMS command (only a subset of commands are allowed) NOT a message to an IMS LTERM IMS will put it on the IMS message queue The application will do a GU to the IOPCB to retrieve the message This is very similar to the implicit LU6.2 process There are no changes to existing IMS programs ALTPCB output may have to be routed by OTMA exits or OTMA Descriptors A remote queue manager can send a message to a local queue destined for IMS via OTMA

1. Message arrives on local queue 4. MQGET from local queue - or - transmission to remote queue 3b. IMS output returned to reply-to-queue-manager and reply-to-queue via OTMA IMS Transactions 2. Message sent via OTMA to IMS message queue and IMS transaction started MQSeries Application (local or remote) IBM Mainframe Queue Manager Inbound local queue (stgclass=IMS) Local or Transmit IMS LTERM 3c. IMS output routed to other LTERM via OTMA exit XCF 3a. Message retrieved by transaction via GU IOPCB IMS Bridge

Define MQ to OTMA in CSQZPARM OTMACON keyword on CSQ6SYSP macro OTMACON(Group,Member,Druexit,Age,TPIPEPrefix) Group = XCF group Member = MQ XCF member (OTMA TMEMBER) Druexit = IMS exit to format OTMA User Data (overrides DFSYDTx) Consider a name of DRU0xxxx (xxxx = MQ Queue Manager name) Age = how long a Userid (ACEE) from MQ is valid in the OTMA cache before it expires TPIPEPrefix = three character prefix for TPIPE name To avoid collision with IMS transaction code names Two characters for MQ shared queues Member CSQ4ZPRM in data set hlq.SCSQPROC has default CSQZPARM members you can use to build your members My strong requirement is that all of these should be able to be specified (and used!!!) on the STGCLASS definition

Define one or more storage classes with the XCFGNAME and XCFMNAME parameters of the IMS systems to which you will connect DEFINE STGCLASS(IMSA) - PSID(02) – XCFGNAME(XCFGROUP) - XCFNAME(XCFIMSA) – PASSTKTA(applname) (6.0) The XCFGNAME will not be used The one in the ZPARM will be used When a STGCLASS is defined for a new IMS MQ will attempt to establish the connection

Changing the Storage Class of an IMS Bridge Queue must be done carefully If there are CM0 messages on the IMS Bridge Queue in MQ Do not ALTER the Queue to a different Storage Class pointing to a different IMS copy It will cause sequence number errors (in both IMS copies) Have two different IMS Bridge Queues each with a different Storage Class for the different IMS Copies You can define an Alias Queue which you can point to one IMS Bridge Queue or the other so that the MQ application only has to MQPUT to one queue name If there are no CM0 messages on the IMS Bridge Queue you can ALTER it to point to a different IMS Storage Class This requires WebSphere MQ APAR PK56503

Define queues Define local queue(s) referencing the storage classes DEFINE QLOCAL(MQID_TO_IMSA) – STGCLASS(IMSA) Define reply-to queue(s) DEFINE QLOCAL(MQID_FROM_IMSA) These could also be remote queues

Operating the WebSphere MQ IMS Bridge After startup MQ will join the XCF group defined in the OTMACON parameter The IMS Bridge will initiate a client bid resync to each active IMS defined in the STGCLASS macros When the bid is successful the IMS Bridge will open the Bridge Queues and messages will flow If a new STGCLASS for IMS is added MQ will attempt to connect to the IMS

Operating the WebSphere MQ IMS Bridge If you GET DISABLE an IMS Bridge Queue it will stop messages from MQ to IMS for just that queue New commands were added with MQSeries V6.0 SUSPEND QMGR BRIDGE(IMS) Stops MQGETs by the IMS Bridge from all IMS Bridge Queues Allows MQPUTs to the reply-to queue by the IMS Bridge of replies from IMS RESUME QMGR BRIDGE(IMS) Starts MQGETs by the IMS Bridge from IMS Bridge Queues

Operating the WebSphere MQ IMS Bridge There are IMS commands that affect the WebSphere MQ IMS Bridge /STOP OTMA Closes the connection to all OTMA clients /START OTMA Opens the connection to all OTMA clients Any clients already in the XCF group (e.g. MQ) are notified /STOP TMEMBER xxxx TPIPE yyyy OTMA sends a message to the OTMA client to stop input for the TPIPE OTMA suspends sending output to the TPIPE /START TMEMBER xxxx TPIPE yyyy OTMA sends a message to the OTMA client to start input for the TPIPE OTMA resumes sending output to the TPIPE There are no /STOP TMEMBER or /START TMEMBER commands

MQ creates two TPIPE’s per local queue defined as using the IMS Bridge One is for “asynchronous” messages Commit mode 0 - commit-then-send Output is “asynchronous” to transaction completion This is a SYNChronized TPIPE Messages sent with valid sequence numbers are recoverable after subsystem failures The TPIPE name for non-shared MQ queues is xxx0nnnn xxx = User defined prefix The TPIPE name if using MQ shared queues is xx0nnnnn xx = User defined prefix

MQ creates two TPIPE control blocks per local queue defined as using the IMS Bridge One is for “synchronous” messages Commit mode 1 - send-then-commit Output is “synchronous” with transaction completion This is a non-SYNChronized TPIPE The TPIPE name for non-shared queues is xxx8nnnn xxx = User defined prefix The TPIPE name for shared queues is xx8nnnnn xx = User defined prefix Required for EMH, Conversational, and IMS commands The TPIPE’s are created when the first message for the type (sync, async) arrives on the IMS Bridge queue

DISPLAY QL(name) TPIPE MQ has a TPIPE keyword to the DISPLAY QL command Display the TPIPE names for local queues ALL local queues have TPIPE names assigned – not just IMS Bridge Queues – their Storage Class may be altered later to IMS !MQ37DIS QL(SYSTEM.DEFAULT.LO*) TPIPE CSQM293I !MQ37 CSQMDRTC 1 QLOCAL FOUND MATCHING REQUEST CSQM201I !MQ37 CSQMDRTC DIS QLOCAL DETAILS QUEUE(SYSTEM.DEFAULT.LOCAL.QUEUE) TYPE(QLOCAL) QSGDISP(QMGR) TPIPE( CSQ00000 CSQ80000 ) END QLOCAL DETAILS CSQ9022I !MQ37 CSQMDRTC ' DIS QLOCAL' NORMAL COMPLETION

There is a limit to the capacity of an IMS TPIPE There are several factors involved SYNChronized versus non-SYNChronized (CM0 versus CM1) MQ Persistent versus non-Persistent IMS RECOVER versus NONRECOVER Message size More capacity will require more TPIPEs which will require more MQ queues There are only 2 TPIPES per queue The application has to round-robin the messages to the queues

WebSphere MQ IMS Bridge – Input Messages
The data stream passed to MQ by the application MQPUT is in LLZZTrancodeDataLLZZData... format This allows for multi-segment input messages via OTMA to IMS The IMS Bridge will create IMS segments for each LLZZdata The MQMD.Format of MQFMT_IMS_VAR_STRING (“MQIMSVS”) or MQFMT_IMS (“MQIMS”) tells MQ that the data contains LLZZ’s

The sending application can optionally provide an IMS sub-header (MQIIH) Specify the presence of the MQIIH sub-header by using the MQMD.Format=MQFMT_IMS (“MQIMS”) WARNING: MQIIH must be fullword aligned An output MQIIH will be returned with the output message This header specifies IMS and MQ parameters See the next two foils The format of this input message is: MQIIHLLZZTrancodeDataLLZZData...

MQIIH parameters are: Several reserved fields... IMS LTERM name to put in the IOPCB MFS Format name to put in the IOPCB No MFS formatting is actually done Reply-To Format (MQ format name) Authenticator (RACF password or PassTicket) Transaction Instance ID (if IMS conversational) Transaction State (conversational or not, architected command) Commit Mode (CM0 or CM1) Security Scope (Check or Full) Only honored if /SEC OTMA PROFILE used Flags Pass Expiration Reply Format None

If no MQIIH is presented to the IMS Bridge (MQMD.Format=MQFMT_IMS_VAR_STRING) (“MQIMSVS”) default values are assumed: LTERM=TPIPEName MODNAME=MQMD.Format Default is “MQIMSVS” MQMD.Format is used as the MFSMapName Non-conversational Commit-then-send (commit mode 0) Security mode is MQISS_CHECK All flags are off

MQ allows the addition of one or more standard MQ User Headers to be passed with messages going to an IMS Bridge Queue The user header(s) will be passed to IMS in the OTMA User Data IMS will log this data and pass it back in the OTMA User Data for the reply from the IOPCB ALTPCB output will probably NOT have this data Only if the ALTPCB output was from a transaction initiated from MQ or was built by the DRU exit The user header(s) will be passed back to the MQ application in the message on the Reply-to queue The format of this input message is: HDR1…HDRnMQIIHLLZZTrancodeDataLLZZData...

Message Delivery Options Confirm On Arrival (COA) is provided when the message reaches the IMS queue IMS ACKs the input message to MQ Confirm On Delivery (COD) is not available MQ does not know when the IMS application retrieves the message from the IMS Message Queue The message is rejected if this option is specified There are user requirements to change this

Message Delivery Options Expiry A message can expire in MQ on the IMS Bridge Queue before being sent to IMS The MQ application that MQPUT the message is notified if one of the following MQMD_REPORT options is set MQRO_EXCEPTION (Just the Expiration report) MQRO_EXCEPTION_WITH_DATA (First 100 bytes of the message) MQRO_EXCEPTION_WITH_FULL_DATA (All of the message)

Message Delivery Options Expiry MQ supports IMS Transaction Expiration MQ passes the remaining Expiry time to IMS as an IMS Transaction Expiration time This is rounded up to whole seconds This requires OR’ing the MQ Service Parameter with x’ ’ to activate this feature ZPARM CSQ6SYSP SERVICE= any other bits being used COMMAND SYSTEM SERVICE( ) + any other bits being used

Message Delivery Options Expiry MQ supports IMS Transaction Expiration If the transaction expires in OTMA before being placed on the IMS message queue it is NAK’ed by IMS (NACK_FOR TRANS_EXPIRED, x’0034’) MQ treats this as if the message had expired before being sent to IMS MQMD_REPORT options are honored

Message Delivery Options Expiry MQ supports IMS Transaction Expiration If the transaction expires at the GU to the IOPCB IMS returns message DFS3688I to MQSeries The DFS3688I message is returned to the Reply_To Queue The MQMD_REPORT options are NOT honored

Message Delivery Options Expiry MQ enhances the support for IMS Transaction Expiration Requires IMS APAR PM05984 If the transaction expires at the GU to the IOPCB OTMA returns the original input message to MQ instead of the DFS3688I message The MQMD_REPORT options are honored This enhanced functionality was retrofit to MQ via APAR PM47795

Message Delivery Options Expiry The Reply message can also Expire MQIIH_FLAGS has value MQIIH_PASS_EXPIRATION MQ will pass the REMAINING expiry time in the OTMA header The reply on the reply-to queue will start the Expiry process with that remaining time Any time in IMS is NOT counted

OTMA Flood Conditions MQ supports (rather than tolerates) OTMA flood conditions If a flood warning condition is detected the IMS Bridge will slow down sending messages to OTMA The messages remain in MQ rather than flooding IMS and causing a virtual storage shortage If this is an MQ Shared Queues environment another Queue Manager may be able to send the message to a different IMS

Messages sizes The maximum OTMA input segment size is 32,767 LLZZ + 32,763 bytes of data IMS will create multiple records in the Large Message Queue if necessary The maximum OTMA total message length which can be put to the IMS Bridge Queue is the MQ maximum message length for that queue This is usually 4MB MQ long message support increases this to 100MB

There are special requirements for Commit Mode 0 input messages to IMS If the IMS transaction is defined as RECOVER then the MQ message can be persistent or non-persistent If the IMS transaction is NORECOV then the MQ message must be non-persistent If it is persistent IMS will reject the message with sense code

Commit Mode 0 input messages (continued) How does IMS know whether a message is persistent or non-persistent? All CM0 messages arrive on SYNChronized TPIPES There are no flags in the OTMA headers for this The answer is that IMS and MQ use a little known but documented OTMA interface If the message is persistent it is sent on the SYNChronized TPIPE with a valid sequence number If the message is non-persistent it is sent on the SYNChronized TPIPE with the sequence number set to zero

If the input message cannot be put to the IMS queue because: Invalid message - input message goes on DLQ and warning sent to system console IMS rejected message (sense 001A e.g.Transaction Stopped) - DFS message from IMS is put into MQ reply message and put on reply-to queue If reply message cannot be PUT, it is placed on the DLQ Input message goes to DLQ IMS rejected message (message error) - input message goes on DLQ and warning sent to system console IMS rejected message (other) - messages go back to their original queue, the IMS Bridge is stopped, and warning sent to system console

If any return messages cannot be PUT to the DLQ, messages go back to their original queue If it was a queue problem the queue is stopped If it was an IMS Bridge problem the IMS Bridge is stopped

WebSphere MQ IMS Bridge - Output Messages
IMS Output Messages All IMS message segments are assembled into a single MQ message which must not exceed the MAXMSGLEN parameter of the queue manager If the PUT to the reply-to queue fails, the message goes on the DLQ If the PUT to the DLQ fails, a NAK is sent to IMS and the message remains in IMS When the problem has been corrected use ALTER QL(reply-to queue) GET ENABLE to restart the flow of messages

WebSphere MQ IMS Bridge - Output Messages
IMS Output Messages IOPCB Output reply messages are placed on the MQ reply-to queue specified in the original message header This could be a local queue or a remote queue If you do nothing ALTPCB output from an IMS Bridge initiated transaction will also be placed on the MQ reply-to queue specified in the original message header This can be overridden by the OTMA routing exits or OTMA Descriptors

WebSphere MQ IMS Bridge – ALTPCB Output
IMS ALTPCB output In order to send ALTPCB output to MQ over the IMS Bridge you must build (or override) the OTMA user data in the OTMA DRU exit or use OTMA descriptors in IMS 13 If the message did not originate from the WebSphere MQ IMS Bridge you must build the entire OTMA user data If the message did originate from the WebSphere MQ IMS Bridge the OTMA user data is set to return the message to the MQ reply-to queue specified in the original message header when the exit is invoked This can be changed by updating the OTMA user data

The format is of the OTMA User Data going to the WebSphere MQ IMS Bridge is: LL | | MQMD | | Reply-to Format The LL must be exactly 2 (LL) + L’MQMD + 8 (Reply-to Format) or MQ will ignore the entire MQMD and use default values You must code DCLVER=SPECIFIED,VERSION=1 on the CMQMDA macro May also contain MQSeries User Headers The LL would be variable in this case The User Headers are placed at the end of the OTMA User Data

The WebSphere MQ sample exits for DFSYPRX0 and DFSYDRU0 in Appendix B of the Websphere MQ for z/OS System Setup Guide are very useful Follow this code VERY carefully

Questions?

Building OTMA User Data for MQ
Appendix 1 Building OTMA User Data for MQ

What you are building in the OTMA User Data is actually an INPUT MQMD as if the message had come from MQ originally You must get this mindset The Reply-to Queue and Reply-to Queue Manager in the MQMD become the message destination There is no official way to provide a RTQ and RTQM to the receiving MQ application Try using MQMD_APPLIDENTITYDATA Consider using an MQ User Header

The MSGID and CORRELID are the INPUT MSGID and CORRELID - not necessarily what will be presented to the MQ application You must use the MQMD_REPORT field to specify that the input MSGID and CORRELID be moved to the corresponding output MQMD The default is that the input MSGID is moved to the output CORRELID and a new MSGID is created

Setting the format name for MQ to use for asynchronous OTMA output messages is not straightforward If you are sending back LLZZdataLLZZData as is probably the case you need to set the format name to MQFMT_IMS_VAR_STRING (“MQIMSVS”) The “Reply-to Format” name in the OTMA User Data is the obvious choice But - the “Reply-to Format” in the OTMA User Data is ignored unless the original message was MQPUT with an MQIIH IMS prefix and the Reply-to Format in the MQIIH was valued

But - there was no input message – how do you tell MQ there was an MQIIH Remember you are building an INPUT MQMD in the OTMA User Data Lie – set the MQMD_FORMAT to MQFMT_IMS (“MQIMS”) This tells MQ the “input message” had an MQIIH Then set the Reply-to Format name in the OTMA User Data to “MQIMSVS” AND THIS DOES NOT WORK EITHER!!!!! MQ will create an MQIIH in front of the output message Your application is probably not expecting this The MQMD_FORMAT field will remain MQ_IMS (“MQIMS”) The Reply-to Format (“MQIMSVS”) will be in the MQIIH

If the Reply-to Format is not valid as previously described or is blank then field TMAMHMAP in the OTMA State Prefix is used as the MQ Format Name This is set by the MODNAME parameter in the IMS ISRT call, if present CALL “CBLTDLI” USING ISRT ALTPCB MSGIO MODNAME Do you trust your application to do this properly?

The TMAMHMAP field can be overridden in the OTMA Input/Output Edit Routine (DFSYIOE0) This lets your application be output independent It can set a valid MODNAME in case the message is going to a terminal The DFSYIOE0 exit can override it to “MQIMSVS” If TMAMHMAP is blanks or nulls or MQIMSNON the MQSeries Format name is set to MQFMT_NONE (blanks) If output is LLZZdata this will cause problems

Determining the MQ Persistence for ALTPCB output messages from OTMA to MQ is affected by commit mode It is also different if the ALTPCB output is going to a TPIPE originally created by MQ This can happen if the CHNG call destination or the DFSYPRX0 exit or the OTMA DRU exit set the TPIPE name to an MQ created TPIPE name The MQMD_PERSISTENCE field in the OTMA user data is NOT used in determining asynchronous output message persistence – it is ignored

Persistence Determination If the output is on a TPIPE created by MQ If the commit mode is 1 (send-then-commit) then the output message is persistent If the commit mode is 0 (commit-then-send) then persistence is determined by message recoverability If the message is “recoverable” then the output message is persistent If the message is not “recoverable” then the output message is nonpersistent A message is “recoverable” to MQ if it is sent on a SYNChronized TPIPE AND has a valid sequence number (TMAMCRSQ not equal 0) Not IMS RECOVERable

Persistence Determination – Phase 1 If the output is on a TPIPE not created by MQ If the commit mode is 1 or flag TMAMCASY is set in the OTMA State Header then the output message is persistent If the commit mode is 0 or the TMAMCASY flag is not set then persistence is determined by message recoverability If the message is “recoverable” then the output message is persistent If the message is not “recoverable” then the output message is nonpersistent TMAMCASY indicates “asynchronous/unsolicited queued messages” It may be set on for some IMS DFS messages These messages must be treated as CM0 even if the input was CM1

Persistence Determination The commit mode is 1 and TMAMCASY is not set The message is persistent

Persistence Determination There are two ways to make an ALTPCB TPIPE SYNChronized when it is created The OTMASP parameter in DFSPBxxx can be used to default newly created TPIPEs to SYNChronized This parameter was created for MQ users who wanted persistent output messages but did not want to write OTMA DRU exits

WebSphere MQ IMS Bridge Security
Appendix 2 WebSphere MQ IMS Bridge Security

Requires RACF (or equivalent) Three levels of security 1. Connections from MQ to IMS (during client bid) Uses the RACF Facility class Not used if OTMA security is set to NONE IMS checks IMSXCF.XCFgroupname.MQ-member-name MQ must have read access to this class MQ checks IMSXCF.XCFgroupname.IMS-member-name The WebSphere MQ subsystem Userid access to this class determines Userid validation for each message that crosses the IMS Bridge

Three levels of security (continued) 2. Userid Validation in MQ MQMD.UserIdentifier field is used Based on MQ Subsystem userid access to IMSXCF.XCFgroupname.IMS-member-name CONTROL/ALTER: Userids trusted, no checks UPDATE: Userid validated by RACF prior to passing to IMS Result of this check is cached and used on subsequent calls READ: Userid/password validated by RACF prior to passing to IMS NONE (or no profile): Userid/password validated by RACF prior to passing to IMS No cache is performed Once validated, a UTOKEN is passed to IMS and can be used for normal IMS security

Three levels of security (continued) 2. Userid Validation in MQ MQMD.UserIdentifier field is used If MQMD.UserIdentifier is blank WebSphere MQ will use the RACF default This is obtained from RACF This could cause security problems in IMS APAR PM68949 will change this If the Queue Manager has UPDATE access the message will be placed on the Dead Letter Queue with a new reason code

Three levels of security (continued) 2. Userid Validation in MQ The amount of time a UTOKEN in the MQ cache is valid is set by the MQ ALTER SECURITY command ALTER SECURITY INTERVAL(m) TIMEOUT(n) INTERVAL = How often (in minutes) to check for expired UTOKENS TIMEOUT = How long a UTOKEN (in minutes) is valid See MQ TECHNOTE for more information The MQ RVERIFY SECURITY command will expire all UTOKENs If the MQ Subsystem Userid access to IMSXCF.XCFgroupname.IMS-member-name is updated the IMS OTMA connection must be stopped and started for it to take effect /STO OTMA - stops ALL OTMA connections – not just this MQ /STA OTMA

Three levels of security (continued) 2. Userid Validation in MQ WebSphere MQ has a ZPARM parameter RESAUDIT(YES|NO) This specifies whether or not RACF audit records are written for MQ This parameter does NOT apply to the IMS Bridge RACF audit records for the IMS Bridge can not be turned off

Three levels of security (continued) 3. OTMA Security (set by /SECURE OTMA) CHECK Existing RACF calls are made IMS commands are checked against the CIMS class IMS transactions are checked against the TIMS class FULL Same as CHECK, but the ACEEs are built in the dependent regions as well as the control region Default at IMS cold start NONE No calls to RACF are made PROFILE Each message defines the level of security checking to be done MQIIH.SecurityScope field allows for FULL or CHECK to be specified

WebSphere MQ security profiles in the MQADMIN class If subsysid.NO.SUBSYS.SECURITY is defined... MQ does not pass Userid to IMS Client bid only succeeds when /SEC OTMA NONE is in effect (no Userid is passed to the IMS transaction) Good for early testing

WebSphere MQ security profiles in the OPERCMDS class If you are using RACF to protect resources in the OPERCMDS class Make sure that MQ has the authority to issue the MODIFY command to an IMS system to which it might connect

Persistence Determination The OTMA DRU exit can specify that a newly created TPIPE is SYNChronized Turn on the high-order bit in the first flag in the member override area But this is not enough There is still no valid sequence number There is a second bit in the first flag of the member override area to also indicate that a valid sequence number is required MVI OUTFLAG,X’C0’

MQMD as seen by the MQGET for OTMA output MQMD_REPORT = (see previous) MQMD_MSGTYPE = MQMT_REPLY (x’02’) PK61626 allows user to override in returned MQMD MQMD_FORMAT = (see previous discussion) MQMD_PERSISTENCE = (see previous) MQMD_MSGID = (see previous) MQMD_CORRELID = (see previous) MQMD_REPLYTOQ = blanks MQMD_REPLYTOQMGR = MVS Queue Manager

MQMD as seen by the MQGET (continued) MQMD_USERIDENTIFIER = blanks Unless you explicitly value it in the OTMA DRU exit or the input message came from the IMS Bridge You can copy the USERID passed in the OTMA DRU exit PARMLIST MQMD_ACCOUNTINGTOKEN = Garbage MQMD_APPLIDENTITYDATA = blanks or the input message came from the IMS Bridge and it was valued by the application that did the MQPUT Then MQ will pass whatever you put in there MQMD_PUTAPPLTYPE = MQAT_XCF (x’14’) MQMD_PUTAPPLNAME = 8 Byte XCF Group Name 16 Byte XCF Member Name of the sending IMS

IMS and WebSphere MQ GSE BENELUX IMS User Group Meeting

Similar presentations

Presentation on theme: "IMS and WebSphere MQ GSE BENELUX IMS User Group Meeting"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

IMS and WebSphere MQ GSE BENELUX IMS User Group Meeting

Similar presentations

Presentation on theme: "IMS and WebSphere MQ GSE BENELUX IMS User Group Meeting"— Presentation transcript:

Similar presentations

About project

Feedback