Michael Florkowsi Information Builders Introduction to the iWay Service Manager Design and Transformation Engine Michael Florkowsi Information Builders

Introduction What is iWay? iWay Service Manager Components iWay Explorer iWay Transformer iWay Designer iWay Adapter Factory iWay SM Architecture

What is iWay? Iway is a suite of products designed to facilitate the transfer of information between applications and systems. With iWay’s tools, an application can be designed to accept information in a variety of different formats, process and enrich the information, and output it in any desired format. iWay SOA Middleware™ - A Complete SOA Implementation Solution The basic function of IT is to deliver data, or make data available in some way. “The flow of information has changed the way we live in today world. Information is the backbone of every system.” Role of Information Technology By Vinay Rana EZine ---Wikipedia - Information technology (IT), as defined by the Information Technology Association of America (ITAA), is "the study, design, development, implementation, support or management of computer-based information systems, particularly software applications and computer hardware." In short, IT deals with the use of electronic computers and computer software to convert, store, protect, process, transmit and retrieve information, securely. System to System Application to Application Data repository (RDBMS/Application) to End User Management uses data to make decisions to run the business All environments have data on many platforms, and tucked away in many different applications. DB2 data on the Mainframe, Application data on the iSeries, Oracle RDBMS data on Unix, SAP ERP system on Unix, SQL Server RDBMS on Windows Server So this is the environment many IT departments have to deal with. What the Universal Adapter Suite gives you is the ability to access all of these different areas. On top of that layer we have the Service Manager layer. Service manager is a lightweight, Highly scalable Enterprise Service BUS. The Graphical Workbench gives you the ability to expose and manage all of these different data sources through a consistent set of tools. Application Explorer Designer Transformer with dictionary builder Trading Partner Agreement capabilities iBSP – iWay Business Service Provider uses an HSQL repository Service Monitor and Policy Manager are also available (AmberPoint partnership) Process Manager (Active Endpoints BPEL engone) – allows us the ability to create long running stateful business processes. Uses an Eclipse based GUI. Enterprise Index is our Google product Trading Partner Manager – the Partner Agreements are imbedded within Service Manager, but the Trading Manager component is an adapter. Any of our adapters can be called through 3rd party tools.

Application Example Several of our business partners send us their orders in a comma-delimited format. Our goal is to build an automated integration process that will receive order line data from our business partners, transform the order data so we can satisfy all of the requirements of our database and add the orders to our order-entry system. We are going to build a process that takes a CSV file as input. We then need to enhance the data by adding a timestamp (Date_Updated) field value to satisfy our internal auditor’s requirement. Another step we are going to do is calculate the total item price (Total_Item_Price) to minimize code in our back-end application. This field is required for accounting and billing operations. We then need to check our quantity in stock, and verify we can satisfy the order before adding it into our order-entry system. If there is stock, we will write the output into a folder named ValidOrders using an Emitter. If there is no stock, we will write the output into a folder named BackOrders using an Emitter. The final output will be an XML file that conforms to the required structure based on all of our requirements.

iWay Service Manager 5.5 SM iSM Components Wikipedia ESB In computing, an enterprise service bus (ESB) refers to a software architecture construct, implemented by technologies found in a category of middleware infrastructure products usually based on standards, that provides foundational services for more complex architectures via an event-driven and standards-based messaging engine (the bus). An ESB generally provides an abstraction layer on top of an implementation of an enterprise messaging system which allows integration architects to exploit the value of messaging without writing code. Contrary to the more classical enterprise application integration (EAI) approach of a monolithic stack in a hub and spoke architecture, the foundation of an enterprise service bus is built of base functions broken up into their constituent parts, with distributed deployment where needed, working in harmony as necessary. ESB does not implement a service-oriented architecture (SOA) but provides the features with which one may be implemented. Although a common belief, ESB is not necessarily web-services based. ESB should be standards-based and flexible, supporting many transport mediums. Based on EAI rather than SOA patterns, it tries to remove the coupling between the service called and the transport medium. Most ESB providers now build ESBs to incorporate SOA principles and increase their sales, e.g. Business Process Execution Language (BPEL).

iWay Components iWay Service Manager (iSM) Host for Service-Oriented Architecture Supports configuring iWay Adapters of all types J2SE Environment (Server) Supports other iWay Components iWay Transformer iWay Designer iWay Explorer

iWay Components iWay Transformer Transforms XML and non-XML documents into XML and non-XML documents GUI mapping of source to target formats Alternative to XSLT Extensible Document Enhancement

iWay Components iWay Designer Performs complex stateless process execution Synchronous/Asynchronous

iWay Components iWay Explorer Allows exploration of EIS via the adapter Creates XML Schemas for Request/Response (Service) Creates XML Schemas for Result (Event) Consistent view of all Enterprise Systems Note: there is a Swing, ASP,Net and Elcipse version of Explorer

iWay Adapters - What is an Adapter ? An adapter is a software component that exposes an application’s functionality and data to iWay Service Manager using open standards. XML is the data format for the business level service.

Adapter Environments iWay Adapters require only a J2SE environment. iWay Adapters are resold in several environments: BEA WebLogic Server Oracle Fusion SAP Netweaver (XI) iWay Adapters can be deployed under any J2EE compliant Application Server: JBOSS IBM WebSphere Many others

iWay Adapter Factory 300+ Intelligent Adapters Adapter Classes Data Adapters: Provide a relational view of proprietary databases and file systems Application Adapters: e.g. SAP, Siebel, PeopleSoft, JDE Transaction Adapters: e.g. CICS, IMS, Tuxedo Technical Adapters: JMS, FTP, e-mail, 3GL, 4GL, etc. Emulation Adapters: e.g. 3270, 5250 eBusiness Adapters: e.g. EDI, SWIFT, cXML, HIPAA A D P T E R S Data Application Touchpoint Technical Emulation eBusiness Transaction Application System Adapters Ariba AXIS Baan BroadVision Clarify Commerce One Hogan Financials i2 Technologies J.D. Edwards Lawson Manugistics Microsoft CRM MySAP.com Oracle Applications PeopleSoft SAP Applications Siebel Vantive Walker Interactive e-Business Adapters AS1/AS2 Comma-Delimited Files cXML ebXML EDI-ANSI X12 (American) EDI-EDIFACT FIX FIXML Flat (positional) Files HIPAA HL7 ISO 15022 OAG BOD Structured Files SWIFT SWIFTML Transora UCCnet XBRL xCBL Data Adapters Adabas Adabas/C ALLBASE/SQL C-ISAM (Informix) C-ISAM (Microfocus) CA-Datacom CA-IDMS/DB CA-IDMS/SQL Cloudscape D-ISAM DB2 DB2/400 DBASE DBMS EJB ENSCRIBE Essbase Excel Flat Files FOCUS Data Access Foxpro IDS-II ImageSQL IMS InfoMan Informix Ingres Interplex ISAM KSAM Lotus Notes Microsoft Access Microsoft OLAP Services Microsoft SQL Server Millennium Model 204 MUMPS (Digital Standard MUMPS) NOMAD NonStop SQL Nucleus Omnidex OpenIngres Oracle PROGRESS QSAM Rdb Red Brick RMS SAP BW SAP R/3 SQL/DS SUPRA Sybase Sybase/IQ System 2000 Teradata TOTAL TurboIMAGE Unisys DMS 1100/2200 UNISYS DMS II UniVerse VSAM XML Transaction Processing Adapters Bull TDS Bull TP8 CICS IMS/TM Software AG Natural Tuxedo Terminal Emulation Adapters 3270 (Mainframe, zSeries) 5250 (AS/400, iSeries) Touchpoint Adapters Analysis Codes Bill of Material Catalogs Change Quality Chart of Accounts Component Issues Container Move Customer Inventory Item Item Maintenance Locations Order Changes Preliminary Invoices Production Order Purchase Order Receipts Requisition Returns Sales Order Suppliers Users Work Order Work Order Status Change Technology Adapters Any J2EE Application Server BEA MessageQ BEA WebLogic Application Server BEA WebLogic Integration C C++ cc:mail (VIM) COBOL COM FOCUS FORTRAN FTP Fujitsu Interstage HTTP IBM WebSphere Application Server IBM WebSphere MQ (MQ Series) IBM WebSphere MQ Integrator Internet Mail (SMTP/MIME) JacORB Java JMS Lotus Notes (VIM) Microsoft BizTalk Server Microsoft Exchange (MAPI) Oracle 9iAS Oracle 9iAS Integration Oracle AQ Oracle PL/SQL Orbix Pascal POP Python RDBMS RPG SAP eXchange Infrastructure (XI) SAP Web Application Server Sonic ESB SonicMQ SunONE Application Server TIBCO JMS TIBCO Rendezvous Visibroker Connector Technologies .NET ActiveX iWay JCA JDBC™ Microsoft DTS MQSeries ODBC OLE DB SOAP Access to your information assets can be broken down into groups or classes of adapters. Listed on this slide, are: Data Adapters – which provide a relational view of any relational or non-relational data structures like DB2, Oracle, SQL Server, VSAM, IDMS, ADABAS, flat files, MUMPS and so much more. Application Adapters provide transaction level access to the most popular ERP, CRM, and SCM packages like SAP, PeopleSoft, Siebel, and more. Transaction Adapters provide fast and efficient access to TP environments like CICS, IMS/TM and Tuxedo. Program Adapters allow organizations to leverage their existing libraries of custom written programs in COBOL, PL1, SQL RPCs and more. Emulation Adapters deliver easy access to legacy green screen applications through screen-scraping technologies for 3270, 5250 and VT100 based applications. e-Business Adapters represent the latest investment of research and development dollars from iWay Software. iWay’s new Adapter Factory has pumped out e-Business transaction adapters like SWIFT and FIX for the finance arena, HIPAA for healthcare industry, and Covisant for the Automotive industry. No other EAI vendor can sole source as many adapters over more adapter classes as iWay Software. In fact, there isn’t even a vendor in the adapter business that can provide the coverage of iWay. So how does everyone else bridge the gap between what iWay delivers and what they don’t?? Well, they ship an SDK – or software development kit. Hey, if you have to build your own access, test it and maintain it . . . That’s not integration assembly. Your back into a integration solution steepedin custom development.

iWay Adapters – iWAF (iWay Adapter Framework) Component with consistent set of XML and Java interfaces for Design Time and Run Time. Through these Interfaces, iWAF provides: Design Time metadata browsing capability Web Service presentation JCA wrapping capability  

Adapter Classes iWay Adapters are divided in 2 classes: Event adapter, designed to publish information from the application. Service adapter, designed to consume messages into application.

Adapter Classes - Event Adapters Event Adapters perform 3 functions: Trigger an “event” that occurs inside the running EIS and extract data about the event from the EIS into the adapter. Transform event data from the EIS-specific format to an XML document that conforms to the XML schema for the event. The XML schema is based on metadata in the EIS. Propagate the event. RDBMS listener SAP IDOC Something that is triggered We listen on an HTTP or TCP port or even a message queue

Event-based integration: real-time, no rekeying and no errors iWay Software - Events Event detected Message sent Systems in sync P.O. P.O. SAP Client SAP Server IMS Transactions Event-based integration: real-time, no rekeying and no errors

Adapter Classes - Service Adapters Service Adapters perform 3 functions: Process the XML Request into the EIS specific data format. Invoke the underlying function in the EIS and wait for its response. Transform the response from the EIS specific data format to an XML document that conforms to the response XML schema for the service. The response XML schema is based on metadata in the EIS. Service defined with a listener a pflow

Service-based integration: real-time, no rekeying and no errors iWay Software - Service Service requested Service fulfilled Lookup Request Application user Application Server SAP Server Service-based integration: real-time, no rekeying and no errors

iWay Service Manager 5.5 SM Architecture

What is an iWay Channel Easily configurable constructs for: accessing applications applying transformations (pre-emit & post-emit) receiving or emitting documents Channels help organize how you design components and then deploy them for execution. The channel concept has been introduced in iWay Service Manager to help organize how you design components and then deploy them for execution. Channels are easily configurable constructs for accessing applications, applying transformations, and receiving or emitting documents. They define a complete integration operation, which includes receipt of inputs, service processing, and transmission of outputs using any supported protocol.

3 Phases of a Channel Every Channel consists of an Inlet, Route and Outlet Inlet - Defines the entry point to a channel. Each channel must contain an inlet. In the inlet, the message may be decrypted and/or preparsed and then assigned to an appropriate route for further processing. Inlets contain listeners, decryptors, and preparsers. Route - Defines the path a message takes through a channel. Each channel must contain a reference to one or more routes. Along each route, transformations and other forms of business logic are applied to the message. Routes contain transforms, processes, and emitters. Outlet - Defines how a message leaves a channel. Each channel must contain one or more outlets. In the outlet, the message may be transformed and/or encrypted, and finally emitted by the system. Each message may pass through multiple outlets of a channel. Outlets contain preemitters, encryptors, and emitters. Note: The exits indicated with an asterisk (*) must be configured for a channel (Inlet, Listener, Process, Outlet).

iWay Service Manager -Channels Users will assemble conduits called Channels easily configurable components for accessing application, applying transformation, receiving or emitting documents into complete These channels define a complete integration operation: receipt of inputs, service processing and transmission of outputs using any supported protocol Channels contain references to Inlets, Routes and Outlets iWay Service Manager defines a conduit as a container of components thru which messages flow. The channel conduit is used to deploy message flows from the design time repository to one or more runtime server instances. The unit of deployment is known as a channel archive (.car).

iWay Service Manager -Inlets Inlets defines how a message enters the channel. Each channel contains exactly one Inlet. In the Inlet, the message may be decrypted and or preparsed (turned into XML) and then assigned to a route for further processing. Each inlet contains a sequence of components defined as follows: Listener ==> Decryptor ==> Preparser Listeners are defined as protocol handlers and are responsible for startup, shutdown, and obtaining the incoming messages. Listeners recieve the messages from the transport protocol, set special registers such as header values, input source, etc. and then pass the message off to a decryptor. Decryptors can apply a decryption algorithm to the incomming message and verify the security of the message. It can be used to verify that the sender is authorized, to check that the message has not been changed, and to decrypt any part of the message that has been encrypted. Finally the decryptor passes the message to a Preparser. Preparsers convert transported messages in wire format into processable documents. For example, some preparsers convert non-XML messages into XML documents. Preparsers can be stacked, so that the output of one preparser becomes the input of the next preparser. Input to the first preparser is a byte stream, and output is a properly encoded string. Subsequent preparsers accept and emit strings.

iWay Service Manager -Listeners Listeners are components that monitor communication protocols waiting for input documents or messages. A listener can be defined for the following protocols: Protocol Name Description AQ Oracle AQ queues exchange documents AS1 AS/1 protocol AS2 AS/2 protocol CS3 A TCP port is monitored for traffic in iWay’s CS3 protocol EMAIL iAM acts as an email client to a designated email system to retrieve messages sent to that system File A file or subdirectory of files is monitored for documents FTP Requests and responses are exchanged with an FTP server FTPD FTP Daemon – iAM operates as an FTP server so that clients can send documents directly to it. HTTP Receives a request from a browser client, adding a security layer Internal JMSQ Java Messaging queues exchange documents MQ IBM MQSeries queues and queue managers exchange documents MQJMS

iWay Service Manager -Listeners Listener Types (Continued) Protocol Name Description MQSI Documents are exchanged with iWay nodes operating in an IBM WMQI message flow MSMQ RDBMS Data is automatically retrieved from any JDBC data source table using an SQL query Relay A special purpose listener is used with WebSphere MQ for remote data access from another instance of iAM working with WMQI SAP A special-purpose listener receives an Idoc from an SAP R/3 system Schedule Triggers a custom agent to be executed at a regular interval to perform the required operations SOAP Receives requests containing Stored Procedure calls using Simple Object Access Protocol (SOAP) Sonic SonicMQ queues exchange documents TCP A TCP port is monitored for the arrival of a document Tibrv Exchanges messages with TIBCO Rendezvous

iWay Service Manager - Routes A Route is defined as a path the message takes thru the Channel. Each Channel must make reference to one or more routes. Along each route transformations and business logic are applied to the message. Routes contain references to Transformers, Processes and Outlets. A Route describes the path that a document takes during its passage through the system. It is the job of the inlet to convert the input message to a process able document. Transformers convert the document information to a common format suitable for general business processing. For example, similar messages from two trading partners might differ slightly in format; a common format is often desirable for business processing. Processes perform the actual business operations on the document. A business process is composed of one or more services, with appropriate switching, testing, iteration and error handling. Processes can call on other processes and web services, and in turn can be packaged as web services for external consumption. Business processes are constructed by a specially designed tool, called the Designer. Outlets pass on the processed document to one or more designated recipients. They convert the document to transport format and then emit the message.

iWay Service Manager – Outlets An Outlet determines how a message leaves a Channel. Each Channel contains one or more outlets. In the outlet, the message maybe transformed or encrypted and finally emitted by the system. It is important to note that each message may pass thru multiple outlets of a channel. Outets contain references to Preemitters, Encryptors and Emitters. Once a document has been processed, it must be sent on to one or more designated recipients. This is the job of the outlet. The outlet is responsible for all aspects of preparing the document for emission and then emitting it. Preemitters convert the document from internal form to an external format message. This may include simply flattening XML or may be more complex, involving transformation logic. An example is converting the document to external EDI or HIPAA format. Preemitters can be chained, so the output of one becomes the input to the next. The first preemitter receives the document in internal form, and flattens it to a message format. Subsequent preemitters can perform extra work on this message. Transformations are often prepared using a specially designed tool called the Transformer. Encryptors operate on the message that is ready for emitting. Parameters such as encryption keys or certificate aliases can be stored by destination address in the optional iWay Trading Manager component. The emitter uses the appropriate transport protocol to send the document on to its destination. Examples include JMS, HTTP, email, AS2, etc. Header information that has been prepared by processes and stored in special registers is applied to the message in a format-appropriate manner.

iWay Service Manager – Build/Deploy/Start Create a Channel Add an Inlet to the Channel. Create and Add a Listener to the Inlet. Add a Route to the Channel. Add a Transform to the Route. Add a Process Flow to the Route. Add an Outlet to the Channel. Build the Channel – create the Channel Archive file. Deploy the Channel to iSM. Start the Channel.

Build a Channel Building a Channel creates an Archive (.car) file. This file is used to deploy to iSM.

Deploy and Start a Channel The Channel Archive must be deployed to the iSM Server The Channel is started, initiating the Listener Protocol.

Deploying Services To The Runtime Business Service Provider Graphical Workbench Application Explorer Designer Transformer Partner Agreements iWay Service Manager: Enterprise Service Bus Deployment: Build from Registry, Deploy to Engine Registry Runtime Engine

Example of a Channel Channel PurchaseOrder PurchaseOrder Inlet PurchaseOrder Route PurchaseOrder Outlet PurchaseOrder File Listener PurchaseOrder Process Flow Default.outlet PurchaseOrder PurchaseOrderReq JDBC Service PurchaseOrder2 File Emitters

Review Discuss the iWay Service Manager Understand the architecture Examine the iWay Service Manager components and tools Next: Demo iSM, Designer, and Transformer