Enterprise Integration Integrating software in the Enterprise As “applications architecture” becomes more complex with a trend away from monolithic ERP systems to implementing best-of-breed point solutions, the integration of applications plays an important role in EA implementation. How can you use integration to assist in application deployment and ensure the suite of applications used by the organisation is scalable, reliable and consistent? Presented by Cameron Hart

Flow Software Founder & MD Software Developer by trade so I like things practical Building integration solutions since 1998 Lived through System Integration, ETL, EAI, SOA, and now ESB. Small, Medium, and Enterprise 500+ customers

Contents Enterprise Architecture needs integration Integration Overview Fundamentals of an Integration solution Practical Examples Cloud and EDI Integration

Enterprise Architecture & Integration

Enterprise Architecture Key Goal - Aligning IT systems with the organisation’s business strategy Viewing IT strategy holistically encompassing the entire organisation (and trading partners)

Enterprise Architecture cont… EA has seen a trend away from monolithic application architecture Modular approach using best- of-breed solutions specific to individual departments need Integration Solution

Enterprise Architecture cont… Better Fit Quicker to deploy Quicker to adapt Integration Solution

Enterprise Architecture cont… EA doesn’t stop at the organisations boundary Integration with Trading Partners necessary EDI Integration Solution

EA & Integration Modular approach means you have a higher reliance on third party application vendors that have predetermined the models, business processes, and data operations of their software What’s left is the task of making them work together for the organisation – selection, configuration & integration

Integration Challenge Multiple vendors and technologies Point to Point solutions get too complex Automation is actually what is needed – Business processes not data

Integration Spectrum Data DrivenProcess Driven Data Migration Irregular Data Loads Regular Data Feeds Data Synchronisation Application Integration EDI / Inter-Company Integration Business Workflows Automation Integration Different solutions are needed to address different integration requirements

Integration Overview

Rise of Integration The modular approach created new challenges – Interoperability – Consistency Point-to-Point ETL EAI SOA ESB – These are not products – they are ideas, approaches, methodologies, processes, or frameworks. – Each of these methodologies only cover a part of an overall integration solution

Fundamentals of Integration Dataset Definitions Format Transformation Data Mapping Event Handling Business Logic Transport Protocols Data Enrichment & Validation Data Persistence Routing Central Configuration Monitoring & Auditing Security Lots of different names; Integration Software Broker Middleware ETL SOA Hub & Spoke ESB EAI Message Queues

Dataset Definitions Define structure of data Different app, Different structure ie Product List, Customer List, Price List – Master Data ie Purchase Order, Creditor Invoice, Shipment Notice – Source documents / Transactions Examples of Metadata SQL DDL XSD Documentation

Format Transformation Data Format is the physical representation of a dataset – ie Purchase Order can be represented as CSV or XML Different applications support different formats Integration solution translates between formats Transformation

Data Mapping Manipulation of single data set to meet application requirements – ie Purchase Order in SAP different to purchase order in MS CRM Often occurs as part of format transformation Two applications using exactly the same format can need data mapping Common Mappings Account identifiers Product identifiers Units of Measure Tax representations

Event Handling Exchanging data like a Purchase Order is a series of events – Creation – Modification – Line Addition – Line Modification – Line Deletion Often multiple events occur at once

Event Handling in Detail Purchase Order – Modification Change of state Change of delivery details – Line Modification Shipment of goods Invoice of line Change of product Change of price Example – products shipped WMS - picking completed INVENTORY - product decreased WEBSITE - lines updated as shipped FINANCIAL - Invoice created JOB - Service Reminder created

Business Logic Business rules determine how data is shared and integrated Less rules in the end application enable quick adaption Different rules needed for each event

Transport Protocols Support multiple protocols Protocol conversion Transports

Data Enrichment and Validation Incoming data may need enrichment before sending to the final destination Data needs to be validated – Receiving apps expect valid data

Data Persistence Temporary persistence to support integration processes Necessary for state control and Long Running Processes Benefits from Long Term persistence

Routing End applications should not know the receiver Routing Branching Consolidation

Central Configuration Easy Management Independent configuration of publishers and consumers Prevents lost integration processes

Monitoring & Auditing Exception Reporting Application Errors VS Data Errors – Data Errors handled by business users Business Activity Monitoring (BAM)

Security Support major security standards – SSL – Signatures – Authentication

Review Key Features of Integration Data Transformation Event Handling Transport Protocols Data Enrichment Data Persistence Routing Central Configuration Monitoring Security Data Modelling Integration is not just a tool or a service Integration should be considered an application like any other application in your Enterprise Architecture Often used to fill the gaps other applications are not providing – this leads to failure if your integration solution is just a tool

Integration Benefits you should look for Reduce processing time Increase productivity Reduce keying errors Reduce mundane tasks – staff retention and morale Ensure processes are followed consistently

Integration Benefits cont… Reduce points of failure Quick adaption to changing business requirements Little or no modification of end applications Simplified management Business Activity Monitoring / Exception Reporting Future Proof

Practical Examples

Automated Data Migration – User Acceptance Testing (UAT) Acquisition strategy Single Sign On (SSO) Procurement

Automated Data Migration Often performed periodically involving many steps and manual manipulation of data Prone to errors and is a big task on Go Live Issues in historic data often appear long after Go Live Automated Data Migration is regular and self repairable. Continues after Go Live. Con: Reliant on application functionality

User Acceptance Testing (UAT) Automated Data Migration allows for better UAT UAT can fail because of bad user data Application errors during UAT can damage user data Recent, relevant data improves UAT Users can spot errors easier

Acquisition Strategy If the organisations strategy involves acquisition Integration solutions provide another option to migrating IT systems of acquisition's Important where acquisitions have unique IT requirements supported by current systems Similar for Franchises and Buying Groups

Single Sign On (SSO) Active Directory (ADFS) oAuth SAML OpenId

Procurement 37 SYSTEMS INVOLVED IN PROCUREMENT PROCESS SUPPLIERS STAFF FINANCIAL PROCUREMENT MIDDLEWARE 3 Way Matching Business Rules Structural Validation Data Validation Transports FormatsEDIFACT XML FTP AS2 Requisition Punch out Auto Re-order PDF FAX HTTP Purchase Order Goods Receipt Goods Costing Invoice Credits Reconciliation Remittance VAN Integration Solution Transports Mapping

Hot Integration Topics

Integration in the Cloud You can integrate to the Cloud whether you have in house integration solution or a hosted integration solution Integrating Cloud applications requires those applications to have Internet-ready APIs (Web Services, REST) Integrating anything over the Internet requires Internet- ready APIs Integration over the Internet is SLOW – carefully plan around volumes (of events) Reduce load by performing data comparison early – only send deltas

EDI Integration doesn’t stop at the corporate boundary – it just changes it name to EDI EDI uses the same technologies as Integration More complex as you only have control of one side of the Integration You can only achieve the lowest common denominator – Events, Data, Protocols, Monitoring, Business Processes

Summary Integration is important for a modular EA strategy Integration is an application – treat it as such Integration products are not an integration solution by themselves

Contacting Me Cameron Hart, Flow Software Ltd