1. 1© 2005 Sonic Software Corporation The BAA Terminal 5 Project – The reality of XML, ESB and Web services

2. 2© 2005 Sonic Software Corporation The reality of serving 35 million people per year 37 million man hours to build T5 6.5 million cubic metres of earth works 15,000 cubic metres of concrete per week 16 major projects, 100 sub-projects Sub projects cost between £30M and £150M 60,000 people involved in the build The IT infrastructure must operate entirely new level of speed, efficiency and availability (they land planes!)

3. 3© 2005 Sonic Software Corporation The IT landscape – must be integrated 6000 display systems, 400 COTS apps, 197 line of business apps, 35 operational IT platforms, over 1000 servers One hour server failure has Europe-wide impact on flights, more than one hour has global impact There have been two failures of over 6 hours in the last 8 months – these even caused several mile tailbacks on major surrounding roads £250M fine for late delivery of T5 !!!

4. CONNECT EVERYTHING. ACHIEVE ANYTHING. ™ Why not Web services alone? If we asked you to solve this problem with WS-*, SOAP and XML – what would happen?

5. 5© 2005 Sonic Software Corporation How would we get there? CRM ERP PARTNER SYSTEMS FINANCE Today’s architecture rigid, costly and difficult to operate Proprietary technologies and skill sets Multiple communication infrastructures High cost of license, consulting and operation Lots of turf control and organizational issues ORDER ENTRY

6. 6© 2005 Sonic Software Corporation J2EE™ APPLICATION PACKAGED APPLICATION & LEGACY SYSTEMS.NET™ APPLICATION PARTNER SYSTEM WEB SERVICE In walks MOM – and Web services Hiding implementation details enables reuse XML-based data easily exchanged Designed for remote access, across heterogeneous platforms Can be easily passed over HTTP(S), JMS, CORBA, Sockets, MQ, RV and almost any other messaging layer Standard Interfaces are Major Step Forward TCP/IP WEB SERVICES INTERFACE XML

7. 7© 2005 Sonic Software Corporation WEB SERVICES INTERFACE J2EE™ APPLICATION PACKAGED APPLICATION & LEGACY SYSTEMS.NET™ APPLICATION PARTNER SYSTEM WEB SERVICE Web Services Is it reliable, scalable and secure? How do you change business processes? How do you manage and monitor distributed services? But Have We Solved The Whole Problem? Web services are interoperable communications stacks and don’t offer routing, service deployment, management, format transformation, guaranteed delivery, etc. You are building standards based spaghetti ! TCP/IP

8. 8© 2005 Sonic Software Corporation Why not Web Services alone? WS-Reliable Messaging creates reliable point-to-point connections –But still 100’s or 1,000’s of them – where’s the manageability? –How do you configure reliability to suit your needs? WS-Security creates security “Swiss cheese”… –Each secure Web service needs to authenticate incoming messages –All accessing corporate security server? At the same time? –Creates 100’s of security holes? Can you run your Web services in the DMZ? –Many customers we work with simply do not allow external Web services WS still lacks federated enterprise features

9. 9© 2005 Sonic Software Corporation Where is the Strategic Inflection Point? When the balance of forces shifts from the old structure, from the old ways of doing business and the old ways of competing, to the new. Before the strategic inflection point, the industry simply was more like the old. After it, it is more like the new. It is a point where the curve has subtly but profoundly changed, never to change back again. - Andy Grove, Only the Paranoid Survive, 1996 By 2008, Gartner predicts that SOA will be a prevailing software-engineering practice, ending the 40-year domination of monolithic software architecture.

10. CONNECT EVERYTHING. ACHIEVE ANYTHING. ™ So, back to BAA…

11. 11© 2005 Sonic Software Corporation BAA – The Strategy “… our strategy is to minimize the interdependencies between products, using open standards to increase operational flexibility and make sure that applications are responsive to change. Therefore a Service Oriented Architecture approach is inevitable. Our first challenge was to find a platform that would work well in our very demanding environment, and could orchestrate the services that will drive T5 operations. Sonic Enterprise Service Bus is a very natural fit." Nick Gains Head of IT BAA

12. 12© 2005 Sonic Software Corporation Why did BAA choose SOA? New technology, regulations, re-organizations, and market demands For years, change velocity has outstripped IT capacity Legacy integration approaches failed Costs -- license and services -- exceeded plans Broker / platform stacks: costly, closed, complex Infrastructure never scaled to the extent of the enterprise For example, not one single terminal has opened successfully in over 25 years The last attempt was Seoul with CORBA – it was a complete failure Very high development costs, particularly with integration Time delays where projects are always behind the business needs Lack of visibility and understanding of systems Typical Result:

13. 13© 2005 Sonic Software Corporation Service Oriented Architectures lead to: Reduced costs — Simplifies the integration process by making application interoperability "plug-and-play". By utilising open standards, there is less software infrastructure to purchase and maintain. Faster time to market — An extended enterprise will be able to respond more quickly to market changes than its competitors, as its business is more agile. Greater operating efficiencies — Companies will be able to reuse existing application components and utilise new services, rather then manually duplicating them in-house. Increased customer satisfaction — Through tighter integration of the business value chain and less manual intervention in business processes, suppliers and customers will have greater reuse of data, and more reliable and timely information.

14. 14© 2005 Sonic Software Corporation INCREMENTAL DEPLOYMENT BAA’s Enterprise SOA Vision BROAD-SCALE INTEROPERABILITY MODULARITY / REUSE FLEXIBILITY APPLICATION SERVER USER-DEFINED SERVICE LEGACY APPLICATION PROCESS SERVER RELATIONAL DATABASE BATCH SYSTEM PORTAL SERVICE Benefits address long-standing IT dilemmas What most people concentrate on are the endpoints But the problem area is the “white-space” of SOA

15. 15© 2005 Sonic Software Corporation Architectural Challenges of SOA platform Dependability – they land planes! –Reliable, high-performance communications between services –High Availability = Business continuity –Security Flexibility – this is a changing environment –Mechanism to orchestrate process through the network –Ability to dynamically re-configure services to new uses –Ability to normalise in-flight documents between services –Bridge multiple low-level middleware technologies Reach and scale –Connect any resources regardless of where they are deployed –Scale from initial phases to arbitrarily large deployment –Retain visibility and control of distributed infrastructure

16. 16© 2005 Sonic Software Corporation INCREMENTAL DEPLOYMENT Enterprise SOA BROAD-SCALE INTEROPERABILITY MODULARITY / REUSE FLEXIBILITY APPLICATION SERVER USER-DEFINED SERVICE LEGACY APPLICATION PROCESS SERVER RELATIONAL DATABASE BATCH SYSTEM PORTAL SERVICE BAA picked an ESB as their SOA framework BAA chose an ESB to address this SOA “white-space” HIGH AVAILABILITY

17. 17© 2005 Sonic Software Corporation WITHOUT AN ESB The Purpose of an ESB WITH AN ESB Connect, Mediate and Control ConnectControl Mediate

18. 18© 2005 Sonic Software Corporation Connect applications and services Multiple on-ramps, dependable communications Connect old and new –Legacy applications, RDBMS –J2EE,.Net –Web services –B2B protocols Link services and processes across the extended enterprise Establish robust, scalable and secure communications Connect All connected resources are first-class citizens Examples: –Securely link internal processes with those of business partners. –Portal integration

19. 19© 2005 Sonic Software Corporation Mediate services Bridge and extend incompatible technologies Mediate Reconcile system incompatibilities –Communication Protocol –Interaction model Transform and enrich data –Map between data formats –Split, aggregate and enrich data Provide flexible routing and process flow –Decoupled, event-driven services –Intelligent routing –Support stateful process management Eliminate service interdependencies Examples: –Aggregate data from multiple SAP systems –Regulation compliance logging

20. 20© 2005 Sonic Software Corporation Control service interaction Deploy, configure, manage Dynamically configure, deploy and upgrade hosted services Establish and alter process flows, routing, Quality of Service Gain control and visibility over services and their interaction Control Configured, not coded Examples: –Deploy and upgrade 1000s of end-points from a single location. –Detect faults and diagnose problems in complex deployment.

21. 21© 2005 Sonic Software Corporation Global reach, global scalability End-to-end SOA ESB spans clusters and security infrastructure to form federated environment Bus topology obviates hub-and-spoke bottlenecks Deploy what you need, where and when you need it

22. CONNECT EVERYTHING. ACHIEVE ANYTHING. ™ What are the technical problems? The Devil is in the detail…

23. 23© 2005 Sonic Software Corporation How do you manage a project this big? How do they leverage their existing IT portfolio? What will this cost? What would be the impact of –Changes? –Expansion? –New security threats? –Regulation changes? How will they accommodate future requirements? Business Process Definitions

24. 24© 2005 Sonic Software Corporation Imagine Project Managing the Internet …would you scope the project? …would you consider all future needs …would you handle training? …would you manage change? …would ensure interoperability? …would you manage the technical differences? …would you manage the scale? …would you manage the risk? How…

25. 25© 2005 Sonic Software Corporation So how was Internet Successful? Built around a few sacred principles Evolved from selected technology standards Deployment abstracted from design Incremental deployment Tactical execution What made it work?

26. 26© 2005 Sonic Software Corporation Strategy Versus Tactics This is how BAA are making T5 a success Everything is broken down into manageable tasks Matrix management Evolutionary project management –NOT Waterfall project management An IT back-bone and architecture from the very start Industry patterns are being exploited What makes some ideas work where others fail?

27. 27© 2005 Sonic Software Corporation Design patterns I encourage you to work with patterns – check out: –http://www.enterpriseintegrationpatterns.com/http://www.enterpriseintegrationpatterns.com/ –http://www.oreilly.com/catalog/esb/index.htmlhttp://www.oreilly.com/catalog/esb/index.html Working with patterns… –Some examples…

28. CONNECT EVERYTHING. ACHIEVE ANYTHING. ™ Scenario from a European Airline Printing

29. 29© 2005 Sonic Software Corporation Solution Scenarios - CITP

30. 30© 2005 Sonic Software Corporation CITP in Pattern language

31. 31© 2005 Sonic Software Corporation Micro Patterns as Services Print Req. ESB Infrastructure PDS JMS Web JCA MDB EJB SSB Servlet Portlet P2P CITP MQ 1 2 34 5 5 5 1.Print Request arrives at CITP 2.Request crosses the MQ Series Bridge 3.Print Token is resolved in PDS 4.Request is routed via CBR 5.Request is consumed in Terminal

32. 32© 2005 Sonic Software Corporation Reservation Applications Using a re-factoring pattern AgentCentral CRMFinance Applications SOAP Browser XML SOAP DB Registration App Audit Adapter Integration Broker ePoS Client Siebel Adapter Mainframe MQ Series

33. 33© 2005 Sonic Software Corporation Incremental Staged Deployment Registration App Browser DB ESB Enterprise SOA – one step at a time Service Containers ePoS SOAP HTTP WS SOAP Integration Broker Audit Adapter COM Client Siebel Adapter MQ Series Mainframe Reservation ApplicationsAgentCentral CRMFinance Applications

34. 34© 2005 Sonic Software Corporation Widely Distributed Enterprises Registration App Browser ESB ePoS SOAP Integration Broker Audit Adapter SOAP HTTP WS Client Siebel Adapter MQ Series Mainframe DB Reservation ApplicationsAgentCentral CRMFinance Applications

35. 35© 2005 Sonic Software Corporation Leave and Layer Registration App Browser DB ESB ePoS SOAP HTTP WS Integration Broker Client Siebel Adapter Audit Adapter ESB MQ Series Mainframe Reservation ApplicationsAgentCentral CRMFinance Applications

36. 36© 2005 Sonic Software Corporation Partner Integration Registration App Browser DB ESB ePoS SOAP MQ Series Siebel Adapter Audit Adapter SOAP HTTP WS ESB Integration Broker Mainframe Client Reservation ApplicationsAgentCentral CRMFinance Applications

37. 37© 2005 Sonic Software Corporation Incremental Adoption I like to say: “Think Strategically, Act Tactically” Registration App Browser DB Mainframe Client ESB Phase 1 Phase 2 Phase n MQ Series ESB ePoS Siebel Adapter Audit Adapter Partner ESB Integration Broker Reservation ApplicationsAgentCentral CRMFinance Applications

38. CONNECT EVERYTHING. ACHIEVE ANYTHING. ™ Some architecture tricks to be aware of… – Dealing with: Trapped messages Out of Order messages

39. 39© 2005 Sonic Software Corporation Machine failure causes trapped messages Two machines, a broker each and clustered Server Client 12 3 4 Machine fails!

40. 40© 2005 Sonic Software Corporation Clustering means system carries on straight away Two machines, a broker each and clustered Server Client 12 Machine fails! 3 4

41. 41© 2005 Sonic Software Corporation Machine recovers and messages delivered Two machines, a broker each and clustered Server Client 12 Machine fails! 3 4 Recovery of messages takes: 1. Machine reboot 2. OS restart 3. Software reload/restart 4. Database recovery 5. Re-send of messages = several minutes…

42. 42© 2005 Sonic Software Corporation Can you wait several minutes? Trader trying to execute a buy Retailer trying to process your credit card at a till Bank trying to process mortgages before the end of the day Airport trying to route baggage Telco network usage just before billing run at end of month Many other situations… It gets worse… Now add a requirement for guaranteed message ordering

43. 43© 2005 Sonic Software Corporation Typical scenario – an airport display board Flight Information (four messages) 12 3 4 Messages: 1 = BMI256 last call gate 3 2 = BA35 to gate 19 3 = BA35 to gate 23 (correction) 4 = BA765 last call gate 15 Machine fails!

44. 44© 2005 Sonic Software Corporation Messages: 1 = BMI256 last call gate 3 2 = BA35 to gate 19 3 = BA35 to gate 23 (correction) 4 = BA765 last call gate 15 Message ordering causes “pile-up” in other broker Flight Information (four messages) 12 3 4 Machine fails! Where are my messages! Due to guaranteed message ordering, can’t deliver other messages! And remember, recovery takes several minutes… (I can see potential for missed flights!) Think of complexity if you add in XA transactions

45. 45© 2005 Sonic Software Corporation Turn to Continuous Availability Architecture CAA Conventional hardware Seconds to recover Simple and flexible Maintains transactional integrity Easy to use with message ordering CAA HOT-HOT deployment Increases overall throughput Each broker has a hot backup broker on another machine Transparent to client Transactional integrity during failover Once and only once delivery ensured Fully scalable as we can also add clustering!

46. 46© 2005 Sonic Software Corporation P Remember our airport? Flight Information (four messages) 11 22 33 44 Machine fails! Secondary becomes primary and sends messages

47. 47© 2005 Sonic Software Corporation Sonic ESB Overview architecture for BAA T5 Sonic ClusterBroker A’ A’ Broker B Broker A Broker B’ B’ Sonic ClusterBroker A’ A’ Broker B Broker A Broker B’ B’ Geographical Site 1 Geographical Site 2 AODB Sonic ESB custom Java services AODB Sonic ESB custom Java services Client App Subscribe Adapter Client App Subscribe Adapter

48. 48© 2005 Sonic Software Corporation Summary – moving to “One Architecture” I suggest you look into the following: –Evolutionary project management Things like extreme programming techniques –Design and architecture patterns www.enterpriseintegrationpatterns.com –Service Oriented Architecture This is one of the most sought after roles in IT right now –(There were 280 architect roles on monster yesterday) Enterprise Service Bus concepts – read the ESB book! SOA has gone past the critical inflection point –Learn SOA, understand the nuances (the detail) –Serious money is being spent on this new technology –Web services is important, but is only one part of the hybrid architecture that companies are moving toward