1. Administrative Technology Services: Enterprise Applications
Enterprise Service Bus Proof of Concept To the service bus and beyond...
Lisa Justiniano
Administrative Technology Services:
Enterprise Applications

2. Agenda – Lisa Justiniano
John Shen
Agenda – Lisa Justiniano

3. Goals Objectives Introduction to ESBs Benefits Recommended Solution
Agenda
Goals
Objectives
Introduction to ESBs
Benefits
Recommended Solution
Impact
Timeline
Q & A
Lisa Justiniano

4. Project Goal
Problem: We keep “inventing” different ways to accomplish similar integration tasks, leading to a variety of software that needs to be maintained, supported and enhanced by staff who understand that particular technique. Solution: Provide standard ways to achieve these common integration tasks, thereby simplifying our software base and making it more easily maintainable and accessible by our partners in a consistent manner.
Lisa Justiniano
Secondary goal: Get our feet wet on Amazon Web Services.

5. POC Goals
Get experience with Enterprise Service Bus (ESB) software in general and with 2 or 3 specific ESB products.
Determine if an ESB could address our problem statement.
Identify an ESB that might best fits our needs.
Achieve these goals in three months (April – June, 2014)
Lisa Justiniano
Secondary goal:
Get our feet wet on Amazon Web Services.
Team did this POC as an extra-curricular activity, since they were already allocated to FY14 deliverables.
POC Team
John Marks (GMAS Practice)
Bill Brickman (HDW Practice)
John Shen (PeopleSoft Practice)
Mike Thomas (Lead Architect)
Sreeni Gunnala (FinProc Practice)
Lisa Justiniano (Sponsor)

6. John Shen
Objectives – John Shen

7. Objectives
Adopt a standard, very high-level language for building integrations between systems.
Replace existing solutions that we use to integrate systems, currently written in a variety of languages including:
Perl
ksh, bash, awk, sed
SQL*Plus, SQL*Loader
Java
Stop implementing a different solution for the same general problem over and over.
Make our integrations between applications more resilient by reducing or eliminating timing dependencies.
John Shen

8. Utilize a centralized solution in order to:
Objectives
Utilize a centralized solution in order to:
Reduce cost and time to production
Improve operational transparency
Reduce security complexity and overhead
Deploy a low cost, open and extendible solution.
Transition slowly and incrementally over time to avoid major disruption and allow measured adoption.
John Shen
What we are not trying to do:
This is about transactional messages between systems, not breaking systems up into Service Oriented Architecture.
We are not looking to replace Informatica, our primary extract / transform / load (ETL) tool.

9. Introduction to ESB Software - John Marks

10. Introduction to Enterprise Service Bus Software
Two Logical Components
ESB
Message Queues
John Marks – May be too complext for executives
Sample Code:
monitor a folder
xpath extract
sql query
write result to file
<from uri="file:/data/db/input"/>
<setBody><xpath resultType="java.lang.String">/query/lastName</xpath></setBody>
<to uri="sql:select count(*) as cnt from person where last_name=#?dataSource=myDS"/>
<setBody><simple>${body[0][CNT]}</simple></setBody>
<setBody><simple>${bodyAs(String)}</simple></setBody>
<to uri="file:/data/db/output"/>

11. ESB Functionality Host web services and call external web services
Monitor folders (local and remote) for incoming files
Transfer files to remote systems
Query and update databases
Query LDAP / AD directories
Send and receive JMS messages
Send and receive
XML parsing, searching etc.
Call shell scripts, Java bundles
Do SNMP queries
TCP sockets
John Marks

12. Benefits – Bill Brickman

13. Benefits Faster to market. More easily maintainable integrations:
Fewer lines of code.
Simple, URL-like, syntax.
Code is provided for us, we just write the configuration.
Integration libraries (DB, web services, LDAP & AD, etc.) means we can do things we wouldn’t have attempted before.
Transformations become possible, both in payload and in transmission protocol, without modifying the source or target systems.
Bill Brickman

14. Benefits
To make both synchronous and asynchronous integrations easier.
To enable all applications to take advantage of asynchronous integration, meaning:
Applications are decoupled so dependencies are relaxed:
One application can be taken down for maintenance without notifying or affecting other applications.
Less worry about missed time windows (as with current batch transfers).
Bill Brickman
Move from batch to transactional: fresh data gets to users faster.

15. Benefits Enforce generic and modular coding style
Many small reusable bundles of code.
Improve operational transparency
One place to see source of issues and usage trends.
Reduce security overhead
a common gateway where security is already controlled (ACLs and logins).
Centralized simplicity vs. point-to-point complexity
Bill Brickman

16. Benefits
Eliminate the “daisy chain” method of obtaining information so that:
Data gets to the end- users faster
Subscribers don’t need to know much about the source systems
There is a single source of “truth”
Bill Brickman – Can anyone take anything? Only authorized subscribers can get at the data.

17. Recommended Solution - Mike Thomas

18. Recommended Solution We looked at several products:
Mule ESB
Apache ServiceMix
Red Hat jBoss Fuse three closely related and use ActiveMQ
Red Hat jBoss Fabric8
Informatica
Our recommendation:
ESB Message Queue
Fabric8 for cloud deployment ActiveMQ
ServiceMix for conventional deployment
Free, open source, standards compliant
Multiple vendors (Apache – free; Red Hat – QA and support)
All support fail-over and load balancing
Mike Thomas

19. Recommended Solution – Deeper Dive
Fabric8 and Service Mix have common components:
OSGi Containers: Apache Felix, Apache Aries (deploy bundles)
Command line UI: Apache Karaf
Integration Library: Apache Camel (routes)
Web Services Library: Apache CXF
Many more
Fabric8 adds cloud deployment features.
Developer (standalone) setup in 5-10 minutes.
Mike Thomas

20. Recommended Solution – Deeper Dive
ActiveMQ JMS message broker
supports both kinds of message queues:
point to point
publish / subscribe
backed by our choice of persistence: Kaha DB, AMQ storage, JDBC (MySQL, Oracle, etc.)
If necessary we will evaluate alternatives to ActiveMQ like:
Oracle DB JMS implementation
Red Hat A-MQ
Mike Thomas

21. Recommended Solution - Architecture
Some possible configurations:
A. Multiple message brokers, private to specific groups of applications.
B. Single message broker, each application has its own ESB.
C. Single ESB and single message broker.
D. Shared message broker, ESBs private to groups of applications.
Mike Thomas

22. Recommended Solution - Architecture
Our recommendation:
Lowest cost and provides most benefit in terms of shared security, logging and auditing.
Reduced deployment time (infrastructure and ACLs are already in place).
Developer environment and deployables are the same regardless of which architecture we choose, so is possible to change architecture over time.
Mike Thomas
In the future we may need to move to a more complex architecture:
for performance reasons
to isolate populations (of developers, or systems, or even users) from one and other

23. Impact – Sreeni Gunnala

24. Impact on ATS Developers:
New language, new deployment mechanisms.
First integration might be hard. Second one is easy.
Training materials will be available.
Reduction in labor and drudgery required to build integrations
Impact on ESB POC team:
Outreach: demos, brown bags, wiki, and training materials.
Best practices for error handling, logging, integrating with scheduling and monitoring tools, and security and authorization.
Build-out TEST and PROD infrastructure (including evaluating alternatives to ActiveMQ like Oracle DB’s JMS).
Sreeni Gunnala
ESB POC Team will be affected because we are developers, and in addition we

25. Impact on Integration partners:
Will need to learn how to “subscribe” to a service
Will need to learn how to “consume” a service
Will need to assist with testing
and they...
Will get new options for integrating with Enterprise Applications
Will see improved reliability.
Will see reduced time to market for new integration requests.
Sreeni Gunnala

26. Next Steps – Lisa Justiniano

27. Next Steps Immediately build our TEST environment
Each Practice (Enterprise Applications) will identify one or two integrations that they would like to deploy via this new ESB as part of their FY15 Innovation Goal
New integration
Existing Service
Change management implications
Confirm our architectural choices
Expand infrastructure to support load balancing and fail-over.
Build our PROD environment
Deploy and stabilize
Lisa Justiniano
TEST environment will be bare-bones to start.
We plan to use Rob Parrot’s open stack and do this in the cloud
Change management
Within Enterprise Apps, we already have integrations between our Enterprise applications, so we may try to keep this within our own systems.
Once we have some integrations near production we will confirm our architectural choices

28. Next Steps 1.Prepare TEST Environ: months 0-4
Build
Pilot
Rollout
1.Prepare TEST Environ: months 0-4
3.Rollout Pilot: months 8 –18
Install and configure bare-bones infrastructure
Upgrade infrastructure for fail-over / load balancing
Determine and document best practices
Build PROD environment
2.Build Integrations: months 2-18
Deploy Pilot Integrations
Stabilize PROD environment and integrations
Build and test internal integrations
Verify assumptions
Refine training materials
Assess learning curve
4.Rollout to ATS: month 18 & beyond
Assess support issues
Confirm low impact and minimized risk
Lisa Justiniano
Build & Test internal integrations – Enterprise Applications will do initial integrations
Assess Support Issues – Performance, scalability , operational visibility, infrastructure Care/Feeding

29. Q & A
Lisa Justiniano

30. Example Camel Route
<!-- 1. Monitor folder for new files and use XPath get the surname to query from the XML input file. -->
<from uri="file:/data/db/input"/>
<setBody><xpath resultType="java.lang.String"> /query/lastName</xpath></setBody>
<!-- 2. DB query. -->
<to uri="sql:select count(*) as cnt from person where last_name=#?dataSource=myDS"/>
<!-- 3. Extract count and convert to string. -->
<setBody><simple>${body[0][CNT]}</simple></setBody>
<setBody><simple>${bodyAs(String)}</simple></setBody>
<!-- 4. write to file. -->
<to uri="file:/data/db/output"/>
Lisa Justiniano

31. Contact Lisa_Justiniano@harvard.edu ( x 60093 ) for more information.