Event Driven Architectures with Apache Synapse Paul Fremantle VP, Apache Synapse Member, ASF CTO and Co-Founder, WSO2

Why am I talking about this? CTO and Co-Founder, WSO2 –Previously STSM in IBM WebSphere VP, Apache Synapse Apache Member PMC Member: –Incubator, WebServices, HTTPComponents, Qpid

Plan of Attack Brief introduction to Apache Synapse Understanding SOA approaches Event Driven Architecture –Why would you use it? –Options and approaches How Events fit into Synapse Case Study Futures

Why use an Event Driven Architecture?

Empowerment

Understanding the point of SOA SOA is fundamentally about empowered distributed systems Giving each department/organization/website/etc control of their own systems –Amazon is a great example of this –Went from a traditional three tier data/app/web architecture –Enabled small teams to be effective

Two different kinds of SOA Consumers ESB Services Consumers System ESB System

Wiring In a process-led or mashup-led SOA, the wiring is done using ESB configuration or BPEL

Scenario – Financial Security blocking Database legacy flat file NEW YORK Existing System Synapse Poll Record->XML XML->XML Send LONDON Synapse Split/Iterate DBLookup/Filter Transform to MQ Send Existing System XML/JMS

Overall ESB flow FileSystemListener NY Sequence Log Out-Only Flatpack E4X Send to London LondonEndpoint LondonSequence Iterate DBReport – log whole msg Filter/Drop NOAC Send

PushMePullYou

PushMePullYou scenario

Logical model Every Service Provider and Consumer understands the “canonical XML” –Of course in real life there will be adapters –These may be running in an ESB But ideally their OWN ESB (or part of a distributed ESB) The aim is that each department/group can “own” their own gunk –Not reliant on central systems to adapt

Event approach Eventing takes this one step further: –Its up to you to publish to the right place –Its up to you to subscribe to the right events You own the wiring too –Allows for situational integration

Loose coupled and Self Organizing

We know this!

Event based models

Actuators and Sensors An actuator emits an event A sensor accepts events

The Synapse model of Events Loosely based on WS-Eventing –But can be augmented with other models –Also more powerful Easily add mediation into the flow Main concept: –Event Source An Actuator

Advanced Web ServicesCore Axis © WSO2 Inc WS-Eventing Event Source Subscribe with EPR of subscriber Respond with EPR of SubsMgr Subscriber Event Notifications (any message) SubsMgr getStatus renew unsubscribe

Synapse configuration example

Eventing in Synapse Synapse Proxy Event Source Event Publisher Subs Mgr Subscriber Publisher Subscriber subscribe Mediation Sequence

Different subscription models WS-Eventing message Static subscription in Synapse XML Store-dependent –E.g. Registry model

Completely transport AND WS- Eventing agnostic You can have any kind of message republished to any kind of endpoint –SMTP -> Jabber –SOAP -> JMS You don’t HAVE to use WS-Eventing to subscribe You can bridge into JMS Topics too –Or bridge from JMS Topics

Registry-based subscription model A header carries the “Topic” –e.g. /config/hardware/server/windows/xp Subscribers can subscribe to a specific topic, or all sub-events The topic space is represented as a tree in the registry –Subscriptions are simply URLs stored as entries at a point in the tree /config/ /software/ /hardware/ /server/ /linux/ /windows /xp/ URL1 / URL2 (etc)`

Case Study

Enterprise IT Management Problem statement: –Customers have multiple installed management systems Network Management User Management Systems Management –All from the same vendor! –These are not just “stock” systems – each has been customized for each installation –Customers have to keep these systems in sync By data entry –Any solution needs to be flexible, extensible, modifiable

A difficult problem Synchronizing multiple different systems But: –Systems have different underlying formats –Some of the systems may be more accurate than others –Need to be able to scale to different numbers of systems –Must be extensible / reprogrammable

Feedback!

Feedback problems Black Box change update

Feedback loops Black Box system Black Box system Adapter Event Broker

EDA + MDR

What is Complex Event Processing? Simple Event Processing –Acting on single events –e.g. a in the ESB –Is this a gold or platinum customer? Event Stream Processing –Looking across multiple events –Finding patterns – e.g. the CPU utilization has been more than 90% for the last 10 minutes Complex Event Processing –Looking across multiple event streams –e.g There has been a significant increase in overall trading activity AND the average price of commodities has fallen 2% in the last 4 hours

Complex Event Processing in Open Source Esper – –A Java library that can be integrated into multiple systems –A GPLv2 project (with a commercial edition available) –Supports multiple query models Based on a SQL-like language Grouping, aggregation, sorting, filtering and merging of event streams Windows based on time, length, sorted, and others –Events can be XML, Map, Object

Complex Event Processing examples Event processing languages are typically based on SQL –With extensions! select * from Ticker.win:length_batch(10) select AVG(CPU) FROM GANGLIA.win:time(30 sec) insert into WarningStream select trainStation, avg(waitTime) as avgWait from MyWindow group by trainStation having avg(waitTime) > 60

Esper + Synapse Separate project – SciFlex – Allows you to “inject” events from Synapse to be processed by Esper Interesting events come back to you

Summary EDA is an interesting approach If you design correctly, you can build a really distributed, loosely coupled system

Questions?