Presentation on theme: "Creating an Event-Driven SOA"— Presentation transcript:
1Creating an Event-Driven SOA Madhusudhan KMChief Architect & Head – Central Architecture Group, IT Services, MindTree Ltd
2Topics Concepts, Sample Use case & EDA characteristics EDA Reference ArchitecturesPresent & Future applications of EDAMajor players in EDASummary & Conclusion
3Event driven worldWe all live in an event driven worldAll animals along with human beings respond to events of two types ; opportunities & threatsZebras on the savannah sense and respond to the opportunity of a water hole or the threat of a lion.Banks track credit card usage to stop fraudulent charges as they occurWe pick telephone calls when it rings. We do not pick other’s telephone (usually)…sometimes, we also filter our own calls if the number if unknownWe have cognitive sensors and filters that help us to concentrate on events that are important to us and ignore events that are not related to us (or at least we think they are not related !!)Are enterprises using sensors to respond to business opportunities and to fight threats at right time ??
4Events – Airline Use Case BLR-Dubai-LHRI will miss my connecting flightI will have to cancel my hotel reservation and car rental bookingI will have to cancel my meetingsFlight gets delayedDubai AirportCabin crew to serve another flight in DubaiGround operations staff at destination will have to arrange alternate crewMake sure the passengers are accommodated on the next connecting flightBook hotels for passengers if the wait time is more than six hoursWe can’t serve the next flight to start from destination of this flight
5Events – Airline Use Case Event - Flight delayedImpact on passenger traveling for business purposesMight miss a connecting flightMight need to cancel/change hotel reservation & car rental reservationMight need to cancel/postpone a meetingImpact on AirlineThe flight crew may miss their next flights. The flights must be delayed or replacement crews assigned.Crew members who are passengers on the current flight may not be available for their assigned flights.Catering provider might have to be notified of this event if the meal preparation should be delayed / cancelled.If there are priority passengers on the flight, airlines may want to see how these priority passengers are affected and what could the airline do to reduce the inconvenience to priority passengers.
6Events – Concepts & Terms What is an Event ?An event is a notable thing that happens inside or outside your business. An event (business or system) may signify a problem or impending problem, an opportunity, a threshold, or a deviation.ExamplesA financial tradeAn airplane landsA sensor outputs a readingA change of state in a database or a finite state machineA key strokeA natural occurrence such as an earthquake
7Events – Concepts & Terms Simple Event : Any event in its simplest formFlight delay eventComplex Event : An event that is an abstraction of other events called its members.The 1929 stock market crash – an abstraction denoting many thousands of member events, including individual stock trades)The 2004 Indonesian Tsunami – an abstraction of many natural eventsDerived event : An event that is generated as a result of applying a method or process to one or more other events.An event reporting that company B has entered the bidding to take over A with probability 0.9, might be derived from an event reporting that the price of company A’s stock has jumped 10% in 5 minutes.The absence of an event, say in a given time interval, can lead to a derived event reporting that the first event did not
8Events – Concepts & Terms Event Driven Architecture : An architectural style in which some of the components are event driven and communicate by means of events.Simple Event Processing: In simple event processing, a notable event happens, initiating downstream action(s). Simple event processing is commonly used to drive the real-time flow of work—taking lag time and cost out of a business.Stream Event Processing : In stream event processing, linearly ordered stream of events are processedEvents (orders, RFID transmissions, etc) are both screened for notability and streamed to information subscribers.Stream event processing is commonly used to drive the real-time flow of information in and around the enterprise––enabling in time decision makingA stream may contain events of different types
9Events – Concepts & Terms Complex Event Processing : A sophisticated form of EDA that is used to extract the information value from multiple events.CEP systems find patterns in event data to detect opportunities and threats. Timely alerts are then pushed to the appropriate recipients, often using Business Activity Monitoring (BAM) dashboards. The result is faster and better operational decisions and more timely responses.In CEP, events usually across types, occur over long period of timeEvent correlation can be casual / temporal / spatialFew possible of application of CEP are Algorithmic trading, Fraud monitoring and detection, Global epidemic warning, Air traffic control, Self tuning databases, Adaptive middleware, etc.
11Events – Airline Use Case Event Management EngineEvent ConsumerSMS to Passenger – Your Flight delayed ; You are booked on alternate flightEvent ConsumerEvent ProducerTrigger a business process in Airline reservations to take care of rebooking passengers who missed connecting flightsEvent ConsumerTrigger a business process in Flight operations to arrange new set of cabin crew for the flight to start at destination
12Events – Tying concepts together Consumers receive events based on rules, filters, etc.Producers could be business processes, services/transactions, state change in state of entity in DB, user actions on applications, etcConsumers have no obligation to producersConsumers are usually producers tooEvent ProducersEvent HubEvent Consumers
13Events – Moving from Loose coupling to Minimal coupling ServiceEventData StoreProcess ServiceServiceEventEventLegacyBusiness Processes and Services are not hardwired…..instead, they are linked via events
15A framework for handling highly concurrent systems
16A framework for handling highly concurrent systems
17A framework for handling highly concurrent systems
18Staged Event Driven Architecture - SEDA Each Stage contains an incoming event queue, a thread pool, an event handler and a controllerEach stage can be independently managedStages can be run in series or in parallel or a combination of the twoEach stage is separated by a queueUse of event queues allows each stage to be individually load conditioned by using proper thresholds
20Simple Event Processing Source : Event-Driven Architecture Overview by Brenda M. Michelson, Sr. VP and Sr. Consultant, Patricia Seybold Group
21Stream Event Processing Source : Event-Driven Architecture Overview by Brenda M. Michelson, Sr. VP and Sr. Consultant, Patricia Seybold Group
22Complex Event Processing Source : Event-Driven Architecture Overview by Brenda M. Michelson, Sr. VP and Sr. Consultant, Patricia Seybold Group
23Event Processing in Business Intelligence Analyst driven BIAd-hoc queries, Periodic reports, data mining, etcOutput is mainly used for planningProcess driven BISome process driven BI systems use near real time information using BAM (Business Activity Monitoring)Input for BAM is event data that arrived in the last few seconds/minutesHistorical data is used to put the new data in in context and to enrich the information before it is distributedStrategy driven BIUsed to measure and manage overall business performanceAre we on track to meet our monthly target ?
24Periodic intelligence v/s Continuous intelligence Event driven system sends alert when things occur and not based on predetermined schedule or upon receiving ad-hoc queryReports complex events like exceptions events on KPIThey operate fast enough to allow operations staff to intervene in a transaction when it is still in flightSource : Event Processing : Designing IT systems for Agile companies by Mani Chandy & Roy Schutle
26Algorithmic tradingAlgorithmic trading is achieved using computer algorithms deciding on aspects of the order such as timing, price & quantity of the orderCEP engines are now used to predict a trend to make use of both sides of the market (ups & downs)As of 2006, a third of all EU and US stock trades were driven by automatic programs or algorithmsMultiple players in capital market segment are using CEP engines for algorithmic tradingBluecrest, Wall street on demand (WSOD), SunGard, etc
27Fraud detectionA 2006 CyberSource report stated that 2.8 billion USD was lost to online fraud in US & Canada in 2005RequirementsStop new account setups from fraudulent IP addressesVerify user identity in every user transaction based on click behaviorIdentify multiple users trying to login from same IP addressIdentify single user logins from multiple IP addresses within a time spanFraud detection systems enabled with CEP products are being used in Banks, Betting companies, Insurance companies, etcIn Banks, multiple withdrawals made from a card within a specific time interval from locations that are distance apart indicates a fraud
28Air Traffic ControlAutonomous event-driven air traffic management system spanning the world.Why is it needed ?Crowded air spaceThe technology used in ATC is dated (1960)Source : The Future Event Driven World Scenario 1: Air Traffic Management by David Luckham
29Potential future applications Business IntelligenceAlgorithmic TradingFraud DetectionEvent-Driven SOABusiness Activity MonitoringSupply Chain VisibilityAutonomic ComputingSelf-tuning databasesPlanning systemsSelf configuring hardware and software systemsAdaptive middlewareTravel IndustryAir Traffic ControlBaggage Tracking and TracingPassenger SecurityAutomatic re-accommodationIntelligent baggage trackingAircraft maintenance based on real time information
30Potential future applications Health CareGlobal Epidemic WarningHospital Patient MonitoringHome Monitoring SystemsRFID In-Body MonitoringAmbient IntelligenceSmart HomesSmart CitiesPersonalized Mobile Information SystemsHomeland SecurityStorm Evacuation SystemsFirst Responder SystemsTerrorism Situational AwarenessPower, Fuel, and Water System Monitoring
31Potential future applications Environmental SystemsWind EngineeringWeather SystemsWater SourcesHabitat ConservationMilitary SystemsCommand and control systemsPhysical Infrastructure MonitoringTransportation SystemsTraffic ManagementProduce Tracking SystemsTruck Routing Systems
33What to look for in EDA & CEP Products Event Pattern MatchingEvent Stream ProcessingAbility to support multiple event patternsFilter, Enrich, Split, Aggregate, CorrelateSQL like Event Query Language supportVariety of event data storage optionsSupport for multiple event type representations like Java Object & XMLMultiple integration optionsTools for event modeling & visualization with rich IDE along with debugging featuresEvent history replayJMX instrumentationAbility to expose services and to integrate into ESB products
34EDA & CEP – Major Players BusinessEventsOracle CEP ; Able to process 1 million events/sec with low latencyWebsphere Business EventsApama Event Processing Platform ; Specialize in solutions to capital markets like Algorithmic trading, Smart order routing, etcAleri Streaming Platform; Specialize in stream event processingEvent Processing Platform ; Specialize in stream event processingVhayu Velocity : High performance solution used widely for tick data management allowing users to perform real-time and historical market data analysis in a single CEP platform.Event Processing & CEP Platform ; Open source CEP platform
36Summary & ConclusionOrganizations should gear up to sense opportunity and threat events from both inside and outsideEvent Driven Architecture combined with SOA will enable organizations to achieve thisTechnology products to support EDA+SOA are maturingThere are many interesting futuristic use cases that are possible in different domainsSo…Let us sense this opportunity and respond by creating some interesting solutions
37AcknowledgementsUnderstanding Event-Driven Architecture : Article by Roy Schutle and Dr K Mani ChandyEvent Processing Glossary by David Luckham, Roy SchulteThe Future Event Driven World Scenario 1: Air Traffic Management by David LuckhamA Design Framework for Highly Concurrent Systems by Matt Welsh, Steven D. Gribble, Eric A. Brewer, and David Culler, Computer Science Division University of California, BerkeleySEDA: An Architecture for Well conditioned, Scalable Internet Services by Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, BerkeleyEvent-Driven Architecture Overview, Event-Driven SOA Is Just Part of the EDA Story by Brenda M. Michelson Sr. VP and Sr. Consultant, Patricia Seybold Group