1. Service Oriented Architecture (SOA) and Complex Event Processing (CEP) – Complementary Views of the Enterprise
John Salasin, Ph. D.
Defense Advanced Research Projects Agency (DARPA)

2. Agenda Objectives Reputed SOA Benefits
Conceptual view of an SOA-based system
Conceptual architecture of an SOA-based system
Reputed CEP Benefits
Conceptual view of a CEP system
Conceptual CEP Architecture
Generalized Event Processing Agent

3. Objectives To provide:
Technical background to justify discussions of similarities and differences;
More on CEP than SOA (less familiar)
Guidance on the contexts in which the techniques are most useful, and;
References to additional information.

4. Reputed SOA Benefits
Core Concept: “Orchestrating” business functions to provide services.
Reducing integration expense;
Increasing asset reuse;
Increasing business agility, and;
Reducing business risk.
1) Reducing integration expense by:
Using loosely-coupled integration components that reduce the complexity and hence the cost of integrating distributed computing environments;
Using standards-based interfaces such as Web Services, and;
Replacing multiple function calls for API-based integration with coarser-grained Services that can handle a wider range of interactions in a more flexible manner.
2) Increasing asset reuse -- Service reuse becomes the theme, rather than application integration. ROI derived from:
Improved time-to-market,
More responsive customer service,
Reduction in overall staffing, and;
A greater ability to outsource Service creation, implementation, and even composition.
3) Increasing business agility
Business users control business process definition and management.
Parts of an organization’s business process design, management, and opreation can be delegated to the responsible part of the organization,
SOA enables embedding a company’s business processes inside the operations of their suppliers or business partners, or visa versa.
4) Reduction of business risk
Increased business agility and control.
Improved business visibility in the face of change
Ensure compliance with changing regulations.
Improved governance offers a reduction of liability.
Reduced security risks by establishing corporate-wide security, privacy, and implementation policies, and providing auditable information trails.

5. Conceptual view of an SOA-based system
Requestor – person or another service – is user of service.
Registry – is automated broker serving the user (and provider).
Provider – where the work is done – service is executed according to agreed on performance and quality standards.

6. Conceptual architecture of an SOA-based system
This is traditional layered architecture view

7. Reputed CEP Benefits
Core concept: specifying and managing conceptual hierarchies, or patterns, of events.
Users receive information at the appropriate semantic level (Aggregation);
Flexibility in monitoring (run-time specification);
Analysis of event patterns (pattern matching against observed patterns or rules/policies), and;
Consolidation (correlation across hierarchic levels).
users receive information at the appropriate semantic level
in terms or concepts that are meaningful to them. We can detect patterns of low level events and generate high level events that correlate and summarize the data in the lower level patterns. Event correlation across hierarchic levels.
Flexibility
Communicating structures of event processing agents provide rapid reaction and interactive on-the-fly reconfiguration of strategies.
Analysis
CEP provides drill down diagnostic analysis – given a high-level event, we can determine what lower level events caused it.
Consolidation:
CEP can correlate a diversity of independent inputs from network-level detectors to application-level sensors (e.g., patterns of events on a manufacturing plant controller to observed defects in products)

8. Conceptual view of a CEP system
NOTE: This has been used to diagram both SOA and CEP.

9. Conceptual CEP Architecture
Event Processing Networks (EPNs) are networks of distributed event processing agents (EPAs) running in parallel .
Event service (Similar to Data Service in SOA systems) – 4 components
Schema Repository
stores Schemas that define the format of events
Event Repository
Stores records of events
templates for processors, and;
predefined templates that can be activated to filter, aggregate, or otherwise process events.
a consumer registry. (Analogous to routing in an SOA)
The consumer registry keeps track of which consumer is interested in which events.

10. Generalized Event Processing Agent
This Figure describes 4 types of Agent.
One to start and stop specified event processing activities,
a second to filter input patterns and pass only those that meet selection criteria,
a third to create an output event on matching one or more input patterns, and
a fourth to generate an alert (an output event) when ever a condition (e.g., involving causality, sequencing, or timing) is satisfied
Stored templates and schema allow rapid configuration and deployment.

11. CEP Illustrative Examples (1 of 2)
Pattern matching and analysis
Match what is actually happening (e.g., a broker “trading ahead” of placing a customer order large enough to change the price of a stock) with pre-defined constraints (e.g. trading ahead is illegal).
Causal Tracking
Identify the Partially Ordered Set (POSET) of events – at all levels -- related to a specific event of interest (e.g., failure of a machine on a chip fab line)
Assure that a sequence of events is correct with respect to policy in complex contexts involving multiparty transactions.

12. CEP Illustrative Examples (2 of 2) Loan Processing Example -- Causality and Aggregation
Events are characterized by:
Time – usually sequence
Causality – red arrows showing process or service orchestration
Multiple (2 in this case) levels of abstraction
Aggregation – dashed arrows showing lower level events contributing to the composite event (completed loan)
We can, for example:
If loan isn’t completed, drill down to discover which lower level event failed
Check to ensure that process is followed – e.g., that risk management process for loan X doesn’t start until after income review credit check, and house appraisal
Ensure that funding decision (event based on funding service) occurs after risk management – t5 >= t4

13. When CEP Is Useful (1 of 2)
SOA is satisfactory when application is data dependent – where the major concerns are with, e.g.:
The magnitude of variables rather than the time at which they are set;
Relationships among these magnitudes (e.g., sales / ft3 at a set of stores), and;
Triggers are based on magnitude (e.g., order when stock level =< “H”).

14. When CEP Is Useful (2 of 2)
CEP should be added when concern is with patterns of events in real-time. E.G., We:
• Know (or suspect) the pattern of events we’re looking for (e.g., trading ahead);
• Can filter out a finite set of events from the “cloud” to start the process (e.g., sequences starting with customer requests for a stock trade), or;
• Want to respond to the pattern immediately, without the delay in retrospective data searches to see if it occurred.

15. Potential Added Value of Combining SOA/CEP
CEP at Enterprise level could support data marshaling in complex processes
Optimize access sequences to allow greatest number (possibly weighted by process priorities or deadlines) of processes to continue.
Allow identification of causality or dependency relationships across independent services.
A potential problem with enterprise-wide services – conflicts regarding data access and concurrency
Since Services can be on indeterminate duration, can’t lock data.
=> “Optimistic” concurrency control may require Service that’s accessed data to check back to see if it’s been changed since access – if so, may require expensive rollback operation.
In a large enterprise, many Services may be trying to access data simultaneously.