1. INNOV-10 Progress® Event Engine™ Technical Overview Prashant Thumma Principal Software Engineer

2. 2© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Agenda Slide n Event Stream Processing (ESP) n Progress ESP products n Progress Event Engine (PSEE) n PSEE Performance n PSEE Integration

3. 3© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Event Stream Processing (CEP) n ESP is about Operational Business Data –Capture –Query –Analysis n Characteristics –High Volume Streaming Data –Time Critical Analysis –Consistent and Reproducible Results –Highly Available n Simple Structured Data –Data is not Relational or complex Object model What is ESP?

4. 4© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Event Stream Processing (ESP) n Event Processing Language –Data Driven –Flexible –Complete n Event Persistence Engine –High Performance n Analytical Capabilities –Temporal Data Analysis –Inbound and Outbound Event Processors n High Availability n Modeling, Deployment and Development Tools ESP Platform Requirements

5. 5© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Event Stream Processing (CEP) n Financial Markets –Program Trading –Testing Trading Strategies n RFID –Inventory Logistics –Asset Tracking –Theft Detection n Manufacturing –Capture and analysis of data from PLCs n Telecommunications –Cell Phone Call Tracking –Multimedia Services Billing –Network Management n Scientific –Sensor Data ESP Applications

6. 6© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Agenda Slide n Event Stream Processing (ESP) n Progress ESP products n Progress Event Engine (PSEE) n PSEE Performance n PSEE Integration

7. 7© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress ESP Products Event Cache for JMS Adapter Event Server - + - + - + XML Configuration Files Java API Progress® Event Engine™ (PSEE) C++ API - + - + - + XML Configuration Files Apama

8. 8© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Agenda Slide n Event Stream Processing (ESP) n Progress ESP products n Progress Event Engine (PSEE) n PSEE Performance n PSEE Integration

9. 9© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n It’s all about Event Data –Collection, storage, management and query n What is Event Data? –Streaming, constantly flowing information n Stock Trades, RFID, IP Packets, Mfg PLC’s –Categorized n Stock tick symbol, RFID tag ID, IP address –Typically Time Ordered n Trade time, RFID scan time What is it for?

10. 10© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine 3. Query query client Event Cache Service 1. Collect Event Data Feed store Convert external events to PSEE events 2. Organize Structures data for fast retrieval sort/merge Persistent Storage disk Data Flow

11. 11© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n High Performance, Scalable –Collection Engine n Store tens of thousands events per second n 20 TB+ database size n Index data is ~1% of database size –Query Engine n High Performance n Pipelined Architecture n Extensible Functionality –Simultaneous Query and Collection n Writes do not block reads! Why PSEE?

12. 12© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n Transparent Data Management –Event Persistence n Requires no coding to persist events –Transactional n Consistent Data n Hidden and Managed by PSEE –Tunable Latency n “Real-time” defined by you! Why PSEE?

13. 13© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n Simple API n Supports Multiple Languages –Java and C++ language support n Multiple Platform Support –Window, Solaris, Linux –Heterogeneous deployment Why PSEE?

14. 14© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n Functor: A unit of work in PSEE –3 Main types (Source, Process, Sink) –Built in library of Functors –Customer coded n Pipeline: A sequence of Functors –Collect, process, transform event streams n Task Manager –Manages pipeline execution n Events: Input and Output of Pipelines Components

15. 15© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n PSEE Event –XML schema –Supports All C++ Primitive Types Event Definition <Event name=“Trade” type=“TimedCategorizedEvent”>

16. 16© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview class MyCollectorFunctor : public Functor { public: void initialize(FunctorContext& ctx); void handle(FunctorContext& ctx); void finalize(FunctorContext& ctx); private: // functor data variables } Progress Event Engine n Implements Event Stream Processing Logic n Interface For Accessing Events Functor

17. 17© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine Collector Pipeline Event-Data File for Bulk Loading Event Stream StoreRealtimeFunctor Real-Time Data Feed Series (PSEE Database) MyCollectorFunctor

18. 18© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine Query Pipeline Series (PSEE Database) RetrieveFunctorFilterFunctorSortFunctor Event Stream Event Stream Event Stream MyXMLwriter Functor

19. 19© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine n Collector Functors –StoreRealtime –StoreBulkload n Query Functors –Retrieve –Filter –Sort –Count –Group Count Predefined Functors

20. 20© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Progress Event Engine Scalable Deployment OSES Query Adapter PSEE OSES Collection Adapter 2 Collection Adapter 1 PSEE Raw Series Optimized db’s for query Series database OSES Query Adapter PSEE Query Adapter OSES PSEE VMMA & CFA Scalability Messaging Scalability

21. 21© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Agenda Slide n Event Stream Processing (ESP) n Progress ESP products n Progress Event Engine (PSEE) n PSEE Performance n PSEE Integration

22. 22© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview PSEE Performance n Event storage (events/sec) –Real-time collector – pipeline using StoreRealtime functor –Bulk load collector – pipeline using StoreBulkload functor n Event retrieval (events/sec) –Threaded execution of query pipeline n Random query n Query on several (randomly chosen) categories Performance Metrics

23. 23© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview PSEE Performance n Database Characteristics –160 Million Events –20 Byte Event –10000 Categories n System Configuration –Sun Fire-V440 –Solaris 8 –4 CPUs (1GHz) –8 GB Memory Performance Testing Environment

24. 24© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview PSEE Performance n Real-time collector –94,062 events / second n Bulk load collector –191,152 events / second n Query –213 Queries / second –10000 events / query –2,134,560 events / sec Performance Testing Results

25. 25© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Agenda Slide n Event Stream Processing (ESP) n Progress ESP products n Progress Event Engine (PSEE) n PSEE Performance n PSEE Integration

26. 26© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview D I S C L A I M E R Under Development n This talk includes information about potential future products and/or product enhancements. n What I am going to say reflects our current thinking, but the information contained herein is preliminary and subject to change. Any future products we ultimately deliver may be materially different from what is described here. D I S C L A I M E R

27. 27© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview PSEE Integration n Provides Event Persistence for Apama n Initial Implementation for Customer –Tick Database for Back-Testing Trading Strategies –Query Interface Using Apama Event Language n Future –Extend Apama Event Language to support Persistence –Seamless Configuration via Apama Management Interfaces Integration with Apama

28. 28© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview PSEE Integration n PSEE on the bus using Event Cache n Event Cache Features –Native JMS Message Based Query –Results are Java Objects –Batched for Optimal Performance Integration with Sonic

29. 29© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview PSEE Integration n PSEE based RFID Accelerator provides 4GL Interface n Web Services based Implementation n RFID Accelerator as a reference implementation n Further 4GL Integration Integration with 4GL

30. 30© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview In Summary n Exponential growth of event data requires ESP solutions n PSEE + Apama is an ESP solution n Based on proven technologies n Simple! Infrastructure problems are handled by PSEE n It can be integrated into existing business systems

31. 31© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Questions?

32. 32© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview Thank you for your time!

33. 33© 2005 Progress Software Corporation INNOV-10 Progress Event Engine Technical Overview