1. A new model and architecture for data stream management

2. Inspiration & Domain Sample Problem

3. Stream Processing Engines  HADP vs DAHP  Events & Triggers  Continuous Queries  Real-time processing  Transient data  Lossy information

4. Application Domains  Online Auctions  Network Traffic Management  Habitat Monitoring  Military Logistics  Immersive Environments  Road Traffic Monitoring  System Monitoring  Smart Energy Grid

5. Overview Aurora

6. The Topic  Aurora  The prototype  DBMS / SPE / DSMS  UI  The query language  The project  The authors

7. The Authors  M.I.T., Department of EECS and Laboratory of Computer Science  Michael Stonebraker  Brandeis University, Department of Computer Science  Daniel J. Abadi  Mitch Cherniack  Brown University, Department of Computer Science  Don Carney  Uğur Çetintemel  Christian Convey  Sangdon Lee  Nesime Tatbul  Stan Zdonik

8. Talk Overview  Stream Processing Engines  SQuAl  Runtime  Related work

9. SQuAl (Stream Query Algebra) Aurora

10. SQuAl Overview  Connection Points  Models  Continuous Query  View  Ad-hoc Query  Operators  Order-agnostic  Order-sensitive

11. SQuAl Operators  Order-agnostic  Filter  Map  Union  Order-sensitive  BSort  Aggregate  Join  Resample  Quirks!

12. Filter (Unordered)

13. Map (Unordered)

14. Union (Unordered)

15. BSort (Ordered)

16. Aggregate (Ordered)

17. Join (Ordered)

18. Resample (Ordered)  Based on RRDTool’s philosophy?  Paper:  Simple interpolation  Use The Force, Read The Source:  Average  Count  Sum  Max  Min  LastVal

19. SQuAl: Example

20. Runtime Aurora

21. Query Optimization  Dynamic Continuous Query Optimization  Inserting projections  Combining boxes  Reordering boxes  Ad-hoc query optimization

22. Real-time Scheduling  Timestamped Tuples  Train scheduling  Interbox nonlinearities  Intrabox nonlinearities  Superboxes  Introspection  Static  Run-time

23. Handling overload  QoS specifications  Response times  Tuple drops  Values produced  Load Shedding  Not Implemented at the time

24. Related work Aurora

25. Related work  STREAM  Stanford University, 2000-2006  Telegraph  UC Berkley, 2000-2007?  SASE  UC Berkley / Mass Amherst, 2006-2008?  Cayuga  Cornell University, 2005-2007?  PIPES  University of Marburg, 2003-2007?  NiagaraCQ  University of Wiscon-Madison, 1999-2002

26. Aurora’s Evolution TimespanProject 2002-2004Aurora (and Aurora*) 2003-2005Medusa 2005-2008Borealis (Medusa + Aurora*) 2003-presentStreamBase (Commercialized)

27. Complex Event Processing Today  Oracle  Oracle CEP  Microsoft  MS SQL Server StreamInsight  Open Source  OpenPDC  Aleri  Coral8  TruViso  StreamBase  Aurora’s Grandchild  IBM  SPADE  Active Middleware Technology