Stream SQL, Rules, Subscriptions: It’s All The Same Hans-Arno Jacobsen Bell University Laboratory Chair Middleware Systems Research Group University of Toronto MIDDLEWARE SYSTEMS RESEARCH GROUP Go PSQL !

Event Processing with PADRES PADRES Event Language – A1: [class = item], [shipID = *], [weight < 10] – P1: [class, item],[shipID, ACME123],[weight, 4] – S1: [class = item], [weight > 3] Initially implemented with JESS (i.e., as rules) – LHS statements encode atomic subscriptions – LHS encode composite subscriptions – RHS encode notification semantics (e.g., forwarding logic) IF LHS op LHS op LHS THEN RHS PADRES SQL (PSQL) modeled after SQL for event and event stream processing

PSQL Schema Definition CREATE TABLE (attr op val [, attr op val]* ) CREATE TABLE (class = invoke, service = *, item = *, id = *) Advertisement: [class, =, invoke],[service, =,*],[item, =, *],[id, =, *] CREATE TABLE (class = result, service = *, item = *, id = *, state = *) Advertisement: [class, =, result],[service, =,*],[item, =, *], [id, =, *],[state,=,*]

Publishing INSERT ( attr [, attr]* ) VALUES ( val [, val]* ) INSERT (class, service, id) VALUES (invoke, ItemView, a0012) Publication: [class,invoke],[service,ItemView],[id, a0012]

Subscribing Querying and subscribing to data SELECT[ attr | function]* [FROM event pattern] WHERE [ attr op val]* [ GROUP BY attr*] [ HAVING function* ] [ … ] clause is optional We support AVG, MIN, MAX, COUNT as AVG(attr,N)

Example Atomic and hybrid subscription: SELECT* WHERE class = invoke, service = ItemView, item = T-Shirt, start_time = NOW – 1h, end_time = NOW + 1h [class, =, invoke], [service, =, ItemView],[item, =, T-Shirt], [start_time, =, NOW – 1h], [end_time, =, NOW + 1h]

Example Composite and hybrid subscription SELECTe2.service, e2.item, e2.id FROMe1 AND e2 WHERE e1.class = result, e1.service = ItemShipped, e1.state = 2, e2.class = invoke, e2.service = ItemView, e1.start_time = NOW – 2months, e1.item = e2.item [class,=,result],[service, =, ItemShipped],[item,=,$X], [state, =, 2],[start_time, =, NOW – 2 months], AND [class,=, invoke],[service, =, ItemView],[item,=,$X]

Historic Event Query DB Admin Publisher Subscriber P1P2 P3 P4P5 Publication space P(A) P S Partition the publication space Assign partitions Partition CANOE Summer School, Norway, 2009

Assigning A Partition AdvertisementSubscriptionPublication DB Admin Publisher Subscriber [class,DB_CONTROL],[command,STORE],[db,B3-DB],[partition_spec,`[class,=,reading],[level,>,0]`][class,=,reading],[level,>,0] Partition Partitioning Partial replication Full replication P S [class,DB_CONTROL],[command,STORE],[db,B3-DB], [partition_spec,” [class,=,reading],[level,>,0] ”] 9CANOE Summer School, Norway, 2009

Challenges & Obstacles Remaining Stream SQL ≈ Rules ≈ Subscriptions A common formal framework – A query algebra or calculus Simple (i.e., can be described in 2 pages or less) Algorithmic issues in processing expressions

The PSQL Language Schema or type definition CREATE TABLE (attr op val [, attr op val]* ) CREATE TABLE (class = invoke, service = *, item = *, id = *) Adv: [class, =, invoke],[service, =,*],[item, =, *],[id, =, *] CREATE TABLE (class = result, service = *, item = *, id = *, state = *) Adv: [class, =, invoke],[service, =,*],[item, =, *],[id, =, *],[state,=,*] Produce data INSERT ( attr [, attr]* ) VALUES ( val [, val]* ) INSERT (class, service, id) VALUES (invoke, ItemView, a0012) Pub: [class,invoke],[service,ItemView],[id, a0012]

Retail Scenario B B B B B Customer Web Services Manager Delivery Services