1. Staged-DB IC-65 Advances in Data Management Systems 1 Scheduling in Staged- DB Systems Nicolas Bonvin, Rammohan Narendula, and Surender Reddy Yerva

2. Staged-DB IC-65 Advances in Data Management Systems 2 Organization What is Staged-DB? Scheduling in Staged-DB Our Contribution –Scheduling in Execution Phase –System Modeling System Design Details Performance Study Future Work

3. Motivation Response time: time needed to produce the first page as output Big advantage for the overlapping case ('1')

4. Staged-DB IC-65 Advances in Data Management Systems 4 Query PARSER OPTIMIZER EXECUTION Answer Query tree Query plan Data catalogs and statistics operators Query Lifetime in DBMS EXECUTION(Disk-IO) : 90% OF TIME

5. Staged-DB IC-65 Advances in Data Management Systems 5 DB Paradigm So Far.. Query  Query Execution Plan (Tree of Operators) Multiple Queries –Each query handled by a DIFFERENT THREAD No cross communication/sharing across threads  Sharing Opportunity is missed DBMS thread pool x no coordination D C D C One Query Multiple Operators

6. Staged-DB IC-65 Advances in Data Management Systems 6 Staged-DB Paradigm DB is remodeled as various stages Stage –“Common execution logic” grouped into a stage –Each operator in QEP can be seen as a stage Query passed through all the needed stages to get an output Common Data needs  Detected by the Stage DBMS thread pool D C D C StagedDB One Operator Multiple queries

7. Staged-DB IC-65 Advances in Data Management Systems 7 Staged Database Systems DB  Stages ; Execution Stage  microEngine Each Stage has a queue, Also each microEngine has a request queue. DBMS queries Stage 3 Stage 2 Stage 1 StagedDB queries Conventional High concurrency  locality across requests

8. Staged-DB IC-65 Advances in Data Management Systems 8 Scheduling In Staged-DB Scheduling at Different levels –Stages (Parser, Optimizer, Execution) –Across MicroEngines (Execution Engine has SCAN,JOIN etc micro-engines) –Within MicroEngine We Consider only scheduling “across microEngines” Scheduling Policies: –Round-Robin –Heavy Load First –Light Load First

9. Staged-DB IC-65 Advances in Data Management Systems 9 Detailed System Design Based on Discrete Event Simulation technique All the computation, data needs, dependencies are modeled using events System components –Global System Queue –Dispatcher –Operator (or)  Engine –Global Scheduler –Main Memory –Overlap Detector

10. Staged-DB IC-65 Advances in Data Management Systems 10 Query Arrival Dispatcher Scheduler Disk-Fetch Engine Insert Engine Exec-Begin Engine Exec-End Memory Global System Queue event eventId componentId functionId firingTime packet

11. Staged-DB IC-65 Advances in Data Management Systems 11  Engine Engine Insert Engine Execution Begin Engine Execution End Input Packet Queue Packet format queryId list queryPlans pageId contextInfo Request packet from parent node/ dispatcher Call Overlap detector Insert packet Pick packet from Q Send packet to Child OR execute and produce output Insert event into Event queue for the scheduler

12. Staged-DB IC-65 Advances in Data Management Systems 12  Engines Join Sort Aggregation Scan Wait and Scan Index Scan

13. Staged-DB IC-65 Advances in Data Management Systems 13 Overlap detection With memory With input queue Two types –Linear –Spike

14. Staged-DB IC-65 Advances in Data Management Systems 14 Memory Manager Pinning and unpinning Put() pageExists() consumePage()

15. Staged-DB IC-65 Advances in Data Management Systems 15 Performance study 5 queries 5 runs Uniform arrival rate

16. Effect of Overlapping Response time: time needed to produce the first page as output Big advantage for the overlapping case ('1')

17. Effect of Overlapping Memory consumption: max # of pages consumed in memory during the life time of the query Higher memory consumption with Overlapping !

18. Effect of Overlapping Throughput: # of queries completed in a unit of time Clear advantage with Overlap detection !

19. Comparing scheduling policies Mean response time Round Robin seems to perform a little better

20. Comparing scheduling policies Memory consumption No differences !

21. Staged-DB IC-65 Advances in Data Management Systems 21 Future Work Few more interesting global scheduling policies are possible. The system did not consider a local scheduling policy to pick one packet among many in the input packet queue, for processing next. It picks the fist packet in the queue at the moment. Regarding implementation, experimentation should be done with more  Engines and a bench mark style input queries.