Presentation is loading. Please wait.

Presentation is loading. Please wait.

Testing adaptive workload management Harumi Kuno HP Labs Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener.

Similar presentations


Presentation on theme: "Testing adaptive workload management Harumi Kuno HP Labs Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener."— Presentation transcript:

1 Testing adaptive workload management Harumi Kuno HP Labs Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener

2 Compile-time Query Optimization Run-time Query Execution Query Plan Performance Actual conditions (work + resources) Expected conditions (work + resources) 2

3 Controlling resources for a complex (dynamic mixed) workload is hard Traditional solution is to isolate components, by partitioning work across hardware or time multiplexing. 3

4 Grand Challenge (1): Managing dynamic mixed workloads Ignore it Avoid problem through isolating systems or using time multiplexing Provide rich tools to be used with manual workload management Adaptive workload management 4

5 Dynamic mixed workloads are difficult to manage because resource contention … changes resource requirements Disk and memory usage needed to execute a sort changes with the amount of available memory Goetz Graefe, Harumi Kuno, Janet Wiener. Visualizing the Robustness of Query Execution. Proc. Conference on Innovative Data Systems Research (CIDR). January 4-7,

6 Dynamic mixed workloads are difficult to manage because resource contention … changes performance Throughput/MPL for a single homogenous workload varies with different cache hit rates 6 Janet L. Wiener, Harumi Kuno, Goetz Graefe. Benchmarking Query Execution Robustness. First TPC Technology Conference on Performance Evaluation and Benchmarking (http://www.tpc.org/tpctc2009), held in conjunction with VLDB 2009.

7 Dynamic mixed workloads are difficult to manage because resource contention … and is difficult to predict Throughput/MPL for OLTP queries changes as various report queries run Report 1 Report 3 Report 2 7 Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.

8 Can static workload management policies handle unreliable cost estimates? under-informed admission control and scheduling decisions 8 Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.

9 Can static workload management policies handle unobserved resource contention? monitored resource not the source of contention 9 Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

10 Can static workload management policies handle system overload? No. Challenge: Static policies have a hard time correcting overload situations. 10 Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

11 Adaptive Workload Management Hypothesis We can build a system that uses feedback to add a policy control loop. 11‹#› Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

12 Testing Adaptive Workload Management Functionality of management tools Configuration for a particular anticipated workload How it handles the unexpected. 12 Grand Challenge (2)

13 Underlying challenges Characterize workloads and service classes: queries, mega-queries (graphs of queries considered as a single unit), loads, continuous inserts, etc. Plus usual features: estimated arrival rates, execution times, resource usage, etc., together with SLOs. Map the components of the workload to service classes. Develop recommendations for policies for the query control loop and policy control loop. Consider elasticity (dynamic scale out of resources) and outage avoidance. Evaluate responses to the unexpected. 13‹#›

14 You’d like to make decisions are made based on expected performance, but…. expectedunexpected Resource Requirements (e.g., degree of skew) Resource Availability expected Performance predictable Performance harder to predict Runtime performance really, really hard to predict BI queries -> skew + complex queries Skew + complex queries -> unexpected amount of work. More work -> more resource usage. Multiple queries with unexpected resource usage -> unexpected resource availability. 14‹#›


Download ppt "Testing adaptive workload management Harumi Kuno HP Labs Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener."

Similar presentations


Ads by Google