1. Lecture 10 Query Optimization II Automatic Database Design

2. Recap: Query Planning Use analytical cost to estimate time needed for a query execution plan tree Selectivity (fraction of tuples returned from input): – col = value: 1/ICARD– 1/nth of # of unique col values, 1/10 if no index – col > value: (value – max) / (max – min) or 1/3 – col1 = col2: 1/max(ICARD(c1), ICARD(c2)) or 1/10

3. Selinger Heuristics Push down all filters and projections Skip cross-joins Left-deep plans only Get from O(n!) to O(2^n) optimization time

4. Selinger Optimizer Algorithm algorithm: compute optimal way to generate every sub-join: size 1, size 2,... n (in that order) e.g. {A}, {B}, {C}, {AB}, {AC}, {BC}, {ABC} R  set of relations to join For i in {1...|R|}: for S in {all length i subsets of R}: optjoin(S) = a join (S-a), where a is the relation that minimizes: cost(optjoin(S-a)) + min. cost to join optjoin(S-a) to a + min. access cost for a Precomputed in previous iteration!

5. Selinger, as code R  set of relations to join For i in {1...|R|}: for S in {all length i subsets of R}: optcost s = ∞ optjoin S = ø for a in S: //a is a relation c = optcost s-a + min. cost to join optjoin s-a to a + min. access cost for a if c < optcost s optcost s = c optjoin s = optjoin s-a joined optimally w/ a This is the same algorithm as on the previous slide, written differently Pre-computed in previous iteration!

6. Example 4 Relations: ABCD (only consider NL join) Optjoin: A = best way to access A (e.g., sequential scan, or predicate pushdown into index...) B = " " " " B C = " " " " C D = " " " " D {A,B} = AB (or BA) {A,C} = AC (or CA) {B,C} = BC (or CB) {A,D} … {B,D} {C,D} R  set of relations to join For i in {1...|R|}: for S in {all length i subsets of R}: optjoin(S) = a join (S-a), where a is the relation that minimizes: cost(optjoin(S-a)) + min. cost to join (S-a) to a + min. access cost for a Optjoin

7. Example (con’t) Optjoin {A,B,C} = remove A: ({B,C})A remove B: ({A,C})B remove C: ({A,B})C {A,C,D} = … {A,B,D} = … {B,C,D} = … … {A,B,C,D} = remove A: ({B,C,D})A remove B: ({A,C,D})B remove C: ({A,B,D})C remove D: ({A,B,C})D R  set of relations to join For i in {1...|R|}: for S in {all length i subsets of R}: optjoin(S) = a join (S-a), where a is the relation that minimizes: cost(optjoin(S-a)) + min. cost to join (S-a) to a + min. access cost for a Optjoin

8. Complexity Number of subsets of set of size n = |power set of n| = 2 n (here, n is number of relations) How much work per subset? Have to iterate through each element of each subset, so this at most n n2 n complexity (vs n!) n=12  49K vs 479M R  set of relations to join For i in {1...|R|}: for S in {all length i subsets of R}: optjoin(S) = a join (S-a), where a is the relation that minimizes: cost(optjoin(S-a)) + min. cost to join (S-a) to a + min. access cost for a Optjoin

9. Interesting Orders Push down sorts when it is profitable – Merge joins usually faster than NLJ Another round of dynamic programming For k interesting orders, have complexity kn2 n

10. Study Break – Join Ordering For the query: SELECT * FROM A,B,C,D WHERE A.v = B.v and B.w = C.w and C.w = D.w; How many left-deep plans are possible? How many plans or subsets of plans do we evaluate using the opt algo? Which one(s) can we eliminate as cross products?

11. Automatic DB Design Key idea: optimize data layout for performance Make a well-known set of queries execute fast Use cost models to estimate utility of different designs

12. Materialized Views sales : (saleid, date, time, register, product, price,...) CREATE MATERIALIZED VIEW sales_by_date AS SELECT date, product, sum(price), count(*) AS quantity FROM sales GROUP BY date, product Key properties: Kept up to date as data is added Selected for use automatically by optimizer when appropriate

13. Conclusions Use dynamic programming to efficiently enumerate costs of different query plans – Start with one table and add more Physical db design is complicated! – Picking the right indexes and materialized views – Combinations of heuristics and what-if cost modeling needed – Designs may be adaptive to changing workloads