CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites) Shivnath Babu

parse Query rewriting Physical plan generation execute result SQL query parse tree logical query planstatistics physical query plan Query Processing - In class order 2; ; 16.2,16.3 1; 13, 15 4; 16.4—16.7

Roadmap Query optimization: problem definition Space of physical plans –Counting exercise Approaches for query optimization –Heuristic-based (Oracle calls them rule-based) –Cost-based –Hybrid Heuristics for query optimization (Query rewrites)

Query Optimization Problem Pick the best plan from the space of physical plans

The Space of Physical Plans is Very Large Algebraic equivalences Different physical operators for the same logical operator –nested loop join, hash join, sort-merge join –index-scan, table-scan Different plumbing details - pipelining vs. materialization Different tree shapes

A Plan Counting Exercise Work on blackboard

Approaches for Query Optimization Approach 1: Pick some plan –Bad plans can be really bad! Approach 2: Heuristics –Ex: maximize use of indexes (MySQL) Approach 3: Cost-based –“Enumerate”, find cost, pick best –Be smart about how you iterate through the plans. Why? Hybrid

Query Optimization in Practice Hybrid Use heuristics, called query rewrite rules –eliminate many of the really bad plans –avoid eliminating good plans Cost-based –Be smart about how you iterate through plans –Ex: dynamic programming, genetic search

parse Query rewriting Physical plan generation execute result SQL query parse tree logical query plan statistics physical query plan Initial logical plan “Best” logical plan Logical plan Rewrite rules

Why do we need Query Rewriting? Pruning the HUGE space of physical plans –Eliminating redundant conditions/operators –Rules that will improve performance with very high probability Preprocessing –Getting queries into a form that we know how to handle best  Reduces optimization time drastically without noticeably affecting quality

Query Rewrite Rules Transform one logical plan into another –Do not use statistics Equivalences in relational algebra Push-down predicates Do projects early Avoid cross-products if possible Use left-deep trees Use of constraints, e.g., uniqueness

Example Query Select B,D From R,S Where R.A = “c”  R.C=S.C

Example: Parse Tree SELECT FROM WHERE AND B R S R.A=“c” R.CS.C= D Select B,D From R,S Where R.A = “c”  R.C=S.C

Along with Parsing … Semantic checks –Do the projected attributes exist in the relations in the From clause? –Ambiguous attributes? –Type checking, ex: R.A > 17.5 Expand views

Initial Logical Plan Relational Algebra :  B,D [  R.A=“c”  R.C = S.C (RXS)] Select B,D From R,S Where R.A = “c”  R.C=S.C  B,D  R.A = “c” Λ R.C = S.C X RS

Apply Rewrite Rule (1)  B,D [  R.C=S.C [  R.A=“c” (R X S)]]  B,D  R.A = “c” Λ R.C = S.C X RS  B,D  R.A = “c” X RS  R.C = S.C

Apply Rewrite Rule (2)  B,D [  R.C=S.C [  R.A=“c” (R)] X S]  B,D  R.A = “c” X R S  R.C = S.C  B,D  R.A = “c” X RS  R.C = S.C

Apply Rewrite Rule (3)  B,D [[  R.A=“c” (R)] S]  B,D  R.A = “c” R S  B,D  R.A = “c” X R S  R.C = S.C Natural join

Equivalences in Relational Algebra R S =SR Commutativity (R S) T = R(S T) Associativity Also holds for: Cross Products, Union, Intersection R x S = S x R (R x S) x T = R x (S x T) R U S = S U R R U (S U T) = (R U S) U T

Rules: Project Let: X = set of attributes Y = set of attributes XY = X U Y  xy (R) =  x [  y (R)]

Let p = predicate with only R attribs q = predicate with only S attribs m = predicate with only R,S attribs  p (R S) =  q (R S) = Rules:  combined [  p (R)] S R [  q (S)]

Rules:  combined (continued)  p  q (R S) = [  p (R)] [  q (S)]  p  q  m (R S) =  m [ (  p R) (  q S) ]  pvq (R S) = [ (  p R) S ] U [ R (  q S) ]

 p1  p2 (R)   p1 [  p2 (R)]  p (R S)  [  p (R)] S R S  S R  x [  p (R)]   x {  p [  xz (R)] } Which are “good” transformations?

Conventional wisdom: do projects early Example: R(A,B,C,D,E) x={E} P: (A=3)  (B=“cat”)  x {  p (R)} vs.  E {  p {  ABE (R)} }

But: What if we have A, B indexes? B = “cat” A=3 Intersect pointers to get pointers to matching tuples

Bottom line: No transformation is always good Some are usually good: –Push selections down –Avoid cross-products if possible –Subqueries  Joins

More Query Rewrite Rules Transform one logical plan into another –Do not use statistics Equivalences in relational algebra Push-down predicates Do projects early Avoid cross-products if possible Use left-deep trees Subqueries  Joins Use of constraints, e.g., uniqueness

Avoid Cross Products (if possible) Which join trees avoid cross-products? If you can't avoid cross products, perform them as late as possible Select B,D From R,S,T,U Where R.A = S.B  R.C=T.C  R.D = U.D

Use Left Deep Plans What are some left-deep, right-deep, and bushy plans for this query? Why is this heuristic useful? –Reason #1: We maximize the possibility of using indexes –Reason #2: Better for nested-loop join What about hash joins? Homework: Construct examples where (i) right-deep plan is best, (ii) where bushy is best Select B,D From R,S,T,U Where R.A = S.A  R.A=T.A  R.A = U.A

More Query Rewrite Rules Transform one logical plan into another –Do not use statistics Equivalences in relational algebra Push-down predicates Do projects early Avoid cross-products if possible Use left-deep trees Subqueries  Joins Use of constraints, e.g., uniqueness

SQL Query with an Uncorrelated Subquery Find the movies with stars born in 1960 MovieStar(name, address, gender, birthdate) StarsIn(title, year, starName) SELECT title FROM StarsIn WHERE starName IN ( SELECT name FROM MovieStar WHERE birthdate LIKE ‘%1960’ );

Parse Tree SELECT FROM WHERE IN title StarsIn ( ) starName SELECT FROM WHERE LIKE name MovieStar birthDate ‘%1960’

Generating Relational Algebra  title  StarsIn IN  name  birthdate LIKE ‘%1960’ starName MovieStar Two-argument selection

Rewrite Rule for Two-argument Selection with Conditions Involving IN  Lexp IN Rexp Two-argument selection  Lexp Rexp δ X

Applying the Rewrite Rule  title  StarsIn IN  name  birthdate LIKE ‘%1960’ starName MovieStar  title  starName=name StarsIn δ  birthdate LIKE ‘%1960’ MovieStar   name

Improving the Logical Query Plan  title starName=name StarsIn  name  birthdate LIKE ‘%1960’ MovieStar  title  starName=name StarsIn δ  birthdate LIKE ‘%1960’ MovieStar   name

SQL Query with an Correlated Subquery MovieStar(name, address, gender, birthdate) StarsIn(title, year, starName) SELECT title FROM StarsIn WHERE starName IN ( SELECT name FROM MovieStar WHERE name LIKE ‘Tom%’ and year = birthdate + 30 );

parse Query rewriting Physical plan generation execute result SQL query parse tree logical query planstatistics physical query plan Query Processing - In class order 2; ; 16.2,16.3 1; 13, 15 4; 16.4—16.7