1 Ullman et al. : Database System Principles Notes 6: Query Processing

2 Query Processing Q  Query Plan Focus: Relational System Others?

3 Example Select B,D From R,S Where R.A = “c”  S.E = 2  R.C=S.C

4 RABC S CDE a11010x2 b12020y2 c21030z2 d23540x1 e34550y3 AnswerB D 2 x

5 How do we execute query? - Do Cartesian product - Select tuples - Do projection One idea

6 RXSR.AR.BR.CS.CS.DS.E a x 2 a y 2. C x 2.

7 RXSR.AR.BR.CS.CS.DS.E a x 2 a y 2. C x 2. Bingo! Got one...

8 Relational Algebra - can be used to describe plans... Ex: Plan I  B,D  R.A =“c”  S.E=2  R.C=S.C  X RS

9 Relational Algebra - can be used to describe plans... Ex: Plan I  B,D  R.A =“c”  S.E=2  R.C=S.C  X RS OR:  B,D [  R.A=“c”  S.E=2  R.C = S.C (RXS)]

10 Another idea:  B,D  R.A = “c”  S.E = 2 R S Plan II natural join

11 R S A B C  ( R )  ( S ) C D E a 1 10 A B C C D E 10 x 2 b 1 20c x 2 20 y 2 c y 2 30 z 2 d z 2 40 x 1 e y 3

12 Plan III Use R.A and S.C Indexes (1) Use R.A index to select R tuples with R.A = “c” (2) For each R.C value found, use S.C index to find matching tuples (3) Eliminate S tuples S.E  2 (4) Join matching R,S tuples, project B,D attributes and place in result

13 R S A B C C D E a x 2 b y 2 c z 2 d x 1 e y 3 AC I1I1 I2I2 =“c”

14 R S A B C C D E a x 2 b y 2 c z 2 d x 1 e y 3 AC I1I1 I2I2 =“c”

15 R S A B C C D E a x 2 b y 2 c z 2 d x 1 e y 3 AC I1I1 I2I2 =“c” check=2? output:

16 Overview of Query Optimization

17 parse convert apply laws estimate result sizes consider physical plans estimate costs pick best execute {P1,P2,…..} {(P1,C1),(P2,C2)...} Pi answer SQL query parse tree logical query plan “improved” l.q.p l.q.p. +sizes statistics

18 Example: SQL query SELECT title FROM StarsIn WHERE starName IN ( SELECT name FROM MovieStar WHERE birthdate LIKE ‘%1960’ ); (Find the movies with stars born in 1960)

19 Example: Parse Tree SELECT FROM WHERE IN title StarsIn ( ) starName SELECT FROM WHERE LIKE name MovieStar birthDate ‘%1960’

20 Example: Generating Relational Algebra  title  StarsIn IN  name  birthdate LIKE ‘%1960’ starName MovieStar Fig. 7.15: An expression using a two-argument , midway between a parse tree and relational algebra

21 Example: Logical Query Plan  title  starName=name StarsIn  name  birthdate LIKE ‘%1960’ MovieStar Fig. 7.18: Applying the rule for IN conditions 

22 Example: Improved Logical Query Plan  title starName=name StarsIn  name  birthdate LIKE ‘%1960’ MovieStar Fig. 7.20: An improvement on fig Question: Push project to StarsIn?

23 Example: Estimate Result Sizes Need expected size StarsIn MovieStar 

24 Example: One Physical Plan Parameters: join order, memory size, project attributes,... Hash join SEQ scanindex scan Parameters: Select Condition,... StarsInMovieStar

25 Example: Estimate costs L.Q.P P1 P2 …. Pn C1 C2 …. Cn Pick best!

26 Query Optimization Relational algebra level Detailed query plan level –Estimate Costs without indexes with indexes –Generate and compare plans

27 Relational algebra optimization Transformation rules (preserve equivalence) What are good transformations?

28 Rules: Natural joins & cross products & union R S=SR (R S) T= R (S T)

29 Note: Carry attribute names in results, so order is not important Can also write as trees, e.g.: T R R SS T

30 R x S = S x R (R x S) x T = R x (S x T) R U S = S U R R U (S U T) = (R U S) U T Rules: Natural joins & cross products & union R S=SR (R S) T= R (S T)

31 Rules: Selects  p1  p2 (R) =  p1vp2 (R) =  p1 [  p2 (R)] [  p1 (R)] U [  p2 (R)]

32 Rules: Project Let: X = set of attributes Y = set of attributes XY = X U Y  xy (R) =  x [  y (R)]

33 Let p = predicate with only R attribs q = predicate with only S attribs m = predicate with only R,S attribs  p (R S) =  q (R S) = Rules:  combined [  p (R)] S R [  q (S)]

34 Some Rules can be Derived:  p  q (R S) =  p  q  m (R S) =  pvq (R S) = Rules:  combined (continued)

35 Rules:  combined Let x = subset of R attributes z = attributes in predicate P (subset of R attributes)  x [  p ( R ) ] =  {  p [  x ( R ) ] } x x  xz

36 Rules:  combined Let x = subset of R attributes y = subset of S attributes z = intersection of R,S attributes  xy (R S) =  xy { [  xz ( R ) ] [  yz ( S ) ] }

37  xy {  p (R S) } =  xy {  p [  xz’ (R)  yz’ (S)] } z’ = z U { attributes used in P }

38 Rules for    combined with X similar... e.g.,  p (R X S) = ?

39  p (R U S) =  p (R) U  p (S)  p (R - S) =  p (R) - S =  p (R) -  p (S) Rules   U  combined:

40  p1  p2 (R)   p1 [  p2 (R)]  p (R S)  [  p (R)] S R S  S R  x [  p (R)]   x {  p [  xz (R)] } Which are “good” transformations?

41 Conventional wisdom: do projects early Example: R(A,B,C,D,E) x={E} P: (A=3)  (B=“cat”)  x {  p (R)} vs.  E {  p {  ABE (R)} } Usually good: early selections

42 What if we have A, B indexes? B = “cat” A=3 Intersect pointers to get pointers to matching tuples But

43 Outline - Query Processing Relational algebra level –transformations –good transformations Detailed query plan level –estimate costs –generate and compare plans

44 Estimating cost of query plan (1) Estimating size of results (2) Estimating # of IOs