1. CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: 845-4259 Email: chen@cse.tamu.edu 1 Notes #9

2. LQP Optimization with Size 2

3. Two techniques: 3

4. LQP Optimization with Size Two techniques: Estimating sizes of immediate relations For natural join: T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 4

5. LQP Optimization with Size Two techniques: Estimating sizes of immediate relations For natural join: T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} Consider different order of an operation (((R S) T) U) = (R U) (S T) 5

6. Consider: A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 We want to have a good LQP for A B C D 6

7. Left-deep join tree 7

8. 8 ? ? ? ?

9. Left-deep join tree (all 4! = 24 permutations) 9 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B

10. 10 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B

11. Left-deep join tree (all 4! = 24 permutations) 11 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B

12. Left-deep join tree (all 4!/2 = 12 permutations) 12 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B

13. Left-deep join tree (all 4!/2 = 12 permutations) 13 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B

14. Left-deep join tree 14 A B C D B D A C A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)}

15. Left-deep join tree 15 A B C D B D A C A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 5000 V(*, c) = 500

16. Left-deep join tree 16 A B C D B D A C A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 5000 V(*, c) = 500 10000

17. Left-deep join tree 17 A B C D B D A C A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 5000 V(*, c) = 500 10000 cost = 15000

18. Left-deep join tree 18 A B C D B D A C A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 5000 V(*, c) = 500 10000 1000000 V(*, a) = 50 V(*,b) = 100 50000 cost = 15000 cost = 1050000

19. Left-deep join tree (all 4!/2 = 12 permutations) 19 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B 60000 1050000 55000 1010000 15000 20000 1002000 12000 4000 11000 3000

20. Left-deep join tree (all 4!/2 = 12 permutations) 20 AB C D BA C D AB D C BA D C CA B D CA D B BC A D CB A D DA B C DA C B DB A C DB C A DC A B DC B A CB D A CD A B CD B A BC D A BD A C BD C A AC B D AC D B AD B C AD C B 60000 1050000 55000 1010000 15000 20000 1002000 12000 4000 11000 3000

21. Left-deep tree: general algorithm Input: n relations R 1, R 2, …, R n Output: the best left-deep join of R 1, R 2, …, R n 1.Construct a left-deep tree T of n leaves; 2.For each P of the permutations of the n relations R 1, R 2, …, R n Do assign the n relations to the leaves of T in order of P; evaluate the cost of the plan; 3. Pick the plan with the permutation that gives the minimum cost. 21

22. Dynamic Programming Consider all tree structures. 22

23. Dynamic Programming Consider all tree structures. Again consider A B C D Five tree structures: Each of (a)-(d) has 12 different assignments, and (e) has 3 different assignments. So totally there are 51 different ways to join the 4 relations. Too many when the number of relations is relatively large. 23 (a) (e) (d) (c)(b)

24. Dynamic Programming Consider 24 D D D D A A A A B B B B C C C C

25. Dynamic Programming Consider 25 D D D D A A A A B B B B C C C C

26. Dynamic Programming Consider We really only need to find the best way to join A B C, then join D with this best join. 26 D D D D A A A A B B B B C C C C

27. Dynamic Programming Consider We really only need to find the best way to join A B C, then join D with this best join. How do we find the best join of A B C? 27 D D D D A A A A B B B B C C C C

28. Dynamic Programming Consider We really only need to find the best way to join A B C, then join D with this best join. How do we find the best join of A B C? We consider all possible ways: (A B) C, (A C) B, (B C) A. 28 D D D D A A A A B B B B C C C C

29. Dynamic programming: general algorithm Input: n relations R 1, R 2, …, R n Output: the best join of R 1, R 2, …, R n 1.FOR each R i DO {cost(R i ) = 0; size(R i ) = 0}; 2.FOR each pair of R i and R j DO {cost(R i, R j ) = 0; compute size(R i R j )}; 3.FOR k = 3 TO n DO FOR any k relations S 1, S 2, …, S k of R 1, R 2, …, R n DO FOR each partition P = {(S i 1, …, S i j ), (S i j+1,…, S i k )} of S 1, S 2, …, S k DO cost(P) = cost(S i 1, …, S i j ) + size(S i 1 … S i j ) + cost(S i j+1, …, S i k ) + size(S i j+1 … S i k ); let cost(S 1, S 2, …, S k ) be the smallest cost(P) among the above partitions; computer size(S 1 S 2 … S k ) (and remember this partition P); 4. Return cost(R 1, R 2, …, R n ). 29

30. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 30 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0

31. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 31 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000

32. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 32 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D

33. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 33 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D

34. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 34 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D

35. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 35 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D

36. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 36 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D

37. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 37 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D 2000

38. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 38 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D 2000

39. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 39 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D 2000 1000 1000000 2000 1000

40. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 40 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, DA, C, DA, B, D CB B D B C C D D 2000 1000 1000000 2000 1000

41. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 41 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, CB, C, D cost = 1000 size = 2000 A, C, DA, B, D B C D

42. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 B C D

43. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 43 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D B C D

44. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 44 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} B C D

45. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 45 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 3000 B C D

46. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 46 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30006000 B C D

47. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 47 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30001100055000120006000200000012000 B C D

48. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 48 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30001100055000120006000200000012000 B C D

49. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 49 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30001100055000120006000200000012000 A B C D

50. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 50 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30001100055000120006000200000012000 A B C D

51. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 51 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30001100055000120006000200000012000 A B C D B C D

52. Dynamic Programming: Example A(a, b): T(A) = 1000, V(A, a) = 100, V(A, b) = 200 B(b, c): T(B) = 1000, V(B, b) = 100, V(B, c) = 500 C(c, d): T(C) = 1000, V(C, c) = 20, V(C, d) = 1000 D(d, a): T(D) = 1000, V(D, d) = 1000, V(D, a) = 50 T(R(X, y) S(y, Z)) = T(R)T(S)/max{V(R, y), V(S, y)} 52 A cost = 0 size = 0 D cost = 0 size = 0 C cost = 0 size = 0 B cost = 0 size = 0 A, B cost = 0 size = 5000 C, D cost = 0 size = 1000 B, D cost = 0 size = 1000000 B, C cost = 0 size = 2000 A, D cost = 0 size = 10000 A, C cost = 0 size = 1000000 A, B, C cost = 2000 size = 10000 B, C, D cost = 1000 size = 2000 A, C, D cost = 1000 size = 10000 A, B, D cost = 5000 size = 50000 A, B, C, D cost = 3000 A {B,C,D}DC B {A,C,D} {A,B,D} {A,B,C} {A,B} {C,D} {A,C} {B,D}{A,D} {B,C} 30001100055000120006000200000012000 A B C D

53. LQP Optimization with Size: Summary Estimating sizes of immediate relations Consider different order of an operation left-deep tree dynamic programming 53

54. Construction of Physical Query Plan

55. Input: an optimized LQP T, and a main memory constraint M × ∩ π σ σ σ G F E D C B A

56. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A)

57. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan

58. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; 3.Replacing each internal node v of T by a proper algorithm; × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan J2P J1P CJ I1P

59. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; 3.Replacing each internal node v of T by a proper algorithm; 4.For each edge e in T, decide if e should be “materialized”; × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan J2P J1P CJ I1P

60. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; 3.Replacing each internal node v of T by a proper algorithm; 4.For each edge e in T, decide if e should be “materialized”; × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan J2P J1P CJ I1P

61. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; 3.Replacing each internal node v of T by a proper algorithm; 4.For each edge e in T, decide if e should be “materialized”; 5.Cut all materialized edges; × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan J2P J1P CJ I1P

62. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; 3.Replacing each internal node v of T by a proper algorithm; 4.For each edge e in T, decide if e should be “materialized”; 5.Cut all materialized edges; 6.Each subtree is a call to the subroutine at the root of the subtree. The order of the calls follows the bottom-up order in the structure. × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan J2P J1P CJ I1P 1 2 3

63. Construction of Physical Query Plan Input: an optimized LQP T, and a main memory constraint M 1.Replacing each leaf R of T by “scan(R)”; 2.Combining the “scan’s” with other operations; 3.Replacing each internal node v of T by a proper algorithm; 4.For each edge e in T, decide if e should be “materialized”; 5.Cut all materialized edges; 6.Each subtree is a call to the subroutine at the root of the subtree. The order of the calls follows the bottom-up order in the structure. × ∩ π σ σ σ scan(G) scan(F) scan(E) scan(D) scan(C) scan(B) scan(A) index-scan J2P J1P CJ I1P 1 2 3 This produces an executable code for the input DB program

64. Physical Query Plan: Summary Replacing internal nodes of a LQP by proper algorithms; Deciding if a subroutine call should be pipelined or materialized; Many optimization techniques are involved here; In practice, heuristic optimization techniques are used to construct good physical query plans; The resulting physical query plan is an executable code.

65. secondary storage (disks) in tables (relations) database administrator DDL language database programmer DML (query) language DBMS file manager buffer manager main memory buffers index/file manager DML complier DDL complier query execution engine transaction manager concurrency control lock table logging & recovery graduate database

66. secondary storage (disks) in tables (relations) database administrator DDL language database programmer DML (query) language DBMS file manager buffer manager main memory buffers index/file manager DML complier DDL complier query execution engine transaction manager concurrency control lock table logging & recovery graduate database