1. CPSC-608 Database Systems
Fall 2018
Instructor: Jianer Chen
Office: HRBB 315C
Phone:
Notes #17

2. parse tree-lqp convertor
Query Optimization
An input database program P
Prepare a collection C of efficient algorithms for operations in relational algebra;
parser
View processing,
Semantic checking
parse tree
preprocessing
parse tree
parse tree-lqp convertor
logic query plan
push selections,
group joins
apply logic laws
logic query plan
reduce the size of
intermediate results
Optimization via logic and size
logic query plan
Lqp-pqp convertor
take care of issues in optimization and security.
physical query plan
choices of algorithms,
data structures, and
computational modes
Optimization via algorithms and cost
Machine executable code

3. parse tree-lqp convertor
Query Optimization
An input database program P
Prepare a collection C of efficient algorithms for operations in relational algebra;
parser
View processing,
Semantic checking
parse tree
preprocessing
parse tree
parse tree-lqp convertor
logic query plan
push selections,
group joins
apply logic laws
logic query plan
reduce the size of
intermediate results
Optimization via logic and size
logic query plan
Lqp-pqp convertor
take care of issues in optimization and security.
physical query plan
choices of algorithms,
data structures, and
computational modes
Optimization via algorithms and cost
Machine executable code

4. Improving logic plan via logic laws
Major Steps:
Move selections σ so they can be applied early (σ reduces table size);
Combine cross product × with selections σ to make natural joins and theta joins
( has more efficient algorithms);
3. group commutative and associative binary
operations (e.g., ∩, U, ) (for later opt.);
4. May consider other operations (e.g., π, δ, τ, γ) C

5. Improving logic plan via logic laws
Major Steps:
Move selections σ so they can be applied early (σ reduces table size);
Combine cross product × with selections σ to make natural joins and theta joins
( has more efficient algorithms);
3. group commutative and associative binary
operations (e.g., ∩, U, ) (for later opt.);
4. May consider other operations (e.g., π, δ, τ, γ) C

6. Improving logic plan via logic laws
σC(R⋃S) = σC(R)⋃σC(S)
σC(R∩S) = σC(R)∩σC(S)
= σC(R)∩S = R∩σC(S)
σC(RS) = σC(R)  σC(S) = σC(R)  S
╳, ⨝, ⨝D: σC can only be pushed to the arguments that have all attributes in C.
σC and D(R) = σC(σD(R)) = σD(σC(R))

7. Improving logic plan via logic laws
Major Steps:
Move selections σ so they can be applied early (σ reduces table size);
Combine cross product × with selections σ to make natural joins and theta joins
( has more efficient algorithms);
3. group commutative and associative binary
operations (e.g., ∩, U, ) (for later opt.);
4. May consider other operations (e.g., π, δ, τ, γ)
Question: Do we also push other unary
operations such as π, δ, τ, γ?
Answer: Yes/maybe, they are in general
less significant and should be done carefully.

8. Improving logic plan via logic laws
Major Steps:
Move selections σ so they can be applied early (σ reduces table size);
Combine cross product × with selections σ to make natural joins and theta joins
( has more efficient algorithms);
3. group commutative and associative binary
operations (e.g., ∩, U, ) (for later opt.);
4. May consider other operations (e.g., π, δ, τ, γ)
Question: Do we also push other unary
operations such as π, δ, τ, γ?
Answer: Yes/maybe, they are in general
less significant and should be done carefully.

9. Improving logic plan via logic laws
Dealing with projections π

10. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.

11. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:

12. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care: π
a,d
b>c
R(a,b)
S(c,d)

13. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care: π
a,d
b>c π a π d
b>c
R(a,b)
S(c,d)
R(a,b)
S(b,c)

14. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care: π
a,d ??
b>c π a π d
b>c
R(a,b)
S(c,d)
R(a,b)
S(b,c)

15. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.

16. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved”

17. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ × σC ⋈
R(A)

18. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ × σC
insert πL ⋈
R(A)

19. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ × σC ⋈ πL
R(A)

20. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ × σC ⋈ πL
the shared attributes
should not be in A-L
R(A)

21. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ × σC
the attributes in C
should not be in A-L ⋈ πL
the shared attributes
should not be in A-L
R(A)

22. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ × σC
no restrictions
the attributes in C
should not be in A-L ⋈ πL
the shared attributes
should not be in A-L
R(A)

23. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩ ×
all attributes are used σC
no restrictions
the attributes in C
should not be in A-L ⋈ πL
the shared attributes
should not be in A-L
R(A)

24. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩
final result,
all attributes are used ×
all attributes are used σC
no restrictions
the attributes in C
should not be in A-L ⋈ πL
the shared attributes
should not be in A-L
R(A)

25. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
Not “moved” × ∩
final result,
all attributes are used ×
all attributes are used
also include those
later-used attributes L’ σC
no restrictions
the attributes in C
should not be in A-L ⋈
πL, L’
the shared attributes
should not be in A-L
R(A)

26. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πL(σC(R))
Not “moved” R πL σC

27. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πL(σC(R)) = πL(σC(πL,L(C)(R)))
Not “moved” R πL σC πL σC
πL,L(P) R

28. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πL(σC(R)) = πL(σC(πL,L(C)(R)))
Not “moved” R πL σC πL σC
C may contain
attributes not in L
πL,L(P) R

29. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πL(σC(R)) = πL(σC(πL,L(C)(R)))
Not “moved”
attributes in
C but not in L R πL σC πL σC
πL,L(C) R

30. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πL(σC(R)) = πL(σC(πL,L(C)(R)))
Not “moved”
attributes in
C but not in L R πL σC πL σC
πL,L(C) R

31. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πL(σC(R)) = πL(σC(πL,L(C)(R)))
Not “moved”
attributes in
C but not in L R πL σC πL σC
“inserted”
instead of
“moved”
πL,L(C) R

32. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”

33. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D σC ⋈
R(A,A1,B)
S(B,D,D1)

34. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D σC ⋈
inserted πA
inserted πD
R(A,A1,B)
S(B,D,D1)

35. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D σC ⋈ πA πD
R(A,A1,B)
S(B,D,D1)

36. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D σC ⋈ πA πD
R(A,A1,B)
S(B,D,D1)

37. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D
final result is fine σC
C may use B, A1 ⋈
B is used πA πD
R(A,A1,B)
S(B,D,D1)

38. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D
final result is fine σC
C may use B, A1, D1 ⋈
B is used πA πD
R(A,A1,B)
S(B,D,D1)

39. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
Not “moved”
πA,D
final result is fine σC
C may use B, A1, D1 ⋈
B is used
πA,A’
πD,D’
A’ = B plus those in A1 that are used in C
D’ = B plus those in D1 that are used in C
R(A,A1,B)
S(B,D,D1)

40. Improving logic plan via logic laws
Dealing with projections π
In general reduce the size of tables, but not as significantly as selections. Extended projections may even increase table size.
Apply with care:
General rule: a projection can be added anywhere as long as the eliminated attributes neither are used by any operation above the projection nor appear (explicitly/implicitly) in the final expression.
πA,D(σC(R(A,A1,B)⋈S(B,D,D1)))
= πA,D(σC(πA,A’(R(A,A1,B)))⋈πD,D’(S(B,D,D1))))
Not “moved”
πA,D σC ⋈
πA,A’
πD,D’
A’ = B plus those in A1 that are used in C
D’ = B plus those in D1 that are used in C
R(A,A1,B)
S(B,D,D1)

41. Improving logic plan via logic laws
Dealing with duplicate elimination δ

42. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).

43. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).
δ can swap with selections: δ(σC(R)) = σC(δ(R))

44. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).
δ can swap with selections: δ(σC(R)) = σC(δ(R))
but not with projections: δ(πL(R)) ≠ πL(δ(R))

45. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).
δ can swap with selections: δ(σC(R)) = σC(δ(R))
but not with projections: δ(πL(R)) ≠ πL(δ(R))
3. δ can be pushed in set operations and bag-intersection:
δ(R ∪S S) = δ(R) ∪S δ(S), δ(R ∩S S) = δ(R) ∩S δ(S),
δ(R −S S) = δ(R) −S δ(S), δ(R ∩B S) = δ(R) ∩B δ(S)

46. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).
δ can swap with selections: δ(σC(R)) = σC(δ(R))
but not with projections: δ(πL(R)) ≠ πL(δ(R))
3. δ can be pushed in set operations and bag-intersection:
δ(R ∪S S) = δ(R) ∪S δ(S), δ(R ∩S S) = δ(R) ∩S δ(S),
δ(R −S S) = δ(R) −S δ(S), δ(R ∩B S) = δ(R) ∩B δ(S)

47. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).
δ can swap with selections: δ(σC(R)) = σC(δ(R))
but not with projections: δ(πL(R)) ≠ πL(δ(R))
3. δ can be pushed in set operations and bag-intersection:
δ(R ∪S S) = δ(R) ∪S δ(S), δ(R ∩S S) = δ(R) ∩S δ(S),
δ(R −S S) = δ(R) −S δ(S), δ(R ∩B S) = δ(R) ∩B δ(S)

48. Improving logic plan via logic laws
Dealing with duplicate elimination δ
δ can be pushed in join operations:
δ(R×S) = δ(R)×δ(S), δ(R⋈S) = δ(R)⋈δ(S),
δ(R ⋈C S) = δ(R) ⋈C δ(S).
δ can swap with selections: δ(σC(R)) = σC(δ(R))
but not with projections: δ(πL(R)) ≠ πL(δ(R))
3. δ can be pushed in set operations and bag-intersection:
δ(R ∪S S) = δ(R) ∪S δ(S), δ(R ∩S S) = δ(R) ∩S δ(S),
δ(R −S S) = δ(R) −S δ(S), δ(R ∩B S) = δ(R) ∩B δ(S)
δ cannot be pushed in other bag operations:
δ(R ∪B S) ≠ δ(R) ∪B δ(S), δ(R −B S) ≠ δ(R) −B δ(S)

49. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ

50. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.

51. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.

52. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.

53. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.
δ(γL,A(R)) = γL,A(R)

54. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.
δ(γL,A(R)) = γL,A(R)
If L and A are entirely contained in M, γL,A(R) = γL,A(πM(R))

55. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.
δ(γL,A(R)) = γL,A(R)
If L and A are entirely contained in M, γL,A(R) = γL,A(πM(R))
Sorting τ:

56. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.
δ(γL,A(R)) = γL,A(R)
If L and A are entirely contained in M, γL,A(R) = γL,A(πM(R))
Sorting τ:
Sorting unchanges table size, and is expensive

57. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.
δ(γL,A(R)) = γL,A(R)
If L and A are entirely contained in M, γL,A(R) = γL,A(πM(R))
Sorting τ:
Sorting unchanges table size, and is expensive
Can be extremely helpful for later computation

58. Improving logic plan via logic laws
Dealing with grouping and sorting γ, τ
More case dependent. Also need to consider earlier/later operations.
Grouping γL,A: group the tuples in terms of the values of the attributes
in L, and for each group, compute the aggregations in A.
Grouping reduces table size, but could be expensive if the table is unorganized.
δ(γL,A(R)) = γL,A(R)
If L and A are entirely contained in M, γL,A(R) = γL,A(πM(R))
Sorting τ:
Sorting unchanges table size, and is expensive
Can be extremely helpful for later computation
Given for free if an index such as B+ tree is used.

59. Improving logic plan via logic laws
General Remarks on LQP Optimization
No transformation is always good;
Except pushing selections down is usually
always good