1. Relational Database Systems 3
Instructor: Prof. James Cheng
Acknowledgement: The slides are extracted from and modified based on the slides provided by Prof. Sourav S. Bhowmick from Nanyang Technological University.

2. Topics to be covered ER model Relational Algebra SQL
Storage and Index Structures
Query Processing and Query Optimization

3. An Example Query Find the movies with stars born in 1960
MovieStar (name, birthdate, ...)
StarsIn (starName, title)
Movie (title, year, language, ... )
SELECT title
FROM StarsIn
WHERE starName IN (
SELECT name
FROM MovieStar
WHERE birthdate LIKE .%1960 );

4. Parse Tree

5. Relational Algebraic Representation
SELECT title
FROM StarsIn
WHERE starName IN (
SELECT name
FROM MovieStar
WHERE birthdate LIKE .%1960);

6. Select logical query plan Select physical query plan
Query Compilation
SQL
Parse tree construction
Parse Query
Query expression tree
Create algebraic representation
Select logical query plan
Select algorithms to implement each operator.
Select order of execution
Logical query plan tree
Select physical query plan
Physical query plan tree
Execute plan

7. Cost Model and Factors Affecting Cost
Measure of cost
No. of disk I/Os
Main memory
Portion of main memory that the operator uses
Main memory divided into buffers
Size of buffer = size of disk blocks
M = No. of main-memory buffers available
Size of the arguments
B(R) = Number of blocks needed to hold tuples of R
T(R) = Number of tuples in R
V(R, a) = Number of distinct values of attribute a in R

8. Natural Join [R(X,Y) S(Y,Z)]
Seen
Before?
Yes!
BST/HT
Search key is Y
No!
Output
buffer
M-1 buffer
Mth buffer
Cost
Num of disk I/O = B(R) + B(S)
Space requirement R S
min(B(R), B(S)) <= M-1

9. Nested Loop Join Why “one-and-a-half”? Nested-loop join
One of the two arguments has its tuples read only once
Other argument is read repeatedly
Nested-loop join
Can be used for relations of any size
Not necessary that one relation fit in main memory

10. Tuple-Based Nested-Loop Join
R(X,Y) S(Y,Z)]
FOR each tuple r in R DO
FOR each tuple s in S DO
IF r and s join to make a tuple t THEN
Output t
Cost
Example
For each tuple in R scan the entire relation S
T(R)*T(S) disk I/Os
T(R)*B(S) + B(R) disk I/Os (when S is clustered)
T(R)*T(S) + B(R) disk I/Os (when S is unclustered)
T(R) = 10000
B(R) = 1000
B(S) = 500
No. Of I/Os =
Assume 100 I/Os per sec
Approx 14 hours!

11. Block-based Nested-Loop Join
Principle
Organizing access to both argument relations by blocks
Use as much main memory as we can store for tuples of the outer relation R

12. Page-At-A-Time NL Join
R(X,Y) S(Y,Z)]
FOR each block br in R DO
FOR each block bs in S DO
IF tuples r and s join -> make a tuple t THEN
Output t
Cost
Example
B(R) + B(R)*B(S) [S is clustered]
B(R) + B(R)*T(S) [S is unclustered]
B(R) = 1000, B(S) = 500
*1000 = 500,500 (S outer)
*500 = 501,000 (R outer)
Approx 1.4 hrs!

13. Example [R(X,Y) S(Y,Z)] S R Input Buffer of S Output Input buffer
Buffer of R
… and so on S R

14. The Idea M R S

15. Block-Based Nested-Loop Join
What is “Block” in NL Join?
Group of (M-1) pages of R is called a block
R(X,Y) S(Y,Z)]
UNTIL all of R blocks been read DO
READ in M-1 blocks of R
FOR each block of S DO
READ in the single S block
Check pairs of tuples in memory and output if they match

16. Example R S R S Input buffer for R … and so on Input buffer for s
Output buffer

17. Cost Analysis R(X,Y) S(Y,Z)] UNTIL all of R blocks been read DO
READ in M-1 blocks of R
FOR each block of S DO
READ in the single S block
Check pairs of tuples in memory and output if they match
Cost
Each iteration: M-1 block of R and B(S) blocks of S
Number of iterations = B(R)/(M-1)
Total number of disk I/O = (M-1 + B(S)) x B(R)/(M-1)
= B(R) + B(S)B(R)/(M-1)

18. Block-Based Nested-Loop Join
Example
B(R) = 10000, B(S) = 500, M = 51
*10000/50 = 100,500 (S outer)
*500/50 = 110,000 (R outer)
Principle
Use smaller relation as outer relation

19. Summary Algorithm Cost Example Tuple-based NL Join T(R)*B(S) + B(R)
14 hrs
Simple Block-based NL Join
B(S) + B(R)*B(S)
1.4 hrs
Block-based NL Join
B(S) + B(S)B(R)/(M-1)
3 min

20. Two-Pass Algorithms Principle
Data from the operand relations are read into main memory
Processed
Written out to disk
Reread from disk to complete operation

21. Types of Two-Pass Algorithms
Sort-Based Algorithms
Partition their argument(s) into main-memory-sized, sorted sublists
The sorted sublists are merged appropriately to produce desired results
Hash-based Algorithms
Use a hash function to partition the argument(s) into buckets
Operation is applied to buckets individually or a pair of buckets
Index-based Algorithms
Use index to speed up selection on indexed attributes

22. Sorting-based Algorithms
Problem
Sorting is used in implementing many relational operations
Relations are typically large, do not fit in main memory
So cannot use traditional in-memory sorting algorithms
Approach
Combine in-memory sorting with clever techniques aimed at minimizing I/O
I/O costs dominate => cost of sorting algorithm is measured in the number of page transfers

23. Two-Way Merge Sort g a d c b e r m p g a e c b d r m p g a d c b e r m24 a 19 d 31 c 33 b 14 e 16 r 21 m 3 p 2 7
Runs
Create
runs g 24 a 19 e 16 c 33 b 14 d 31 r 21 m 3 p 2 7
Merge g 24 a 19 d 31 c 33 b 14 e 16 r 21 m 3 p 2 7 g 24 a 19 d 31 c 33 b 14 e 16 r 21 m 3 p 2 7
Sorted output
Merge R R R W W W
Initial relation

24. 3 Buffers (2 for reading), B(R)=416 r g 24 d 31 g 24 a 19 d 31 c 33 a 19 c 33 b 14 d 21 a 19 g 24 a 19 g 24 a 19 d 31 c 33 b 14 e 16 r 21
Block b 14 c 33 d 31 c 33 d 31 d 21 g 24 b 14 e 16 b 14 e 16 r d 21 e 16 r 16 d 21 r 16
Initial relation
Create
run
Merge
run
Sorted
output R R R W W W
Number of passes = log2B + 1
Cost = 2B(log2B + 1)

25. Two-Phase Multiway Merge-Sort
Motivation
Computation involved in sorting records in buffers in main memory
I/O necessary to move records between disk and main memory
Goal: Minimize number of runs and number of passes
Phase 1
Phase 2
Repeatedly fill the M buffers with new tuples from R and sort them using main-memory sorting algorithm
Write out each sorted sublist to disk
Merge the sorted sublist .
Requirement
At most M-1 sorted sublist
One input block to each sorted sublist
One block to the output

26. Phase 2
Buffer
M pages

27. Algorithm for Merge Step 1 Step 2 Step 4 Step 3 Step 5
Find the smallest key among the first
remaining elements of all the lists
(linear search or use priority queue)
Move the smallest element to the
first available position of the
output block
Step 4
Step 3
If the block from which the smallest
element was just taken is now
exhausted of records
Read the next block from the same
sorted sublist into thesame buffer
If the output block is full, write it
to disk and reinitialize the same
buffer in main memory to hold
the next output block
Step 5
If no blocks remain, then leave its buffer empty and do not consider
elements from that list

28. Example Input runs Output run Input buffers Output buffer 2 3 5 6 2 5 2 5 3 6 10 3 6 1 2 5 15 7 10 1 15 7
Input buffers
Output buffer

29. Cost Analysis Bounds on number of blocks Cost
There must be no more than M-1 sublists
Each sublist = M blocks
Number of sublists = B/M
B/M <= M-1
=> B <= M2
Cost
Reads B blocks in the first pass
B disk I/O to write sorted sublists
Sorted sublists are read again in the second pass
Total: 3B disk I/O

30. What if R is too big? Multi-Phase, Multiway Merge Sort
Apply the idea recursively
Divide R into chunks of size M(M-1)
Use 2Phase-Multiway Merge Sort to sort each one
Take resulting sorted lists as input for a third (merge) phase

31. Duplicate Elimination
Importance
A major step in computing projection, union, and difference relational operators
Algorithm
Sort
At the last stage of the merge step repeatedly select the first unconsidered tuple t among all the sorted sublists
Write one copy to the output.
Eliminate from the input blocks all occurrences of t
No additional cost (with respect to sorting) in terms of I/O
Total: 3B disk I/O

32. Duplicate elimination During Merge
Input runs
Last key used
Output run 5 2 6 3 15 2 3 6 1 5 3 6 1 5 15 2
5 15 1 5 15 3
Key 3 ignored: duplicate
Key 5 ignored: duplicate
Input buffers
Output buffer

33. Basic Idea of Sort-based Join
Principle
Sort both relations on the join attributes
Groups all tuples with the same value in the join col. together (partitions)
Look for qualifying tuples by merging the two relations
Compare R tuples in a partition with only the S tuples in the same partition
Challenge
Number of tuples share a common join attribute(s) can exceed what fits in memory

34. Sort-Based Join Algorithms
Sort-Merge Join
Two relations are sorted first and then merged in distinct pass
Makes the greatest possible number of buffers available for joining tuples with common value.
Refined Sort-Merge Join
Combine merging phase of sort with the merging phase of the join
Problems when there are large number of tuples with common join attribute value.

35. Sort-Merge Join [R(X,Y) S(Y,Z)]
Step 1
Step 2
Sort R and S with Y as the sort key
Merge the sorted R and S.
Use two buffers: the current block of R and the current block of S
Find the least value y of the join attributes Y that is currently at the front of the blocks of R and S
Step 3
If y does not appear at the front
of the other relation, then remove
the tuple(s) with sort key y
Step 5
Step 4
Output all the tuples that can be formed by joining over y.
Otherwise, identify all the tuples from both relations having sort key y.
Read blocks if necessary (M buffer available)
Step 6
If either relation has no more unconsidered tuples in memory, reload the buffer for that relation

36. Overview R S R(X,Y) S(Y,Z)] sort R on Y; sort S on Y;
while !eof(R) and !eof(S) do {
Scan R and S concurrently until tR.Y=tS.Y=c;
Output Y=c(R)Y=c (S) }
Y=c(R) R  S
Y=c (S)

37. Example R S R S X Y Y Z 1 3 p p 0 9 q q 8 7 3 s s s 5 7 u u 1 1 v v
1 3
p p
0 9
q q
s s s
5 7
u u
1 1
v v
p p p p
s s s
u u u u u u Y Z
p p
4 0 r 9 s 7
t t
2 5
u u u x
R S S

38. Cost Analysis How big M needs to be for it to work? Cost
We need to be able to perform 2Phase Multiway Merge Sort
B(R) <= M2 and B(S) <= M2
All tuples with common Y-value must fit in M buffer
Cost
Sorting: four disk I/Os per block (two in each phase)
Merging: Read each block of R and S one time
Single scan is guaranteed if at least one of the relation has no duplicates in the join attribute
Total: 5(B(R) + B(S))

39. Refinement of Sort-Merge Join
3(B(R) + B(S)) disk I/O
B(R) + B(S) <= M2
Idea
Combine the merging phase of sorting with the merging phase of the join
Step 1
Step 3
Repeatedly find the least Y-value y among the first available tuples of all the sublists
Identify all tuples in R and S that have y
Output the join of all tuples of R with all of S with common Y-value
If the buffer for one of the sublist is exhausted, refill
Create sorted sublists of size M, using
Y as the sort key, for both R and S
Step 2
Bring the first block of each sublist into a buffer
Assume no more than M sublists

40. Example Sort-Merge Join B(R) = 1000, B(S) = 500, M = 101
7500 disk I/O (Sort-Merge)
4500 disk I/O (Enhanced)
Approx 1 min!

41. Summary Algorithm Cost I/Os Time Tuple-based NL Join T(R)*B(S) + B(R)
14 hrs
Simple Block-based NL Join
B(S) + B(R)*B(S)
500,500
1.4 hrs
Block-based NL Join*
B(S) + B(S)B(R)/(M-1)
5,500
1 min
Sort-Merge Join**
3(B(R) + B(S))
4,500
** M = 101

42. Hashing-based Two Pass Algorithms
The idea
If the data is too big to store in main-memory buffers, hash all the tuples of the argument(s) using an appropriate hash key
How it works?
Select the hash key so that all tuples that need to be considered together falls into the same bucket
Operate on a bucket (or a pair of buckets) at a time
Size of the relations are reduced by a factor equal to number of buckets (generally M)

43. Partitioning Relation R into M-1 buckets
Initialize M-1 buckets using M-1 empty buffers;
FOR each block b of R DO BEGIN
read b into the Mth buffer;
FOR each tuple t in b DO BEGIN
IF the buffer for bucket h(t) has no room for t THEN
BEGIN
copy the buffer to disk;
empty the buffer;
END
copy t to the buffer for bucket h(t);
END;
FOR each bucket DO
IF the buffer for this bucket is not empty THEN
write the buffer to disk;

44. Partitioning of Relations
Compute the hash value of a whole tuple
Same tuples are hashed to the same bucket
Relation R
Buckets
OUTPUT 1 1 R1
hash
function h 2
INPUT 2 R2
. . .
M-1
B(R)
RM-1
Disk
Disk
M main memory buffers
Each bucket has size approx. B(R)/M-1

45. Partition Hash Join Step 1 Step 2 Step 3
Hash R to M-1 buckets using the join attribute(s) as hash key
Send all buckets to disk
Step 2
Hash S to M-1 buckets using the join attribute(s) as hash key
Send all buckets to disk
Step 3
Join every pair of buckets using one-pass algorithm

46. Steps 1 and 2
Relation R
Buckets
OUTPUT 1 1 R1
hash
function h 2
INPUT 2 R2
. . .
M-1
B(R)
RM-1
Disk
Disk
M main memory buffers
Tuples in partition Ri only match tuples in partition Si.
OUTPUT 1 1 S1
hash
function h 2
INPUT 2 S2
. . .
M-1
B(S)
SM-1
Disk
Disk
M main memory buffers

47. Step 3
Steps
Read in a partition of Si, hash it using another hash function h’
Scan matching partition of R, hash each tuple in the partition using h’, and search for matching tuples of Si in the same bucket S1
hash
function h’ R1 S2 R2
Join
RM-1
SM-1
M-1 blocks
Buckets
for S
Buckets
for R
Main Memory Buffers

48. Requirements Requirements
Each Si needs to fit into (M-1) blocks or buffers in memory
S has (M-1) buckets
B(S) <= (M-1)(M-1)
Min(B(R), B(S)) <= M2 S1
hash
function h’ R1 S2 R2
Join
RM-1
SM-1
M-1 blocks
Buckets
for S
Buckets
for R
Main Memory Buffers

49. Cost of Partition Hash Join
Step 1
Hash R to M-1 buckets using the join attribute(s) as hash key
Send all buckets to disk
=> (B(R) + B(R)) disk I/Os
read + write
Step 2
Hash S to M-1 buckets using the join attribute(s) as hash key
Send all buckets to disk
=> (B(S) + B(S)) disk I/Os
read + write
Step 3
Join every pair of buckets using one-pass algorithm
=> (B(R) + B(S)) disk I/Os: read from disk
Cost
3(B(R) + B(S)) disk I/Os

50. Hash Join Tricks Question Solution
What if a partition is too large to fit into memory?
Solution
Repeat Step 1 and/or Step 2 recursively

51. Example Algorithm Cost I/Os Time Tuple-based NL Join T(R)*B(S) + B(R)
14 hrs
Simple Block-based NL Join
B(S) + B(R)*B(S)
500,500
1.4 hrs
Block-based NL Join*
B(S) + B(S)B(R)/(M-1)
15,651
3 min
Sort-Merge Join**
3(B(R) + B(S))
4,500
1 min
Partition Hash Join
* M = 34
** M = 101

52. Hybrid Hash Join h What if there is extra memory available?
B(S) << M2
Use it to avoid writing/re-reading partitions
Both R and S
hash
function h
INPUT
Disk
M main memory buffers

53. Hybrid Hash Join Algorithm
Step 1
Step 2
Partition R into k buckets
First t buckets joined immediately with S
Other k-t buckets to disk
Partition S into k buckets
But keep t buckets (S1 , S2 ,…, St) in memory, k-t buckets to disk
Step 3
Finally join k-t pairs of bucket
(Rt+1, St+1), (Rt+2, St+2),…, (Rk, Sk)
hash
function h
INPUT
Disk
M main memory buffers

54. How to choose k and t? h Choosing k and t
Average bucket size for S is B(S)/k
Need to fit (B(S)/k)*t + (k-t) blocks in memory
B(S)*t/k + (k-t) <= M
Assume B(S)*t/k >> (k-t)
t/k = M/B(S)
hash
function h
INPUT

55. Cost of Hybrid Hash Join
How many I/Os?
Cost of partitioned hash join: 3B(R) + 3B(S)
Now we save 2 disk operations for t/k fraction of buckets
There are k buckets
We save 2t/k(B(R)+B(S))
Cost: (3-2t/k)(B(R)+B(S)) = (3-2M/B(S))(B(R) + B(S))

56. Sort-based and Hash-based Join Algorithms
I/O Cost
Same
Size requirement
sqrt(min(B(R), B(S)) < sqrt(B(R) + B(S))
Hash join wins big when two relations have very different sizes
Join performance
Hash join performance depends on the quality of the hash
Depends on the buckets being of equal size, which may not be true if the number of different hash keys is small

57. Sort-based and Hash-based Join Algorithms
Sorted order
Sort-based algorithms sometimes allow us to produce a result in sorted order and take advantage of that sort later
Sort-Merge Join wins if R and/or S are already sorted
Equality and Non-equality join predicate
For equi-join, where relations not sorted and no indexes exist, hash join usually best
Sort-Merge Join can be adapted for inequality join predicates

58. Index-based Algorithms
Existence of Index on R
Makes available some algorithm that would not be feasible without the index
Where it is useful?
Selection operation
Joins and other binary operations use it for good advantage

59. Clustering Indexes Clustering Index Clustered relation
Tuples are packed in as few blocks as possible
Clustering Index
All tuples with the same value of the key are clustered on as few blocks as possible
A non-clustered relation cannot have clustering index
a a a
a a a a a
a a
All the ‘a’ tuples

60. Index-based Selection
Selection on equality
Cost
sa=v(R)
Clustered index on a: B(R)/V(R,a)
Unclustered index on a: T(R)/V(R,a)
We here ignore the cost of reading
index blocks
Example
Relation R
Index
Cost
Clustered No
B(R) = 1000
Not clustered
T(R) = 20000
V(R, a) = 100
Clustering
1000/100 = 10
V(R, a) = 10
Non-clustering
20000/10 = 2000
V(R, a) = 20000
Yes
1 disk I/O
B(R) = 1000
T(R) = 20,000
sa=0(R)

61. SELECT operation – B+ tree3 5 11 30 35
100
101
110
120
130
150
156
179
180
200
205
Usefulness
Equality and inequality
Variety of conditions possible, e.g. =, , ,  and 
Duplicates can be handled

62. SELECT operation – B+ tree3 5 11 30 35
100
101
110
120
130
150
156
179
180
200
205
cost = x + 1 x
Equality check over unique attribute
Retrieve ‘=100’: cost = 3 + 1
Retrieve ‘=205’: cost = 3 + 1 …

63. SELECT operation – B+ tree3 5 11 30 35
100
110
111
120
130
140
180
200
205
cost = x + v x
Equality check over non-unique attribute
Retrieve ‘=110’: cost = 3 + 2
Retrieve ‘=180’: cost = 3 + 3 …

64. SELECT operation – B+ tree3 5 11 30 35
100
110
111
120
130
140
180
200
205
cost = x (# matching leaf blocks) (# matching records) x
Range Check
Retrieve ‘>111’: cost =
Retrieve ‘>150’: cost = …

65. R(X,Y) S(Y, Z) Cost Basic Idea
Assume S has an index on the join attribute Y
Iterate over R, for each tuple fetch corresponding tuple(s) from S
Cost
Assume R is clustered
If index is clustered: B(R) + T(R)B(S)/V(S,y)
If index is unclustered: B(R) + T(R)T(S)/V(S,y)

66. Algorithm FOR each block b of R DO BEGIN read b into the buffer;
FOR each tuple t in b DO BEGIN
Let ty be the component containing join attributes
Find all tuples of S using index with matching ty
IF matching tuples ty are found in S THEN
BEGIN
join each pair of matching tuples;
output the joined tuple;
END
END;

67. T(R)=10,000, T(S) = 5000, B(R) = 1000, B(S) = 500, V(S, y) = 100
Algorithm
Cost
I/Os
Time
Tuple-based NL Join
T(R)*B(S) + B(R)
50,001,000
14 hrs
Simple Block-based NL Join
B(S) + B(R)*B(S)
500,500
1.4 hrs
Block-based NL Join
B(S) + B(S)B(R)/(M-1)
15,651
3 min
Sort-Merge Join
3(B(R) + B(S))
4,500
1 min
Partition Hash Join
Hybrid Hash Join
(3-2M/B(S))(B(R)+B(S)))
3,894
39s
Index-based Join
B(R) + T(R)B(S)/V(S,y)
51000
8.5 min
Is it a bad idea?
R is very small compared to S and V(S,y) is high

68. Join Using a Sorted Index
Basic Idea
Assume both R and S have a sorted index (B- tree) on the join attribute
Then perform a merge join (called zig-zag join)
Tuples from R with a Y-value that doesn’t appear in S need never be retrieved and vice versa
Cost
B(R) + B(S)

69. Join Using a Sorted Index
Index

70. Summary Study of Query Execution Query Plans Table Scanning
Knowing methods for executing operations of relation algebra with some extensions to support SQL capabilities
Query Plans
Compiled into logical query plan (like expressions of relational algebra)
Converted to physical query plan by selecting an implementation of each operator and other decisions
Table Scanning
Cost Measures
Table scan
Index scan
Sort scan
Number of disk I/O taken to execute an operation is the dominant cost

71. Summary One-pass Algorithms Nested-Loop Join Two-pass Algorithms
One of the arguments of a relational-algebra operator fit in main memory
Read the smaller relation to memory
Read the other relation one block at a time
Nested-Loop Join
Works even when neither arguments fit in main memory
Reads as much it can for the smaller relation into memory
Compare that with the entire other relation
Two-pass Algorithms
None of the arguments fit in main memory
Sort-based, hash-based, index-based

72. Summary Sort-Based Algorithms Hash-based Algorithms
Partition their argument(s) into main-memory-sized, sorted sublists
The sorted sublists are merged appropriately to produce desired results
Hash-based Algorithms
Use a hash function to partition the argument(s) into buckets
Operation is applied to buckets individually or a pair of buckets
Index-based Algorithms
Use index to speed up selection on indexed attributes
Joins are excellent when one of the relation is small and other has index on the join attributes

73. QUERY
OPTIMIZATION

74. Overview SQL Query Compilation Query Execution Storage Query plan
metadata
data
Storage

75. Select logical query plan Select physical query plan
Query Compilation
SQL
Parse tree construction
Parse Query
Query expression tree
Create algebraic representation
Select logical query plan
Select algorithms to implement each operator.
Select order of execution
Logical query plan tree
Select physical query plan
Physical query plan tree
Execute plan

76. Query Evaluation Problem Solution Issue
An SQL query is declarative – does not specify a query execution plan.
A relational algebra expression is procedural – there is an associated query execution plan.
Solution
Convert SQL query to an equivalent relational algebra and evaluate it using the associated query execution plan.
Issue
But which equivalent expression is ”best”?

77. Very inefficient query execution plan!
Naïve Conversion
Query
Very inefficient query execution plan!
SELECT DISTINCT TargetList
FROM R1, R2, …, RN
WHERE Condition
Algebra
TargetList (Condition (R1  R2  ...  RN))
Find an equivalent relational algebra expression that can be evaluated “efficiently”.
Example
Name (Id=ProfId CrsCode=‘CSCI3170’ (Professor  Teaching))
Result can be < 10
If each relation is 50K then Professor  Teaching is of size 2500K before shrinking it down to just a few tuples.

78. Conceptual Model of Cost-based Optimization
Idea
Enumerate all possible plans
Estimate costs
Pick the “best” one
Query optimizers do not “optimize”
Often the goal is not getting the optimum plan, but instead avoiding the horrible ones

79. Two-Phased Query Optimization
Goal of Query Optimizer
Given an initial Logical Plan generate a physical plan that is “Optimal” with respect to one or more performance metrics.
Step 1
Use rewriting laws to enumerate logical query plan
Prune search using heuristics to eliminate bad plans
Step 2
Use cost-based enumeration to choose a good physical
plan corresponding to logical plan.

80. Logical query plan generator
Overview
Parse Tree
Initial expression tree of relational algebra
Logical query plan generator
Improved logical query plan
Query rewriter
Optimized logical
query plan

81. Equivalence Preserving Transformation
Motivation
To transform a relational expression into another equivalent expression we need transformation rules that preserve equivalence
Transformation Rule
Is provably correct (i.e., does preserve equivalence)
Has a heuristic associated with it

82. Commutative and Associative Laws
Commutative Law
It does not matter in which order you present the arguments of the operator; the result will be the same
x+y=y+x
x-y != y-x
Product & Union
Associative Law
R  S = S  R
(R  S)  T = R  (S  T)
R  S = S  R
R  (S  T) = (R  S)  T
We may group two uses of the operator either from the left or the right
(x+y)+z=x+(y+z)
(x-y)-z != x-(y-z)

83. Commutativity and Associativity of Join
Join Commutativity
R S = S R
Used to reduce cost of nested loop evaluation strategies
(smaller relation should be in outer loop)
Join Associativity
R (S T)  (R S) T
Used to reduce the size of intermediate relations in computation of multi-relational join – first compute the join that yields smaller intermediate result

84. Laws Involving Selection
Splitting Laws
Cond1 AND Cond2 (R) = Cond1 (Cond2 (R) )
Cond1 OR Cond2 (R) = (Cond1 (R))  (Cond2 (R) ) (only if R is set)
Ordering doesn’t matter
Cond1 (Cond2 (R) ) = Cond2 (Cond1 (R) )
Pushing selection
Cond (R  S ) = Cond (R)  Cond (S)

85. Laws Involving Selection
Pushing selection
Cond (R  S) = R Cond S
Cond relates attributes of both R and S
Reduces size of intermediate relation since rows can be discarded sooner
Pushing selection
Cond (R  S) = Cond (R)  S
Cond relates attributes of R only
Reduces size of intermediate relation since rows of R can be discarded sooner

86. Laws Involving Projection
Pushing Projection
attr(R  S) = attr(attr (R)  S)
if attributes(R)  attr  attr ∩ attributes(R)
Reduces the size of an operand of product
Projection and Join
Let x = subset of R attributes
y = subset of S attributes
z = intersection of R,S attributes
xy (R S)=xy {[xz(R)] [ yz (S)]}

87. An Example Algebraic equivalence
C1 C2 C3 (R  S) =C1 (C2 (C3 (R  S) ) )
=C1 (C2 (R)  C3 (S) )
= C2 (R) C1 C3 (S)
assuming C2 involves only attributes of R,
C3 involves only attributes of S,
and C1 relates attributes of R and S

88. Improving Logical Query Plan
Find the last names of employees born after 1975 who work on a project named ‘Aquarius’
SELECT LName
FROM EMPLOYEE E, WORKS_ON W, PROJECT P
WHERE PName = ‘Aquarius’ AND W.PNumber = P.PNo
AND W.ESSN=E.SSN AND BDate‘31/Dec/1975’;
LName
EMPLOYEE
WORKS_ON 
PROJECT
PName=‘Aquarius’ AND Pnumber=PNo
AND ESSN=SSN AND Bdate>‘31/Dec/1975’
Initial logical query plan

89. Improving Logical Query Plan
Rule 1: Push selection down …
EMPLOYEE
WORKS_ON 
PROJECT
...
Move selection down the tree
LName
PNumber=PNo 
ESSN=SSN
PName=‘Aquarius’ 
PROJECT
Bdate>‘31/Dec/1975’
WORKS_ON
EMPLOYEE

90. Improving Logical Query Plan
Rule 2: Restrictive Select
Position the leaf node relations with the most restrictive SELECT operations so they are executed first
LName …
WORKS_ON 
PROJECT …
EMPLOYEE
ESSN=SSN 
PNumber=PNo
Bdate>‘31/Dec/1975’ 
EMPLOYEE
WORKS_ON
PName=‘Aquarius’
PROJECT

91. Improving Logical Query Plan
Rule 3: Replace Product with Join
Combine a PRODUCT operation with a subsequent SELECT operation into a join if the condition represents a join condition … …
WORKS_ON 
EMPLOYEE
PROJECT
LName
ESSN=SSN
PNumber=PNo
Bdate>‘31/Dec/1975’
PName=‘Aquarius’
WORKS_ON
EMPLOYEE
PROJECT

92. Optimized Logical Query Plan
Rule 4: Move projection down
Move list of projection attributes down the tree as far as possible by creating new PROJECT operations as needed …
WORKS_ON
EMPLOYEE …
PROJECT
LName
ESSN=SSN
ESSN
SSN,LName
Bdate>‘31/Dec/1975’
PNumber=PNo
PNumber
ESSN,PNo
EMPLOYEE
PName=‘Aquarius’
WORKS_ON
PROJECT

93. Estimating Cost of Logical Query Plans
LName
WORKS_ON
EMPLOYEE
ESSN=SSN
PName=‘Aquarius’
PROJECT
PNumber=PNo
Bdate>‘31/Dec/1975’
LName
WORKS_ON
EMPLOYEE
ESSN=SSN
PNumber
PROJECT
SSN,LName
ESSN
PName=‘Aquarius’
ESSN,PNo
PNumber=PNo
Bdate>‘31/Dec/1975’

94. Estimating Intermediate Results Size
Factors involved in estimating output size
T(R): number of tuples
B(R): number of blocks
V(R,A): number of distinct values of A
MaxV(R,A): maximum value of A
MinV(R,A): minimum value of A
Selectivity
Fraction of records satisfying an equality condition on the attribute
Join Selectivity
join selectivity =
|R S|
|R x S|

95. Estimating the Size of a Selection
Equality comparison
Inequality comparison
A=c (R)
T(S) = T(R)/V(R, A)
Selection cardinality
A<c (R)
T(S) = T(R)/3
AND conditions
Negation comparison
Cond1 AND Cond2 (R)
Treat as cascade of simple selections
Size of the original relation multiplied by selectivity factor for each condition
Ac (R)
T(S) = T(R)(V(R, A)-1)/V(R, A)
T(S) = T(R)
OR conditions
 C1 OR C2 (R)
Assume C1 and C2 are independent
T(S) = T(R)(1-(1- m1/T(R))(1-m2/T(R)))

96. Example AND Condition OR Condition R(a, b, c) T(R) = 10,000 V(a) = 50
 a=10 AND b<20 (R)
T(S) = 10000/(50 *3) = 67
OR Condition
R(a, b, c)
T(R) = 10,000 V(a) = 50
 a=10 OR b<20 (R)
Number of tuples satisfy a=10 is 10000/50 = 200
Number of tuples satisy b<20 is 10000/3 = 3333
T(S) = 10000(1-(1-200/10000)(1-3333/10000)) = 3466

97. Estimating the Size of Join
Size of Join on Single Attribute
T(R S) = T(R)T(S)/max(V(R,Y), V(S, Y))
Size of Join on Multiple Attributes
T(R)T(S) is divided by larger of V(R, y) and V(S, y) for each attribute y that is common to R and S

98. Example Join of R, S, and U on single attribute
R(a, b), S(b, c), U( c, d)
T(R) = T(S) = 2000, T(U) = 5000
V(R,b) = 20, V(S,b) = 50, V(S,c)=100, V(U,c)=500
T(R S) = 1000*2000/50 = 40,000
T((R S) U) = 40000*5000/max(100, 500) =
Join of R, and S on multiple attributes
R(a, b, c), S(b, c, f)
T(R) = T(S) = 2000
V(R,b) = 20, V(S,b) = 50, V(R,c)=100, V(S,c)=50
T(R S) = 1000*2000/(50*100) = 400

99. Selection of Algorithms
Goal
How to select algorithms to implement the operations of the query plan?
LName
WORKS_ON
EMPLOYEE
ESSN=SSN
PNumber
PROJECT
SSN,LName
ESSN
PName=‘Aquarius’
ESSN,PNo
PNumber=PNo
Bdate>‘31/Dec/1975’
Sort merge join
Index-based join
Linear scan
Use index

100. Choosing Order for Joins
Selection of Order
How do we select the order for the (natural) join of three or more relations?
Significance of Join Arguments
ONE-PASS JOIN
Roles played by left and right arguments are different
Cost of join depends on which relation plays what role R1 R2
Choice of order
Only two choices of order for two relations
Number of choices grows rapidly for more than two relations
Probe relation
R1 is smaller than R2
Build relation

101. Join Trees R4 R2 R3 R2 R4 R5 R3 R1 R5 R1 R3 R1 R5 R2 R4 Left-deep
Bushy R3 R1 R5 R1 R3 R1
Right-deep R5 R2 R4

102. Join Order Problem Problem Advantages of choosing left-deep tree
Given: a query R1 R … Rn
Assume we have a function cost() that gives us the cost of every join tree
Find the best join tree for the query
Advantages of choosing left-deep tree
The number of possible left-deep tree with a given number of leaves is large, but not nearly as large as the number of all trees.
Interact well with common join algorithms (NL and one-pass in particular)
Query plans based on left-deep tree tend to be more efficient

103. Our Example Cost-based Optimization Suppliers(sid,sname,scity,sstate)
• Supplies(pno,sid,quantity)
• Some statistics
T(Supplier) = 1000
B(Supplier) = 100
T(Supplies) = 10,000
B(Supplies) = 100
V(Supplier,scity) = 20, V(Supplier,state) = 10
V(Supplies,pno) = 3,000
Both relations are clustered

104. Physical Query Plan 1

105. Physical Query Plan 2

106. Physical Query Plan 3

107. More on Relational DB Systems
Reference:
Hector Garcia-Molina, Jeffrey Ullman, Jenifer Widom.
Database Systems - the Complete Book ,
Second Edition(Prentice Hall)