Presentation is loading. Please wait.

Presentation is loading. Please wait.

Relational Database Systems 3

Similar presentations


Presentation on theme: "Relational Database Systems 3"— Presentation transcript:

1 Relational Database Systems 3
Instructor: Prof. James Cheng Acknowledgement: The slides are extracted from and modified based on the slides provided by Prof. Sourav S. Bhowmick from Nanyang Technological University.

2 Topics to be covered ER model Relational Algebra SQL
Storage and Index Structures Query Processing and Query Optimization

3 An Example Query Find the movies with stars born in 1960
MovieStar (name, birthdate, ...) StarsIn (starName, title) Movie (title, year, language, ... ) SELECT title FROM StarsIn WHERE starName IN ( SELECT name FROM MovieStar WHERE birthdate LIKE .%1960 );

4 Parse Tree

5 Relational Algebraic Representation
SELECT title FROM StarsIn WHERE starName IN ( SELECT name FROM MovieStar WHERE birthdate LIKE .%1960);

6 Select logical query plan Select physical query plan
Query Compilation SQL Parse tree construction Parse Query Query expression tree Create algebraic representation Select logical query plan Select algorithms to implement each operator. Select order of execution Logical query plan tree Select physical query plan Physical query plan tree Execute plan

7 Cost Model and Factors Affecting Cost
Measure of cost No. of disk I/Os Main memory Portion of main memory that the operator uses Main memory divided into buffers Size of buffer = size of disk blocks M = No. of main-memory buffers available Size of the arguments B(R) = Number of blocks needed to hold tuples of R T(R) = Number of tuples in R V(R, a) = Number of distinct values of attribute a in R

8 Natural Join [R(X,Y) S(Y,Z)]
Seen Before? Yes! BST/HT Search key is Y No! Output buffer M-1 buffer Mth buffer Cost Num of disk I/O = B(R) + B(S) Space requirement R S min(B(R), B(S)) <= M-1

9 Nested Loop Join Why “one-and-a-half”? Nested-loop join
One of the two arguments has its tuples read only once Other argument is read repeatedly Nested-loop join Can be used for relations of any size Not necessary that one relation fit in main memory

10 Tuple-Based Nested-Loop Join
R(X,Y) S(Y,Z)] FOR each tuple r in R DO FOR each tuple s in S DO IF r and s join to make a tuple t THEN Output t Cost Example For each tuple in R scan the entire relation S T(R)*T(S) disk I/Os T(R)*B(S) + B(R) disk I/Os (when S is clustered) T(R)*T(S) + B(R) disk I/Os (when S is unclustered) T(R) = 10000 B(R) = 1000 B(S) = 500 No. Of I/Os = Assume 100 I/Os per sec Approx 14 hours!

11 Block-based Nested-Loop Join
Principle Organizing access to both argument relations by blocks Use as much main memory as we can store for tuples of the outer relation R

12 Page-At-A-Time NL Join
R(X,Y) S(Y,Z)] FOR each block br in R DO FOR each block bs in S DO IF tuples r and s join -> make a tuple t THEN Output t Cost Example B(R) + B(R)*B(S) [S is clustered] B(R) + B(R)*T(S) [S is unclustered] B(R) = 1000, B(S) = 500 *1000 = 500,500 (S outer) *500 = 501,000 (R outer) Approx 1.4 hrs!

13 Example [R(X,Y) S(Y,Z)] S R Input Buffer of S Output Input buffer
Buffer of R … and so on S R

14 The Idea M R S

15 Block-Based Nested-Loop Join
What is “Block” in NL Join? Group of (M-1) pages of R is called a block R(X,Y) S(Y,Z)] UNTIL all of R blocks been read DO READ in M-1 blocks of R FOR each block of S DO READ in the single S block Check pairs of tuples in memory and output if they match

16 Example R S R S Input buffer for R … and so on Input buffer for s
Output buffer

17 Cost Analysis R(X,Y) S(Y,Z)] UNTIL all of R blocks been read DO
READ in M-1 blocks of R FOR each block of S DO READ in the single S block Check pairs of tuples in memory and output if they match Cost Each iteration: M-1 block of R and B(S) blocks of S Number of iterations = B(R)/(M-1) Total number of disk I/O = (M-1 + B(S)) x B(R)/(M-1) = B(R) + B(S)B(R)/(M-1)

18 Block-Based Nested-Loop Join
Example B(R) = 10000, B(S) = 500, M = 51 *10000/50 = 100,500 (S outer) *500/50 = 110,000 (R outer) Principle Use smaller relation as outer relation

19 Summary Algorithm Cost Example Tuple-based NL Join T(R)*B(S) + B(R)
14 hrs Simple Block-based NL Join B(S) + B(R)*B(S) 1.4 hrs Block-based NL Join B(S) + B(S)B(R)/(M-1) 3 min

20 Two-Pass Algorithms Principle
Data from the operand relations are read into main memory Processed Written out to disk Reread from disk to complete operation

21 Types of Two-Pass Algorithms
Sort-Based Algorithms Partition their argument(s) into main-memory-sized, sorted sublists The sorted sublists are merged appropriately to produce desired results Hash-based Algorithms Use a hash function to partition the argument(s) into buckets Operation is applied to buckets individually or a pair of buckets Index-based Algorithms Use index to speed up selection on indexed attributes

22 Sorting-based Algorithms
Problem Sorting is used in implementing many relational operations Relations are typically large, do not fit in main memory So cannot use traditional in-memory sorting algorithms Approach Combine in-memory sorting with clever techniques aimed at minimizing I/O I/O costs dominate => cost of sorting algorithm is measured in the number of page transfers

23 Two-Way Merge Sort g a d c b e r m p g a e c b d r m p g a d c b e r m
24 a 19 d 31 c 33 b 14 e 16 r 21 m 3 p 2 7 Runs Create runs g 24 a 19 e 16 c 33 b 14 d 31 r 21 m 3 p 2 7 Merge g 24 a 19 d 31 c 33 b 14 e 16 r 21 m 3 p 2 7 g 24 a 19 d 31 c 33 b 14 e 16 r 21 m 3 p 2 7 Sorted output Merge R R R W W W Initial relation

24 3 Buffers (2 for reading), B(R)=4
16 r g 24 d 31 g 24 a 19 d 31 c 33 a 19 c 33 b 14 d 21 a 19 g 24 a 19 g 24 a 19 d 31 c 33 b 14 e 16 r 21 Block b 14 c 33 d 31 c 33 d 31 d 21 g 24 b 14 e 16 b 14 e 16 r d 21 e 16 r 16 d 21 r 16 Initial relation Create run Merge run Sorted output R R R W W W Number of passes = log2B + 1 Cost = 2B(log2B + 1)

25 Two-Phase Multiway Merge-Sort
Motivation Computation involved in sorting records in buffers in main memory I/O necessary to move records between disk and main memory Goal: Minimize number of runs and number of passes Phase 1 Phase 2 Repeatedly fill the M buffers with new tuples from R and sort them using main-memory sorting algorithm Write out each sorted sublist to disk Merge the sorted sublist . Requirement At most M-1 sorted sublist One input block to each sorted sublist One block to the output

26 Phase 2 Buffer M pages

27 Algorithm for Merge Step 1 Step 2 Step 4 Step 3 Step 5
Find the smallest key among the first remaining elements of all the lists (linear search or use priority queue) Move the smallest element to the first available position of the output block Step 4 Step 3 If the block from which the smallest element was just taken is now exhausted of records Read the next block from the same sorted sublist into thesame buffer If the output block is full, write it to disk and reinitialize the same buffer in main memory to hold the next output block Step 5 If no blocks remain, then leave its buffer empty and do not consider elements from that list

28 Example Input runs Output run Input buffers Output buffer 2 3 5 6 2 5
2 5 3 6 10 3 6 1 2 5 15 7 10 1 15 7 Input buffers Output buffer

29 Cost Analysis Bounds on number of blocks Cost
There must be no more than M-1 sublists Each sublist = M blocks Number of sublists = B/M B/M <= M-1 => B <= M2 Cost Reads B blocks in the first pass B disk I/O to write sorted sublists Sorted sublists are read again in the second pass Total: 3B disk I/O

30 What if R is too big? Multi-Phase, Multiway Merge Sort
Apply the idea recursively Divide R into chunks of size M(M-1) Use 2Phase-Multiway Merge Sort to sort each one Take resulting sorted lists as input for a third (merge) phase

31 Duplicate Elimination
Importance A major step in computing projection, union, and difference relational operators Algorithm Sort At the last stage of the merge step repeatedly select the first unconsidered tuple t among all the sorted sublists Write one copy to the output. Eliminate from the input blocks all occurrences of t No additional cost (with respect to sorting) in terms of I/O Total: 3B disk I/O

32 Duplicate elimination During Merge
Input runs Last key used Output run 5 2 6 3 15 2 3 6 1 5 3 6 1 5 15 2 5 15 1 5 15 3 Key 3 ignored: duplicate Key 5 ignored: duplicate Input buffers Output buffer

33 Basic Idea of Sort-based Join
Principle Sort both relations on the join attributes Groups all tuples with the same value in the join col. together (partitions) Look for qualifying tuples by merging the two relations Compare R tuples in a partition with only the S tuples in the same partition Challenge Number of tuples share a common join attribute(s) can exceed what fits in memory

34 Sort-Based Join Algorithms
Sort-Merge Join Two relations are sorted first and then merged in distinct pass Makes the greatest possible number of buffers available for joining tuples with common value. Refined Sort-Merge Join Combine merging phase of sort with the merging phase of the join Problems when there are large number of tuples with common join attribute value.

35 Sort-Merge Join [R(X,Y) S(Y,Z)]
Step 1 Step 2 Sort R and S with Y as the sort key Merge the sorted R and S. Use two buffers: the current block of R and the current block of S Find the least value y of the join attributes Y that is currently at the front of the blocks of R and S Step 3 If y does not appear at the front of the other relation, then remove the tuple(s) with sort key y Step 5 Step 4 Output all the tuples that can be formed by joining over y. Otherwise, identify all the tuples from both relations having sort key y. Read blocks if necessary (M buffer available) Step 6 If either relation has no more unconsidered tuples in memory, reload the buffer for that relation

36 Overview R S R(X,Y) S(Y,Z)] sort R on Y; sort S on Y;
while !eof(R) and !eof(S) do { Scan R and S concurrently until tR.Y=tS.Y=c; Output Y=c(R)Y=c (S) } Y=c(R) R S Y=c (S)

37 Example R S R S X Y Y Z 1 3 p p 0 9 q q 8 7 3 s s s 5 7 u u 1 1 v v
1 3 p p 0 9 q q s s s 5 7 u u 1 1 v v p p p p s s s u u u u u u Y Z p p 4 0 r 9 s 7 t t 2 5 u u u x R S S

38 Cost Analysis How big M needs to be for it to work? Cost
We need to be able to perform 2Phase Multiway Merge Sort B(R) <= M2 and B(S) <= M2 All tuples with common Y-value must fit in M buffer Cost Sorting: four disk I/Os per block (two in each phase) Merging: Read each block of R and S one time Single scan is guaranteed if at least one of the relation has no duplicates in the join attribute Total: 5(B(R) + B(S))

39 Refinement of Sort-Merge Join
3(B(R) + B(S)) disk I/O B(R) + B(S) <= M2 Idea Combine the merging phase of sorting with the merging phase of the join Step 1 Step 3 Repeatedly find the least Y-value y among the first available tuples of all the sublists Identify all tuples in R and S that have y Output the join of all tuples of R with all of S with common Y-value If the buffer for one of the sublist is exhausted, refill Create sorted sublists of size M, using Y as the sort key, for both R and S Step 2 Bring the first block of each sublist into a buffer Assume no more than M sublists

40 Example Sort-Merge Join B(R) = 1000, B(S) = 500, M = 101
7500 disk I/O (Sort-Merge) 4500 disk I/O (Enhanced) Approx 1 min!

41 Summary Algorithm Cost I/Os Time Tuple-based NL Join T(R)*B(S) + B(R)
14 hrs Simple Block-based NL Join B(S) + B(R)*B(S) 500,500 1.4 hrs Block-based NL Join* B(S) + B(S)B(R)/(M-1) 5,500 1 min Sort-Merge Join** 3(B(R) + B(S)) 4,500 ** M = 101

42 Hashing-based Two Pass Algorithms
The idea If the data is too big to store in main-memory buffers, hash all the tuples of the argument(s) using an appropriate hash key How it works? Select the hash key so that all tuples that need to be considered together falls into the same bucket Operate on a bucket (or a pair of buckets) at a time Size of the relations are reduced by a factor equal to number of buckets (generally M)

43 Partitioning Relation R into M-1 buckets
Initialize M-1 buckets using M-1 empty buffers; FOR each block b of R DO BEGIN read b into the Mth buffer; FOR each tuple t in b DO BEGIN IF the buffer for bucket h(t) has no room for t THEN BEGIN copy the buffer to disk; empty the buffer; END copy t to the buffer for bucket h(t); END; FOR each bucket DO IF the buffer for this bucket is not empty THEN write the buffer to disk;

44 Partitioning of Relations
Compute the hash value of a whole tuple Same tuples are hashed to the same bucket Relation R Buckets OUTPUT 1 1 R1 hash function h 2 INPUT 2 R2 . . . M-1 B(R) RM-1 Disk Disk M main memory buffers Each bucket has size approx. B(R)/M-1

45 Partition Hash Join Step 1 Step 2 Step 3
Hash R to M-1 buckets using the join attribute(s) as hash key Send all buckets to disk Step 2 Hash S to M-1 buckets using the join attribute(s) as hash key Send all buckets to disk Step 3 Join every pair of buckets using one-pass algorithm

46 Steps 1 and 2 Relation R Buckets OUTPUT 1 1 R1 hash function h 2 INPUT 2 R2 . . . M-1 B(R) RM-1 Disk Disk M main memory buffers Tuples in partition Ri only match tuples in partition Si. OUTPUT 1 1 S1 hash function h 2 INPUT 2 S2 . . . M-1 B(S) SM-1 Disk Disk M main memory buffers

47 Step 3 Steps Read in a partition of Si, hash it using another hash function h’ Scan matching partition of R, hash each tuple in the partition using h’, and search for matching tuples of Si in the same bucket S1 hash function h’ R1 S2 R2 Join RM-1 SM-1 M-1 blocks Buckets for S Buckets for R Main Memory Buffers

48 Requirements Requirements
Each Si needs to fit into (M-1) blocks or buffers in memory S has (M-1) buckets B(S) <= (M-1)(M-1) Min(B(R), B(S)) <= M2 S1 hash function h’ R1 S2 R2 Join RM-1 SM-1 M-1 blocks Buckets for S Buckets for R Main Memory Buffers

49 Cost of Partition Hash Join
Step 1 Hash R to M-1 buckets using the join attribute(s) as hash key Send all buckets to disk => (B(R) + B(R)) disk I/Os read + write Step 2 Hash S to M-1 buckets using the join attribute(s) as hash key Send all buckets to disk => (B(S) + B(S)) disk I/Os read + write Step 3 Join every pair of buckets using one-pass algorithm => (B(R) + B(S)) disk I/Os: read from disk Cost 3(B(R) + B(S)) disk I/Os

50 Hash Join Tricks Question Solution
What if a partition is too large to fit into memory? Solution Repeat Step 1 and/or Step 2 recursively

51 Example Algorithm Cost I/Os Time Tuple-based NL Join T(R)*B(S) + B(R)
14 hrs Simple Block-based NL Join B(S) + B(R)*B(S) 500,500 1.4 hrs Block-based NL Join* B(S) + B(S)B(R)/(M-1) 15,651 3 min Sort-Merge Join** 3(B(R) + B(S)) 4,500 1 min Partition Hash Join * M = 34 ** M = 101

52 Hybrid Hash Join h What if there is extra memory available?
B(S) << M2 Use it to avoid writing/re-reading partitions Both R and S hash function h INPUT Disk M main memory buffers

53 Hybrid Hash Join Algorithm
Step 1 Step 2 Partition R into k buckets First t buckets joined immediately with S Other k-t buckets to disk Partition S into k buckets But keep t buckets (S1 , S2 ,…, St) in memory, k-t buckets to disk Step 3 Finally join k-t pairs of bucket (Rt+1, St+1), (Rt+2, St+2),…, (Rk, Sk) hash function h INPUT Disk M main memory buffers

54 How to choose k and t? h Choosing k and t
Average bucket size for S is B(S)/k Need to fit (B(S)/k)*t + (k-t) blocks in memory B(S)*t/k + (k-t) <= M Assume B(S)*t/k >> (k-t) t/k = M/B(S) hash function h INPUT

55 Cost of Hybrid Hash Join
How many I/Os? Cost of partitioned hash join: 3B(R) + 3B(S) Now we save 2 disk operations for t/k fraction of buckets There are k buckets We save 2t/k(B(R)+B(S)) Cost: (3-2t/k)(B(R)+B(S)) = (3-2M/B(S))(B(R) + B(S))

56 Sort-based and Hash-based Join Algorithms
I/O Cost Same Size requirement sqrt(min(B(R), B(S)) < sqrt(B(R) + B(S)) Hash join wins big when two relations have very different sizes Join performance Hash join performance depends on the quality of the hash Depends on the buckets being of equal size, which may not be true if the number of different hash keys is small

57 Sort-based and Hash-based Join Algorithms
Sorted order Sort-based algorithms sometimes allow us to produce a result in sorted order and take advantage of that sort later Sort-Merge Join wins if R and/or S are already sorted Equality and Non-equality join predicate For equi-join, where relations not sorted and no indexes exist, hash join usually best Sort-Merge Join can be adapted for inequality join predicates

58 Index-based Algorithms
Existence of Index on R Makes available some algorithm that would not be feasible without the index Where it is useful? Selection operation Joins and other binary operations use it for good advantage

59 Clustering Indexes Clustering Index Clustered relation
Tuples are packed in as few blocks as possible Clustering Index All tuples with the same value of the key are clustered on as few blocks as possible A non-clustered relation cannot have clustering index a a a a a a a a a a All the ‘a’ tuples

60 Index-based Selection
Selection on equality Cost sa=v(R) Clustered index on a: B(R)/V(R,a) Unclustered index on a: T(R)/V(R,a) We here ignore the cost of reading index blocks Example Relation R Index Cost Clustered No B(R) = 1000 Not clustered T(R) = 20000 V(R, a) = 100 Clustering 1000/100 = 10 V(R, a) = 10 Non-clustering 20000/10 = 2000 V(R, a) = 20000 Yes 1 disk I/O B(R) = 1000 T(R) = 20,000 sa=0(R)

61 SELECT operation – B+ tree
3 5 11 30 35 100 101 110 120 130 150 156 179 180 200 205 Usefulness Equality and inequality Variety of conditions possible, e.g. =, , ,  and  Duplicates can be handled

62 SELECT operation – B+ tree
3 5 11 30 35 100 101 110 120 130 150 156 179 180 200 205 cost = x + 1 x Equality check over unique attribute Retrieve ‘=100’: cost = 3 + 1 Retrieve ‘=205’: cost = 3 + 1

63 SELECT operation – B+ tree
3 5 11 30 35 100 110 111 120 130 140 180 200 205 cost = x + v x Equality check over non-unique attribute Retrieve ‘=110’: cost = 3 + 2 Retrieve ‘=180’: cost = 3 + 3

64 SELECT operation – B+ tree
3 5 11 30 35 100 110 111 120 130 140 180 200 205 cost = x (# matching leaf blocks) (# matching records) x Range Check Retrieve ‘>111’: cost = Retrieve ‘>150’: cost =

65 R(X,Y) S(Y, Z) Cost Basic Idea
Assume S has an index on the join attribute Y Iterate over R, for each tuple fetch corresponding tuple(s) from S Cost Assume R is clustered If index is clustered: B(R) + T(R)B(S)/V(S,y) If index is unclustered: B(R) + T(R)T(S)/V(S,y)

66 Algorithm FOR each block b of R DO BEGIN read b into the buffer;
FOR each tuple t in b DO BEGIN Let ty be the component containing join attributes Find all tuples of S using index with matching ty IF matching tuples ty are found in S THEN BEGIN join each pair of matching tuples; output the joined tuple; END END;

67 T(R)=10,000, T(S) = 5000, B(R) = 1000, B(S) = 500, V(S, y) = 100
Algorithm Cost I/Os Time Tuple-based NL Join T(R)*B(S) + B(R) 50,001,000 14 hrs Simple Block-based NL Join B(S) + B(R)*B(S) 500,500 1.4 hrs Block-based NL Join B(S) + B(S)B(R)/(M-1) 15,651 3 min Sort-Merge Join 3(B(R) + B(S)) 4,500 1 min Partition Hash Join Hybrid Hash Join (3-2M/B(S))(B(R)+B(S))) 3,894 39s Index-based Join B(R) + T(R)B(S)/V(S,y) 51000 8.5 min Is it a bad idea? R is very small compared to S and V(S,y) is high

68 Join Using a Sorted Index
Basic Idea Assume both R and S have a sorted index (B- tree) on the join attribute Then perform a merge join (called zig-zag join) Tuples from R with a Y-value that doesn’t appear in S need never be retrieved and vice versa Cost B(R) + B(S)

69 Join Using a Sorted Index
Index

70 Summary Study of Query Execution Query Plans Table Scanning
Knowing methods for executing operations of relation algebra with some extensions to support SQL capabilities Query Plans Compiled into logical query plan (like expressions of relational algebra) Converted to physical query plan by selecting an implementation of each operator and other decisions Table Scanning Cost Measures Table scan Index scan Sort scan Number of disk I/O taken to execute an operation is the dominant cost

71 Summary One-pass Algorithms Nested-Loop Join Two-pass Algorithms
One of the arguments of a relational-algebra operator fit in main memory Read the smaller relation to memory Read the other relation one block at a time Nested-Loop Join Works even when neither arguments fit in main memory Reads as much it can for the smaller relation into memory Compare that with the entire other relation Two-pass Algorithms None of the arguments fit in main memory Sort-based, hash-based, index-based

72 Summary Sort-Based Algorithms Hash-based Algorithms
Partition their argument(s) into main-memory-sized, sorted sublists The sorted sublists are merged appropriately to produce desired results Hash-based Algorithms Use a hash function to partition the argument(s) into buckets Operation is applied to buckets individually or a pair of buckets Index-based Algorithms Use index to speed up selection on indexed attributes Joins are excellent when one of the relation is small and other has index on the join attributes

73 QUERY OPTIMIZATION

74 Overview SQL Query Compilation Query Execution Storage Query plan
metadata data Storage

75 Select logical query plan Select physical query plan
Query Compilation SQL Parse tree construction Parse Query Query expression tree Create algebraic representation Select logical query plan Select algorithms to implement each operator. Select order of execution Logical query plan tree Select physical query plan Physical query plan tree Execute plan

76 Query Evaluation Problem Solution Issue
An SQL query is declarative – does not specify a query execution plan. A relational algebra expression is procedural – there is an associated query execution plan. Solution Convert SQL query to an equivalent relational algebra and evaluate it using the associated query execution plan. Issue But which equivalent expression is ”best”?

77 Very inefficient query execution plan!
Naïve Conversion Query Very inefficient query execution plan! SELECT DISTINCT TargetList FROM R1, R2, …, RN WHERE Condition Algebra TargetList (Condition (R1  R2  ...  RN)) Find an equivalent relational algebra expression that can be evaluated “efficiently”. Example Name (Id=ProfId CrsCode=‘CSCI3170’ (Professor  Teaching)) Result can be < 10 If each relation is 50K then Professor  Teaching is of size 2500K before shrinking it down to just a few tuples.

78 Conceptual Model of Cost-based Optimization
Idea Enumerate all possible plans Estimate costs Pick the “best” one Query optimizers do not “optimize” Often the goal is not getting the optimum plan, but instead avoiding the horrible ones

79 Two-Phased Query Optimization
Goal of Query Optimizer Given an initial Logical Plan generate a physical plan that is “Optimal” with respect to one or more performance metrics. Step 1 Use rewriting laws to enumerate logical query plan Prune search using heuristics to eliminate bad plans Step 2 Use cost-based enumeration to choose a good physical plan corresponding to logical plan.

80 Logical query plan generator
Overview Parse Tree Initial expression tree of relational algebra Logical query plan generator Improved logical query plan Query rewriter Optimized logical query plan

81 Equivalence Preserving Transformation
Motivation To transform a relational expression into another equivalent expression we need transformation rules that preserve equivalence Transformation Rule Is provably correct (i.e., does preserve equivalence) Has a heuristic associated with it

82 Commutative and Associative Laws
Commutative Law It does not matter in which order you present the arguments of the operator; the result will be the same x+y=y+x x-y != y-x Product & Union Associative Law R  S = S  R (R  S)  T = R  (S  T) R  S = S  R R  (S  T) = (R  S)  T We may group two uses of the operator either from the left or the right (x+y)+z=x+(y+z) (x-y)-z != x-(y-z)

83 Commutativity and Associativity of Join
Join Commutativity R S = S R Used to reduce cost of nested loop evaluation strategies (smaller relation should be in outer loop) Join Associativity R (S T)  (R S) T Used to reduce the size of intermediate relations in computation of multi-relational join – first compute the join that yields smaller intermediate result

84 Laws Involving Selection
Splitting Laws Cond1 AND Cond2 (R) = Cond1 (Cond2 (R) ) Cond1 OR Cond2 (R) = (Cond1 (R))  (Cond2 (R) ) (only if R is set) Ordering doesn’t matter Cond1 (Cond2 (R) ) = Cond2 (Cond1 (R) ) Pushing selection Cond (R  S ) = Cond (R)  Cond (S)

85 Laws Involving Selection
Pushing selection Cond (R  S) = R Cond S Cond relates attributes of both R and S Reduces size of intermediate relation since rows can be discarded sooner Pushing selection Cond (R  S) = Cond (R)  S Cond relates attributes of R only Reduces size of intermediate relation since rows of R can be discarded sooner

86 Laws Involving Projection
Pushing Projection attr(R  S) = attr(attr (R)  S) if attributes(R)  attr  attr ∩ attributes(R) Reduces the size of an operand of product Projection and Join Let x = subset of R attributes y = subset of S attributes z = intersection of R,S attributes xy (R S)=xy {[xz(R)] [ yz (S)]}

87 An Example Algebraic equivalence
C1 C2 C3 (R  S) =C1 (C2 (C3 (R  S) ) ) =C1 (C2 (R)  C3 (S) ) = C2 (R) C1 C3 (S) assuming C2 involves only attributes of R, C3 involves only attributes of S, and C1 relates attributes of R and S

88 Improving Logical Query Plan
Find the last names of employees born after 1975 who work on a project named ‘Aquarius’ SELECT LName FROM EMPLOYEE E, WORKS_ON W, PROJECT P WHERE PName = ‘Aquarius’ AND W.PNumber = P.PNo AND W.ESSN=E.SSN AND BDate‘31/Dec/1975’; LName EMPLOYEE WORKS_ON PROJECT PName=‘Aquarius’ AND Pnumber=PNo AND ESSN=SSN AND Bdate>‘31/Dec/1975’ Initial logical query plan

89 Improving Logical Query Plan
Rule 1: Push selection down … EMPLOYEE WORKS_ON PROJECT ... Move selection down the tree LName PNumber=PNo ESSN=SSN PName=‘Aquarius’ PROJECT Bdate>‘31/Dec/1975’ WORKS_ON EMPLOYEE

90 Improving Logical Query Plan
Rule 2: Restrictive Select Position the leaf node relations with the most restrictive SELECT operations so they are executed first LName … WORKS_ON PROJECT … EMPLOYEE ESSN=SSN PNumber=PNo Bdate>‘31/Dec/1975’ EMPLOYEE WORKS_ON PName=‘Aquarius’ PROJECT

91 Improving Logical Query Plan
Rule 3: Replace Product with Join Combine a PRODUCT operation with a subsequent SELECT operation into a join if the condition represents a join condition … … WORKS_ON EMPLOYEE PROJECT LName ESSN=SSN PNumber=PNo Bdate>‘31/Dec/1975’ PName=‘Aquarius’ WORKS_ON EMPLOYEE PROJECT

92 Optimized Logical Query Plan
Rule 4: Move projection down Move list of projection attributes down the tree as far as possible by creating new PROJECT operations as needed … WORKS_ON EMPLOYEE … PROJECT LName ESSN=SSN ESSN SSN,LName Bdate>‘31/Dec/1975’ PNumber=PNo PNumber ESSN,PNo EMPLOYEE PName=‘Aquarius’ WORKS_ON PROJECT

93 Estimating Cost of Logical Query Plans
LName WORKS_ON EMPLOYEE ESSN=SSN PName=‘Aquarius’ PROJECT PNumber=PNo Bdate>‘31/Dec/1975’ LName WORKS_ON EMPLOYEE ESSN=SSN PNumber PROJECT SSN,LName ESSN PName=‘Aquarius’ ESSN,PNo PNumber=PNo Bdate>‘31/Dec/1975’

94 Estimating Intermediate Results Size
Factors involved in estimating output size T(R): number of tuples B(R): number of blocks V(R,A): number of distinct values of A MaxV(R,A): maximum value of A MinV(R,A): minimum value of A Selectivity Fraction of records satisfying an equality condition on the attribute Join Selectivity join selectivity = |R S| |R x S|

95 Estimating the Size of a Selection
Equality comparison Inequality comparison A=c (R) T(S) = T(R)/V(R, A) Selection cardinality A<c (R) T(S) = T(R)/3 AND conditions Negation comparison Cond1 AND Cond2 (R) Treat as cascade of simple selections Size of the original relation multiplied by selectivity factor for each condition Ac (R) T(S) = T(R)(V(R, A)-1)/V(R, A) T(S) = T(R) OR conditions  C1 OR C2 (R) Assume C1 and C2 are independent T(S) = T(R)(1-(1- m1/T(R))(1-m2/T(R)))

96 Example AND Condition OR Condition R(a, b, c) T(R) = 10,000 V(a) = 50
 a=10 AND b<20 (R) T(S) = 10000/(50 *3) = 67 OR Condition R(a, b, c) T(R) = 10,000 V(a) = 50  a=10 OR b<20 (R) Number of tuples satisfy a=10 is 10000/50 = 200 Number of tuples satisy b<20 is 10000/3 = 3333 T(S) = 10000(1-(1-200/10000)(1-3333/10000)) = 3466

97 Estimating the Size of Join
Size of Join on Single Attribute T(R S) = T(R)T(S)/max(V(R,Y), V(S, Y)) Size of Join on Multiple Attributes T(R)T(S) is divided by larger of V(R, y) and V(S, y) for each attribute y that is common to R and S

98 Example Join of R, S, and U on single attribute
R(a, b), S(b, c), U( c, d) T(R) = T(S) = 2000, T(U) = 5000 V(R,b) = 20, V(S,b) = 50, V(S,c)=100, V(U,c)=500 T(R S) = 1000*2000/50 = 40,000 T((R S) U) = 40000*5000/max(100, 500) = Join of R, and S on multiple attributes R(a, b, c), S(b, c, f) T(R) = T(S) = 2000 V(R,b) = 20, V(S,b) = 50, V(R,c)=100, V(S,c)=50 T(R S) = 1000*2000/(50*100) = 400

99 Selection of Algorithms
Goal How to select algorithms to implement the operations of the query plan? LName WORKS_ON EMPLOYEE ESSN=SSN PNumber PROJECT SSN,LName ESSN PName=‘Aquarius’ ESSN,PNo PNumber=PNo Bdate>‘31/Dec/1975’ Sort merge join Index-based join Linear scan Use index

100 Choosing Order for Joins
Selection of Order How do we select the order for the (natural) join of three or more relations? Significance of Join Arguments ONE-PASS JOIN Roles played by left and right arguments are different Cost of join depends on which relation plays what role R1 R2 Choice of order Only two choices of order for two relations Number of choices grows rapidly for more than two relations Probe relation R1 is smaller than R2 Build relation

101 Join Trees R4 R2 R3 R2 R4 R5 R3 R1 R5 R1 R3 R1 R5 R2 R4 Left-deep
Bushy R3 R1 R5 R1 R3 R1 Right-deep R5 R2 R4

102 Join Order Problem Problem Advantages of choosing left-deep tree
Given: a query R1 R … Rn Assume we have a function cost() that gives us the cost of every join tree Find the best join tree for the query Advantages of choosing left-deep tree The number of possible left-deep tree with a given number of leaves is large, but not nearly as large as the number of all trees. Interact well with common join algorithms (NL and one-pass in particular) Query plans based on left-deep tree tend to be more efficient

103 Our Example Cost-based Optimization Suppliers(sid,sname,scity,sstate)
• Supplies(pno,sid,quantity) • Some statistics T(Supplier) = 1000 B(Supplier) = 100 T(Supplies) = 10,000 B(Supplies) = 100 V(Supplier,scity) = 20, V(Supplier,state) = 10 V(Supplies,pno) = 3,000 Both relations are clustered

104 Physical Query Plan 1

105 Physical Query Plan 2

106 Physical Query Plan 3

107 More on Relational DB Systems
Reference: Hector Garcia-Molina, Jeffrey Ullman, Jenifer Widom. Database Systems - the Complete Book , Second Edition(Prentice Hall)


Download ppt "Relational Database Systems 3"

Similar presentations


Ads by Google