1. File Processing : Query Processing
Spatiotemporal Database Laboratory Pusan National University
2004, Spring
Pusan National University
Ki-Joune Li

2. Basic Concepts of Query
Retrieve records satisfying predicates
Types of Query
Operators
Aggregate Query
Sorting
Spatiotemporal Database Laboratory Pusan National University

3. Relational Operators : Select
Selection (condition)
Retrieve records satisfying predicates
Example
Find Student where Student.Score > 3.5
score>3.5(Student)
Index or Hash
Predicate
Spatiotemporal Database Laboratory Pusan National University
Select

4. Relational Operators : Project
Project (attributes)
Extract interesting attributes
Example
Find Student.name where score > 3.5
name(acore>3.5(Student))
Full Scan
Interesting attributes to get
Spatiotemporal Database Laboratory Pusan National University
Extract

5. Cartisan Product Cartisan Product () Join ( ) Two Tables : R1  R2
Produce all cross products
Join ( )
r11 …
r21
r22
r2n
r12
r1m
r11
r12 …
r1m R1
r21
r22 …
r2n R2
Spatiotemporal Database Laboratory Pusan National University  =

6. Join
Join ( )
Select combined records of cartisan product with same value of a common attribute (Natural Join)
Example
Student (StudentName, AdvisorProfessorID, Department, Score)
Professor(ProfessorName, ProfessorID, Department)
Student AdivsorProfessorID=ProfessorID Professor
=  AdivsorProfessorID=ProfessorID(Student Professor)
Double Scan : Expensive Operation
Spatiotemporal Database Laboratory Pusan National University

7. Relational Algebra Relational Algebra Operand : Table (Relation)
Operator : Relational Operator (, , , etc)
Example
Find Student Name where Student Score > 3.5 and Advisor Professor belongs to CSE Department
student.name(acore>3.5(Student) Department=‘CSE’ (Professor) )
Relational Algebra Specifies the sequence of operations
Spatiotemporal Database Laboratory Pusan National University   

8. Query Processing Mechanism
Query Processing Steps
1. Parsing and translation
2. Optimization
3. Evaluation
Spatiotemporal Database Laboratory Pusan National University

9. Parsing and Translation
Parsing Query Statement (e.g. in SQL)
Translation into relational algebra
Equivalent Expression
For a same query statement
several relation algebraic expressions are possible
Example
balance  2500(name(account ))  name(balance  2500(account ))
Different execution schedules
Query Execution Plan (QEP)
Determined by relational algebra
Several QEPs may be produced by Parsing and Translation
Spatiotemporal Database Laboratory Pusan National University

10. Query Optimization Choose ONE QEP among QEPs with the consideration on
Execution Cost of each QEP
Cost : Execution Time
How to find cost of QEP ?
Real Execution
Exact but Not Feasible
Cost Estimation
Types of Operations
Number of Records
Selectivity
Distribution of data
Spatiotemporal Database Laboratory Pusan National University

11. Cost Model : Basic Concepts
Cost Model : Number of Block Accesses
Cost
C = Cindex + Cdata
where Cindex : Cost for Index Access
Cdata : Cost for Data Block Retrieval
Cindex vs. Cdata ?
Cindex : depends on index
Cdata
depends on selectivity
Random Access or Sequential Access
Selectivity
Number (or Ratio) of Objects Selected by Query
Spatiotemporal Database Laboratory Pusan National University

12. Cost Model : Type of Operations
Cost model for each type of operations
Select
Project
Join
Aggregate Query
Query Processing Method for each type of operations
Index/Hash or Not
Spatiotemporal Database Laboratory Pusan National University

13. Cost Model : Number of Records
Nrecord  Nblocks
Number of Scans
Single Scan
O(N) : Linear Scan
O(logN ) : Index
Multiple Scans
O(NM ) : Multiple Linear Scans
O(N logM ) : Multiple Scans with Index
Spatiotemporal Database Laboratory Pusan National University

14. Selectivity Selectivity Affects on Cdata Random Access
Scattered on several blocks
Nblock  Nselected
Sequential Access
Contiguously stored on blocks
Nblock = Nselected / Bf
Spatiotemporal Database Laboratory Pusan National University

15. Selectivity Estimation
Depends on Data Distribution
Example
Q1 : Find students where 60 < weight < 70
Q2 : Find students where 80 < weight < 90
How to find the distribution
Parametric Method
e.g. Gaussian Distribution
No a priori knowledge
Non-Parametric Method
e.g. Histogram
Smoothing is necessary
Wavelet, Discrete Cosine
Spatiotemporal Database Laboratory Pusan National University 30 40 50 60 70 80 90
100
Frequency

16. Select : Linear Search Algorithm : linear search
Scan each file block and test all records to see whether they satisfy the selection condition.
Cost estimate (number of disk blocks scanned) = br
br denotes number of blocks containing records from relation r
If selection is on a key attribute (sorted), cost = (br /2)
stop on finding record
Linear search can be applied regardless of
selection condition or
ordering of records in the file, or
availability of indices
Spatiotemporal Database Laboratory Pusan National University

17. Select : Range Search Algorithm : primary index, comparison
Relation is sorted on A
For A  V (r)
Step 1: use index to find first tuple  v and
Step 2: scan relation sequentially
For AV (r)
just scan relation sequentially till first tuple > v; do not use index
Algorithm : secondary index, comparison
Step 1: use index to find first index entry  v and
Step 2: scan index sequentially to find pointers to records.
scan leaf nodes of index finding pointers to records, till first entry > v
In either case, retrieve records that are pointed to
requires an I/O for each record
Linear file scan may be cheaper if records are scattered on many blocks
Spatiotemporal Database Laboratory Pusan National University

18. Select : Complex Query Conjunction : 1  2 . . . n(r)
Algorithm : selection using one index
Step 1: Select a combination of i (i (r) )
Step 2: Test other conditions on tuple after fetching it into memory buffer.
Algorithm : selection using multiple-key index
Use appropriate multiple-attribute index if available.
Algorithm : selection by intersection of identifiers
Step 1: Requires indices with record pointers.
Step 2: Intersection of all the obtained sets of record pointers.
Step 3: Then fetch records from file
Disjunction : 1 2 . . . n (r)
Algorithm : Disjunctive selection by union of identifiers
Spatiotemporal Database Laboratory Pusan National University

19. Join Operation Several different algorithms to implement joins
Nested-loop join
Block nested-loop join
Indexed nested-loop join
Merge-join
Hash-join
Choice based on cost estimate
Examples use the following information
Number of records of customer: 10, depositor: 5000
Number of blocks of customer: depositor: 100
Spatiotemporal Database Laboratory Pusan National University

20. Nested-Loop Join
Algorithm NLJ the theta join r  s For each tuple tr in r do begin For each tuple ts in s do begin test pair (tr,ts) to see if they satisfy the join condition  if they do, add tr • ts to the result. End End
r : outer relation, s : inner relation.
No indices, any kind of join condition.
Expensive
Spatiotemporal Database Laboratory Pusan National University

21. Nested-Loop Join (Cont.)
In the worst case, if there is enough memory only to hold one block of each relation, the estimated cost is nr  bs + br disk accesses.
If the smaller relation fits entirely in memory, use that as the inner relation. Reduces cost to br + bs disk accesses.
Assuming worst case memory availability cost estimate is
5000  = 2,000,100 disk accesses with depositor as outer relation, and
1000  = 1,000,400 disk accesses with customer as the outer relation.
If smaller relation (depositor) fits entirely in memory, the cost estimate will be 500 disk accesses.
Block nested-loops algorithm (next slide) is preferable.
Spatiotemporal Database Laboratory Pusan National University

22. Block Nested-Loop Join
Algoritm BNLJ
For each block Br of r do begin For each block Bs of s do begin For each tuple tr in Br do begin For each tuple ts in Bs do begin Check if (tr,ts) satisfy the join condition if they do, add tr • ts to the result. End End End End
Spatiotemporal Database Laboratory Pusan National University

23. Block Nested-Loop Join (Cont.)
Worst case estimate: br  bs + br block accesses.
Each block in the inner relation s is read once for each block in the outer relation (instead of once for each tuple in the outer relation
Best case: br + bs block accesses.
Improvements to nested loop and block nested loop algorithms:
In block nested-loop, use (M-2) disk blocks as blocking unit for outer relations, where M = memory size in blocks; use remaining two blocks to buffer inner relation and output
Cost = br / (M-2)  bs + br
If equi-join attribute forms a key or inner relation, stop inner loop on first match
Scan inner loop forward and backward alternately, to make use of the blocks remaining in buffer (with LRU replacement)
Use index on inner relation if available (next slide)
Spatiotemporal Database Laboratory Pusan National University

24. Indexed Nested-Loop Join
Index lookups can replace file scans if
join is an equi-join or natural join and
an index is available on the inner relation’s join attribute
Can construct an index just to compute a join.
For each tuple tr in the outer relation r, use the index to look up tuples in s that satisfy the join condition with tuple tr.
Worst case: buffer has space for only one page of r, and, for each tuple in r, we perform an index lookup on s.
Cost of the join: br + nr  c
Where c is the cost of traversing index and fetching all matching s tuples for one tuple or r
c can be estimated as cost of a single selection on s using the join condition.
If indices are available on join attributes of both r and s, use the relation with fewer tuples as the outer relation.
Spatiotemporal Database Laboratory Pusan National University

25. Example of Nested-Loop Join Costs
Compute depositor customer, with depositor as the outer relation.
Let customer have a primary B+-tree index on the join attribute customer-name, which contains 20 entries in each index node.
Since customer has 10,000 tuples, the height of the tree is 4, and one more access is needed to find the actual data
depositor has 5000 tuples
Cost of block nested loops join
400* = 40,100 disk accesses assuming worst case memory (may be significantly less with more memory)
Cost of indexed nested loops join
* 5 = 25,100 disk accesses.
CPU cost likely to be less than that for block nested loops join
Spatiotemporal Database Laboratory Pusan National University

26. Hash-Join Applicable for equi-joins and natural joins.
A hash function h is used to partition tuples of both relations
h maps JoinAttrs values to {0, 1, ..., n}, where JoinAttrs denotes the common attributes of r and s used in the natural join.
r0, r1, . . ., rn denote partitions of r tuples
Each tuple tr  r is put in partition ri where i = h(tr [JoinAttrs]).
s0, s1. . ., sn denotes partitions of s tuples
Each tuple ts s is put in partition si, where i = h(ts [JoinAttrs]).
Note: In book, ri is denoted as Hri, si is denoted as Hsi and n is denoted as nh.
Spatiotemporal Database Laboratory Pusan National University

27. Hash-Join (Cont.)
Spatiotemporal Database Laboratory Pusan National University

28. Hash-Join (Cont.)
r tuples in ri need only to be compared with s tuples in si . Need not be compared with s tuples in any other partition, since:
an r tuple and an s tuple that satisfy the join condition will have the same value for the join attributes.
If that value is hashed to some value i, the r tuple has to be in ri and the s tuple in si .
Spatiotemporal Database Laboratory Pusan National University

29. The hash-join of r and s is computed as follows.
Hash-Join Algorithm
The hash-join of r and s is computed as follows.
1. Partition the relation s using hashing function h. When partitioning a relation, one block of memory is reserved as the output buffer for each partition.
2. Partition r similarly.
3. For each i:
(a) Load si into memory and build an in-memory hash index on it using the join attribute. This hash index uses a different hash function than the earlier one h.
(b) Read the tuples in ri from the disk one by one. For each tuple tr locate each matching tuple ts in si using the in-memory hash index. Output the concatenation of their attributes.
Spatiotemporal Database Laboratory Pusan National University
Relation s is called the build input and r is called the probe input.