1. Storage and Indexes Chapter 8 & 9
The slides for this text are organized into chapters. This lecture covers Chapter 7.
Chapter 1: Introduction to Database Systems
Chapter 2: The Entity-Relationship Model
Chapter 3: The Relational Model
Chapter 4 (Part A): Relational Algebra
Chapter 4 (Part B): Relational Calculus
Chapter 5: SQL: Queries, Programming, Triggers
Chapter 6: Query-by-Example (QBE)
Chapter 7: Storing Data: Disks and Files
Chapter 8: File Organizations and Indexing
Chapter 9: Tree-Structured Indexing
Chapter 10: Hash-Based Indexing
Chapter 11: External Sorting
Chapter 12 (Part A): Evaluation of Relational Operators
Chapter 12 (Part B): Evaluation of Relational Operators: Other Techniques
Chapter 13: Introduction to Query Optimization
Chapter 14: A Typical Relational Optimizer
Chapter 15: Schema Refinement and Normal Forms
Chapter 16 (Part A): Physical Database Design
Chapter 16 (Part B): Database Tuning
Chapter 17: Security
Chapter 18: Transaction Management Overview
Chapter 19: Concurrency Control
Chapter 20: Crash Recovery
Chapter 21: Parallel and Distributed Databases
Chapter 22: Internet Databases
Chapter 23: Decision Support
Chapter 24: Data Mining
Chapter 25: Object-Database Systems
Chapter 26: Spatial Data Management
Chapter 27: Deductive Databases
Chapter 28: Additional Topics
Chapter 8 & 9 1

2. DBMS Structure

3. Disks and Files DBMS stores information on (“hard”) disks.
A disk is a sequence of bytes, each has a disk address.
READ: transfer data from disk to main memory (RAM).
WRITE: transfer data from RAM to disk.
Both are high-cost, relative to in-memory operations.
Data is stored and retrieved in units called disk blocks or pages.
Each page has a fixed size, say 512 bytes. It contains a sequence of records.
In many (not always) cases, records in a page have same size, say 100 bytes.
In many (not always) cases, records implement relational tuples.

4. Disks and Files (cont.) How to identify a record? By record id.
Record id = <page id, record position in page>
In many cases, record id can be viewed as a binary number,
High significant bits for page id, and low significant bits for record position in page.
record position in page is the record address relative to the beginning of the page.
For example, assume the disk can hold only 4 pages, each containing 8 records. For record id = we have
page id = 10, and record position in page is 101.
That is, it is the 5th record in the 2nd page.

5. Disks and Files (cont.) How to retrieve a record
Step 1: Fetch the page that contains the record into the main memory
Step 2: Get the record from the page
The cost of step 1 dominates. So the cost of retrieving a record is measured by the number of pages that must be fetched
If the rid is known, then the cost is one.
If something other than rid is known, then more pages may need to be fetched to lead the DBMS to the page that contains the record.
Thus how to minimize the number of pages fetched is vital to the performance (detail later).

6. FILE: A collection of pages, must support: insert/delete/modify record
Files of Records
FILE: A collection of pages, must support:
insert/delete/modify record
read a particular record (specified using record id)
scan all records (possibly with some conditions on the records to be retrieved) 13

7. Alternative File Organizations
Many alternatives exist, each ideal for some situation , and not so good in others:
Heap files:
Records in file have no particular order
Suitable when typical access is a file scan retrieving all records.
Sorted Files:
Records are sorted on some fields.
Best if records must be retrieved in some order, or only a `range’ of records is needed.
Hashed Files: (later) 2

8. Indexes
An index on a file speeds up selections on the search key fields for the index.
Any subset of the fields of a relation can be the search key for an index on the relation.
Search key is not the same as key (minimal set of fields that uniquely identify a record in a relation).
An index contains a collection of data entries
A data entry is denoted as k*, where k is a search key value and * tells where to find the record containing k
Index must support efficient retrieval of all data entries k* with a given key value k
Structure of data entry in more detail 7

9. Alternatives for Data Entry k* in Index
Two alternatives:
<k, rid of data record with search key value k> (often used)
<k, list of rids of data records with search key value k>
Examples, assuming field ‘name’ is the search key
<“John Smith”, 10101> where is the rid of a record that contains “John Smith”
<“John Smith”, 10101, 10111, 11010> where 10101, 10111, are records which all contain “John Smith”. 8

10. How to use an index
We want to find a data record that contains the search key value k
Step 1: find the data entry k* in the index file
Step 2: follow the pointers in the data entry k* to retrieve all the data records that contain k
Key point: the index file is built in such a way that the data entry k* can be found much faster than the data records can.

11. Index Classification
Primary vs. secondary: If search key contains primary key, then called primary index, otherwise secondary.
Unique index: Search key contains a candidate key.
Clustered vs. unclustered: If both the data entries and the data records are sorted on the search key values
A file can be clustered on at most one search key.
Cost of retrieving data records through index varies greatly based on whether index is clustered or not! 11

12. Examples of Indexes (name is primary key in data file)
name age salary
0101
0111
1101
1010
1011
1001
,3000
,5004
,4003
,2007
Ashby
Cass
Smith
Ashby,25,3000
Basu, 33, 4003
Bristow,29,2007
Smith, 44, 3000
Jones, 40, 6003
Tracy, 44, 5004
Ashby, 25, 3000
Basu, 33, 4003
Bristow,29,2007
,6003
0110
Cass, 50, 5004
Daniels,22,6003
Jones, 40, 6003
1110
Cass,50, 5004
Daniels,22,6003
Smith, 44, 3000
Tracy, 44, 5004
Index on search key ‘salary’
Class: unclustered,
secondary
Index on search
Key ‘name’
Class: clustered
primary
Heap file
Sorted file on
name field
This index uses alternative 2
This index uses alternative 1

13. Examples of Indexes (name is the primary key in data file)
name age salary
name age salary
Ashby
Basu
Bristow
Cass 22 33 44
Daniels,22,6003
Ashby,25,3000
Bristow,29,2007
Smith, 44, 3000
Jones, 40, 6003
Tracy, 44, 5004
Basu, 33, 4003
Jones, 40, 6003
Smith, 44, 3000
Ashby, 25, 3000
Basu, 33, 4003
Bristow,29,2007
Daniels
Jones
Smith
Tracy
Cass,50, 5004
Daniels,22,6003
Tracy, 44, 5004
Cass, 50, 5004
Index on search key ‘name’
Class: unclustered,
primary
Index on search
Key ‘age’
Class: clustered
secondary
Heap file
Sorted file on
age field
This index uses alternative 1
This index uses alternative 1

14. Index Classification (Contd.)
Index 2 on search key ‘age’
Dense vs. Sparse: If there is at least one data entry per search key value (in some data record), then dense.
Every sparse index is clustered!
Sparse indexes are smaller; however, some useful optimizations are based on dense indexes.
Index 1 on
Search key
‘name’
name age salary
Ashby, 25, 3000
Smith, 44, 3000
Data File sorted on name
Bristow, 30, 2007
Basu, 33, 4003
Cass, 50, 5004
Tracy, 44, 5004
Daniels, 22, 6003
Jones, 40, 6003
Ashby
Cass
Smith 40 44 50 22 25 30 33
Sparse Index
Dense Index 13

15. Index Classification (Contd.)
Composite Search Keys: Search on a combination of fields.
Equality query: Every field value is equal to a constant value. E.g. wrt <sal,age> index:
age=20 and sal =75
Range query: Some field value is not a constant. E.g.:
age =20; or age=20 and sal > 10
Data entries in index sorted by search key to support range queries.
Examples of composite key
indexes using lexicographic order.
11,80 11
12,10 12
name
age
sal
12,20 12
13,75
bob 12 10 13
<age, sal>
cal 11 80
<age>
joe 12 20
10,12
sue 13 75 10
20,12
Data records
sorted by name 20
75,13 75
80,11 80
<sal, age>
<sal>
Data entries in index
sorted by <sal,age>
Data entries
sorted by <sal> 13

16. Using index to speed up query processing
Consider the basic SQL structure:
SELECT S.name
FROM Sailors S
WHERE S.rating = 7
If there is no index on rating field, then we must scan all the tuples in Sailors relation
If there is an index file with search key rating, then
Perform binary search to find all the data entries with rating = 7 in the index file
For each data entry found, follow the coupling rid(s) to retrieve all the data record containing rating = 7
For each data record retrieved, output the sailor’s name
This is must faster than if there is no index on rating field