1. CENG 351 Introduction to Data Management and File Structures
Nihan Kesim Çiçekli
Department of Computer Engineering
METU
CENG 351

2. CENG 351 Instructor: Nihan Kesim Çiçekli Office: A308
Lecture Hours:
Section 1: Mon. 13:40, 14:40 (BMB4); Thu. 10:40 (BMB4)
Section 2: Wed. 9:40, 10:40 (BMB5); Thu. 11:40 (BMB1)
Course Web page:
Teaching Assistants:
Emre Işıklıgil Office: A402
Alev Mutlu Office: A302
Abdullah Doğan Office: A206
CENG 351

3. References
Raghu Ramakrishnan, Database Management Systems (3rd. ed.), McGraw Hill, 2003.
R. Elmasri, S.B. Navathe, Fundamentals of Database Systems, 4th edition, Addison-Wesley, 2004.
CENG 351

4. Course Outline Introduction to relational database systems
Relational Model and E/R Modeling
Relational Algebra, Relational Calculus
Structural Query language (SQL)
Secondary Storage Media
Fundamental File Structure Concepts
Sequential File Processing
External Sorting of Large Files
Indexing: Multilevel Indexing and B+ trees
Hashing (static, linear, extendible hashing)
SQL Query Evaluation and optimization issues
CENG 351

5. Grading Assignments 25% Attendence/Quiz 5% Midterm Exam 1 20%
Final %
Tentative Exam Dates:
Midterm Exam 1: Nov. 15, 2012
Midterm Exam 2: Dec. 20, 2012
CENG 351

6. Grading Policies Policy on missed midterm: Lateness policy:
no make-up exam
Lateness policy:
Every student has a total of 5 days for late submission for assignments. One can spend this credit for any of the assignments or distribute it for all. If total of late submissions exceeds the limit, a penalty of 10*day*day is applied.
All assignments and programs are to be your own work. No group projects or assignments are allowed.
CENG 351

7. Introduction to File management
CENG 351

8. Motivation
Most computers are used for data processing. A big growth area in the “information age”
This course covers data processing from a computer science perspective:
Storage of data
Organization of data
Access to data
Processing of data
CENG 351

9. Data Structures vs File Structures
Both involve:
Representation of Data +
Operations for accessing data
Difference:
Data structures: deal with data in main memory
File structures: deal with data in secondary storage
CENG 351

10. Where do File Structures fit in Computer Science?
Application
DBMS
File system
Operating System
Hardware
CENG 351

11. Computer Architecture
data is manipulated here
- Semiconductors
- Fast, expensive, volatile, small
Main Memory (RAM)
data transfer
Secondary
Storage
- disks, tape
- Slow,cheap, stable, large
data is stored here
CENG 351

12. Advantages Disadvantages
Main memory is fast
Secondary storage is big (because it is cheap)
Secondary storage is stable (non-volatile) i.e. data is not lost during power failures
Disadvantages
Main memory is small. Many databases are too large to fit in main memory (MM).
Main memory is volatile, i.e. data is lost during power failures.
Secondary storage is slow (10,000 times slower than MM)
CENG 351

13. How fast is main memory? Typical time for getting info from:
Main memory: ~12 nanosec = 120 x 10-9 sec
Magnetic disks: ~30 milisec = 30 x 10-3 sec
An analogy keeping same time proportion as above:
Looking at the index of a book : 20 sec
versus
Going to the library: 58 days
CENG 351

14. Normal Arrangement
Secondary storage (SS) provides reliable, long-term storage for large volumes of data
At any given time, we are usually interested in only a small portion of the data
This data is loaded temporarily into main memory, where it can be rapidly manipulated and processed.
As our interests shift, data is transferred automatically between MM and SS, so the data we are focused on is always in MM.
CENG 351

15. Goal of the file structures
Minimize the number of trips to the disk in order to get desired information
Grouping related information so that we are likely to get everything we need with only one trip to the disk.
CENG 351

16. Physical Files and Logical Files
physical file: a collection of bytes stored on a disk or tape
logical file: a "channel" (like a telephone line) that connects the program to a physical file
The program (application) sends (or receives) bytes to (from) a file through the logical file. The program knows nothing about where the bytes go (came from).
The operating system is responsible for associating a logical file in a program to a physical file in disk or tape. Writing to or reading from a file in a program is done through the operating system.
CENG 351

17. Files The physical file has a name, for instance myfile.txt
The logical file has a logical name (a varibale) inside the program.
In C :
FILE * outfile;
In C++:
fstream outfile;
CENG 351

18. Basic File Processing Operations
Opening
Closing
Reading
Writing
Seeking
CENG 351

19. File Systems Data is not scattered hither and thither on disk.
Instead, it is organized into files.
Files are organized into records.
Records are organized into fields.
CENG 351

20. Example
A student file may be a collection of student records, one record for each student
Each student record may have several fields, such as
Name
Address
Student number
Gender
Age
GPA
Typically, each record in a file has the same fields.
CENG 351

21. Properties of Files
Persistance: Data written into a file persists after the program stops, so the data can be used later.
Sharability: Data stored in files can be shared by many programs and users simultaneously.
Size: Data files can be very large. Typically, they cannot fit into main memory.
CENG 351

22. Introduction to Database Systems
Ref. Ramakrishnan & Gehrke Chapter 1
The slides for this text are organized into chapters. This lecture covers Chapter 1.
Chapter 1: Introduction to Database Systems
Chapter 2: The Entity-Relationship Model
Chapter 3: The Relational Model
Chapter 4 (Part A): Relational Algebra
Chapter 4 (Part B): Relational Calculus
Chapter 5: SQL: Queries, Programming, Triggers
Chapter 6: Query-by-Example (QBE)
Chapter 7: Storing Data: Disks and Files
Chapter 8: File Organizations and Indexing
Chapter 9: Tree-Structured Indexing
Chapter 10: Hash-Based Indexing
Chapter 11: External Sorting
Chapter 12 (Part A): Evaluation of Relational Operators
Chapter 12 (Part B): Evaluation of Relational Operators: Other Techniques
Chapter 13: Introduction to Query Optimization
Chapter 14: A Typical Relational Optimizer
Chapter 15: Schema Refinement and Normal Forms
Chapter 16 (Part A): Physical Database Design
Chapter 16 (Part B): Database Tuning
Chapter 17: Security
Chapter 18: Transaction Management Overview
Chapter 19: Concurrency Control
Chapter 20: Crash Recovery
Chapter 21: Parallel and Distributed Databases
Chapter 22: Internet Databases
Chapter 23: Decision Support
Chapter 24: Data Mining
Chapter 25: Object-Database Systems
Chapter 26: Spatial Data Management
Chapter 27: Deductive Databases
Chapter 28: Additional Topics

23. Basic Definitions Data Database Database Management System (DBMS)
Database System

24. Basic Definitions
Data: Known facts that can be recorded and have an implicit meaning.
Database: A collection of related data.
Database Management System (DBMS): A software package/ system to facilitate the creation and maintenance of a computerized database.
Database System: The DBMS software together with the data itself. Sometimes, the applications are also included.

25. Files vs. DBMS
Application must stage large datasets between main memory and secondary storage (e.g., buffering, page-oriented access, etc.)
Special code for different queries
Must protect data from inconsistency due to multiple concurrent users
Crash recovery
Security and access control

26. Typical DBMS Functionality
Define a database : in terms of data types, structures and constraints
Construct or load the database on a secondary storage medium
Manipulating the database : querying, generating reports, insertions, deletions and modifications to its content
Concurrent Processing and Sharing by a set of users and programs – yet, keeping all data valid and consistent

27. Data Models
A data model is a collection of concepts for describing data.
A schema is a description of a particular collection of data, using the given data model.
The relational model of data is the most widely used model today.
Main concept: relation, basically a table with rows and columns.
Every relation has a schema, which describes the columns, or fields. 5

28. Example: University Database
Conceptual schema:
Students(sid: string, name: string, login: string,
age: integer, gpa:real)
Courses(cid: string, cname:string, credits:integer)
Enrolled(sid:string, cid:string, grade:string)
Physical schema:
Relations stored as unordered files.
Index on first column of Students.
External Schema (View):
Course_info(cid:string,enrollment:integer) 7

29. Instance of Students Relation
Students( sid: string, name: string, login: string,
age: integer, gpa: real )
sid name login age gpa
53666 Jones
53688 Smith
53650 Smith

30. Levels of Abstraction
Many external schemata, single conceptual(logical) schema and physical schema.
External schemata describe how users see the data.
Conceptual schema defines logical structure
Physical schema describes the files and indexes used.
External
Schema 1
External
Schema 2
External
Schema 3
Conceptual Schema
Physical Schema
Schemas are defined using DDL; data is modified/queried using DML. 6

31. Data Independence
Applications insulated from how data is structured and stored.
Logical data independence: Protection from changes in logical structure of data.
Physical data independence: Protection from changes in physical structure of data.
One of the most important benefits of using a DBMS!

32. Files and Access Methods
These layers
must consider
concurrency
control and
recovery
Structure of a DBMS
A typical DBMS has a layered architecture.
This is one of several possible architectures; each system has its own variations.
Query Optimization
and Execution
Relational Operators
Files and Access Methods
Buffer Management
Disk Space Management DB 22