File Organization Lecture 1 Introduction to the Design and Specification of File Structures File Organization

Previous Lecture

Lecture 0 Course Outline Course Aims and Objectives. Course Contents. Course Textbook and Schedule. Course Link http://hedar.info/Courses/INF211/

Today Lecture

Lecture Objectives Introduce the primary design issues that characterize file structure design. Survey the history of file structure. Introduce the notions of file structure literacy and of a conceptual toolkit for file structure design. Discuss the need for precise specification of data structure and operations Develop an object-oriented toolkit that makes file structure easy to use.

Lecture Contents The heart of file structure design. A short history of file structure design. A conceptual toolkit: File structure literacy. An object-oriented toolkit: Making file structure usable.

The heart of file structure design Section 1.1 The heart of file structure design

File Structure Definition & Functions A combination of representations for data in files and of operations for accessing the data. Functions Allowing applications to read, write and modify data. It might also support finding the data that matches some search criteria reading through the data in some particular order.

File Structure Design: Need of Study Data Storage Computer Data can be stored in 3 kinds of locations: Primary Storage Memory Secondary Storage Tertiary Storage Archival Data Offline Disk Tape CD Rom “not accessed by the computer” Online Disk Tape CD Rom “accessed by the computer” Computer Memory

File Structure Design: Need of Study Memory versus Secondary Storage Secondary storage such as disks can pack 1000’s of megabytes in a small physical location. Computer Memory (RAM) is limited. Comparing to Memory, access to secondary storage is extremely slow. Getting information from slow RAM takes 120. 10-9 seconds (= 120 nanoseconds) while getting information from Disk takes 30. 10-3 seconds (= 30 milliseconds) Roughly, 20 second on RAM ≈ 58 days on Disk

representation of the data the implementation of the operations File Structure Design: Need of Study Improve Secondary Storage Access Time representation of the data the implementation of the operations ⇒ the efficiency of the file structure for particular applications

File Structure Design General Goals Get the information we need with one access to the disk. If that’s not possible, then get the information with as few accesses as possible. Group information so that we are likely to get everything we need with only one trip to the disk.

File Structure Design Fixed versus Dynamic Files It is relatively easy to come up with file structure designs that meet the general goals when the files never change. When files grow or shrink when information is added or deleted, it is much more difficult.

A short history of file structure design Section 1.2 A short history of file structure design

Early Work Early Work assumed that files were on tape. Access was sequential and the cost of access grew in direct proportion to the size of the file.

The emergence of Disks and Indexes As files grew very large, unaided sequential access was not a good solution. Disks allowed for direct access. Indexes made it possible to keep a list of keys and pointers in a small file that could be searched very quickly. With the key and pointer, the user had direct access to the large, primary file.

The emergence of Tree Structures As indexes also have a sequential flavor, when they grew too much, they also became difficult to manage. The idea of using tree structures to manage the index emerged in the early 60’s. Trees can grow very unevenly as records are added and deleted resulting in long searches requiring many disk accesses to find a record.

Balanced Trees In 1963, researchers came up with the idea of AVL trees for data in memory. heights of the two child subtrees of any node differ by at most one. named after its two inventors, G.M. Adelson-Velsky and E.M. Landis AVL trees, however, did not apply to files because they work well when tree nodes are composed of single records rather than dozens or hundreds of them.

Balanced Trees In the 1970’s came the idea of B-Trees In B-trees, internal nodes can have a variable number of child nodes within some pre-defined range. When data is inserted or removed from a node, its number of child nodes changes. B-Trees can guarantee that one can find one file entry among millions of others with only 3 or 4 trips to the disk.

Hash Tables Retrieving entries in 3 or 4 accesses is good, but it does not reach the goal of accessing data with a single request. From early on, Hashing was a good way to reach this goal with files that do not change size greatly over time. Recently, Extendible Dynamic Hashing guarantees one or at most two disk accesses no matter how big a file becomes.

A conceptual toolkit: File structure literacy Section 1.3 A conceptual toolkit: File structure literacy

Conceptual tools For File Structure Design Tree Structure Direct Access Sequentially Decrease the number of disk accesses by collecting data into buffers, blocks, or buckets. Manage their growth by splitting them. Find a way to increase our address or index space. Find new ways to combine the basic tools.

An object-oriented toolkit: Making file structure usable Section 1.4 An object-oriented toolkit: Making file structure usable

Object-Oriented Toolkit For File Structure Design Making file structure usable needs application programming interfaces. Invoking an object-oriented approach Data types and operations are defined as classes.

Object-Oriented Toolkit Difficulties Describing the classes for file structure design are: Progressive New classes are often modifications or extensions of other classes. Complicated The details of the data representations and operations become more complex.

Next Lecture

Fundamental File Processing Operations Physical and logical file. Opening and closing files. Reading and writing. Seeking. Special Characters in files. Physical devices and logical files. File-related header files.

Questions?