1. File StructureSNU-OOPSLA Lab1 서울대학교 컴퓨터공학부 객체지향시스템연구실 SNU-OOPSLA-LAB 김 형주 교수 Chap 5. Managing Files of Records File Structures by Folk, Zoellick, and Ricarrdi

2. File StructureSNU-OOPSLA Lab2 Chapter Objectives u Extend the file structure concepts of Chapter 4: u Search keys and canonical forms u Sequential search and Direct access u Files access and file organization u Examine other kinds of the file structures in terms of u Abstract data models u Metadata u Object-oriented file access u Extensibility u Examine issues of portability and standardization.

3. File StructureSNU-OOPSLA Lab3 Contents 5.1 Record Access 5.2 More about Record Structures 5.3 Encapsulating Record I/O Ops in a Single Class 5.4 File Access and File Organization 5.5 Beyond Record Structures 5.6 Portability and Standardization

4. File StructureSNU-OOPSLA Lab4 5.1 Record Access Record Access u Record Key u Canonical form : a standard form of a key u e.g. Ames or ames or AMES (need conversion) u Distinct keys : uniquely identify a single record u Primary keys, Secondary keys, Candidate keys u Primary keys should be dataless (not updatable) u Primary keys should be unchanging u Social-securiy-number: good primary key u but, 999-99-9999 for all non-registered aliens

5. File StructureSNU-OOPSLA Lab5  O (n), n : the number of records  Use record blocking A block of several records fields < records < blocks O(n), but blocking decreases the number of seeking e.g.- 4000 records, 512 bytes length Unblocked (sector-sized buffers): 512byte size buffer => average 2000 READ() calls Blocked (16 recs / block) : 8K size buffer ==> average 125 READ() call 5.1 Record Access Sequential Search (1)

6. File StructureSNU-OOPSLA Lab6 5.1 Record Access Sequential Search (2) u UNIX tools for sequential processing u cat, wc, grep u When sequential search is useful u Searching patterns in ASCII files u Processing files with few records u Searching records with a certain secondary key value

7. File StructureSNU-OOPSLA Lab7 5.1 Record Access Direct Access u O(1) operation u RRN ( Relative Record Number ) u It gives relative position of the record u Byte offset = N X R u r : record size, n : RRN value u In fixed length records u Class IOBuffer includes u direct read (DRead) u direct write (DWrite)

8. File StructureSNU-OOPSLA Lab8 Record length is related to the size of the fields Access vs. fragmentaion vs. implementation Fixed length record (a) With a fixed-length fields (b) With a variable-length fields Unused space portion is filled with null character in C Ames John 123 Maple Stillwater OK74075 Mason Alan 90 Eastgate Ada OK74820 Ames|John|123 Maple|Stillwater|OK|74075| Mason|Alan|90 Eastgate|Ada|OK|74820| Unused space (a) (b) 5.2 More about Record Structures Choosing a record length and structure

9. File StructureSNU-OOPSLA Lab9 5.2 More about Record Structures Header Records u General information about file u date and time of recent update, count of the num of records u Header record is often placed at the beginning of the file u Header records are a widely used, important file design tool u Pascal does not naturally support header records (File is a repeated collection of the same type) u Use variant records (depending on context)

10. File StructureSNU-OOPSLA Lab10 IO Buffer Class definition(1) 5.2 More about Record Structures class IOBuffer Abstract base class for file buffers public : virtual int Read( istream & ) = 0; // read a buffer from the stream virtual int Write( ostream &) const = 0; // write a buffer to the stream // these are the direct access read and write operations virtual int DRead( istream &, int recref ); //read specified record virtual int DWrite( ostream &, int recref ) const; // write specified record // these header operations return the size of the header virtual int ReadHeader ( istream & ); virtual int WriteHeader ( ostream &) const; protected : int Initialized ; // TRUE if buffer is initialized char *Buffer; // character array to hold field values

11. File StructureSNU-OOPSLA Lab11 u The full definition of buffer class hierarchy u write method : adds header to a file and return the number of bytes in the header u read method : reads the header and check for consistency u WriteHeader method : writes the string IOBuffer at the beginning of the file. u ReadHeader method : reads the record size from the header and checks that its value is the same as that of the BufferSize member of the buffer object u DWrite/DRead methods : operates using the byte address of the record as the record reference. Dread method begins by seeking to the requested spot. 5.2 More about Record Structures IO Buffer Class definition(2)

12. File StructureSNU-OOPSLA Lab12 Encapsulation Record I/O Ops in a Single Class(1) u Good design for making objects persistent u provide operation to read and write objects directly u Write operation until now : u two operation : pack into a buffer + write the buffer to a file u Class ‘RecordFile’ u supports a read operation that takes an object of some class and writes it to a file. u the use of buffers is hidden inside the class u problem with defining class ‘RecordFile’: u how to make it possible to support files for different object types without needing different versions of the class 5.3 Encapsulating Record I/O Operations in a Single Class

13. File StructureSNU-OOPSLA Lab13 template class RecordFile : public BufferFile { public: int Read(RecType& record, int recaddr = -1); int Write(const RecType& record, int recaddr = -1 ); RecordFile(IOBuffer& buffer) : BufferFile(buffer) { } }; u Class ‘RecordFile’ u uses C++ template features to solve the problem u definition of the template class RecordFile 5.3 Encapsulating Record I/O Operations in a Single Class Encapsulation Record I/O Operation in a Single Class(2)

14. File StructureSNU-OOPSLA Lab14 // template method bodies template int RecordFile ::Read (RecType & record, int recaddr = -1) { int writeAdd, result; writeAddr = BufferFile::Read (recaddr); if (!writeAddr) return -1; result = record.Unpack(Buffer); if (!result) return -1; return writeAddr; } template int RecordFile ::Write (const RecType & record, int recaddr = -1) { int result; result = record. Pack (Buffer); if (!result) return -1; return BufferFile::Write (recaddr); }

15. File StructureSNU-OOPSLA Lab15 There is difference between file access and file organization. Variable-length records Sequential access is suitable Fixed-length records Direct access and sequential access are possible 5.4 File Access and File Organization File Organization File Access Variable-length Records Sequential access Fixed-length records Direct access File Access and File Organization

16. File StructureSNU-OOPSLA Lab16 5.5 Beyond Record Structures Abstract Data Model u Data object such as document, images, sound u e.g. color raster images, FITS image file u Abstract Data Model does not view data as it appears on a particular medium. ->application-oriented view u Headers and Self-describing files

17. File StructureSNU-OOPSLA Lab17 Metadata u Data that describe the primary data in a file u A place to store metadata in a file is the header record u Standard format u FITS (Flexible Image Transport System) by International Astronomers’ union (see Figure 5.7) 5.5 Beyond Record Structures

18. File StructureSNU-OOPSLA Lab18 Mixing object Types in a file u Each field is identified using “keyword = value” u Index table with tags u e.g. 5.5 Beyond Record Structures

19. File StructureSNU-OOPSLA Lab19 Object-oriented file access u Separate translating to and from the physical format and application (representation-independent file access) 5.5 Beyond Record Structures RAM image : star1star2 Disk Program find_star : read_image(“star1”, image) process image : end find_star

20. File StructureSNU-OOPSLA Lab20 Extensibility u Advantage of using tags u Identify object within files is that we do not have to know a priori what all of the objects will look like u When we encounter new type of object, we implement method for reading and writing that object and add the method. 5.5 Beyond Record Structures

21. File StructureSNU-OOPSLA Lab21 Factor affecting Portability Factor affecting Portability u Differences among operating system u Differences among language u Differences in machine architecture u Differences on platforms u EBCDIC and ASCII 5.6 Portability and Standardization

22. File StructureSNU-OOPSLA Lab22 Achieving Portability (1) u Standardization u Standard physical record format u extensible, simple u Standard binary encoding for data elements u IEEE, XDR u File structure conversion u Number and text conversion 5.6 Portability and Standardization

23. File StructureSNU-OOPSLA Lab23 Achieving Portability (2) u File system difference u Block size is 512 bytes on UNIX systems u Block size is 2880 bytes on non-UNIX systems u UNIX and Portability u UNIX support portability by being commonly available on a large number of platforms u UNIX provides a utility called dd u dd : facilitates data conversion 5.6 Portability and Standardization

24. File StructureSNU-OOPSLA Lab24 Basic Files:Sequential File(1) u Organized by appending records to the file in the order they arrive u First record id oldest, last record is newest u Records either fixed or variable length u Most useful for magnetic tape storage u Sometimes called, “sequential-chronological” file Addition : Basic Files

25. File StructureSNU-OOPSLA Lab25 Basic Files: Sequential File (2) Sequential Files with fixed length records Addition : Basic Files

26. File StructureSNU-OOPSLA Lab26 Basic Files:Relative Files(1) u Records can be accessed directly by using record number 0,..., N-1 as external keys u Must use random or pseudorandom access device such as disk u With fixed-length records, the file system can easily compute the sector address of a block containing a certain record when given the record number, especially when the file is stored contiguously(in successive sectors) Addition : Basic Files

27. File StructureSNU-OOPSLA Lab27 Basic Files:Relative Files(2) u Assume the file system marks all record positions(slots) as empty when the file is create, and enforce the requirement that no record can be read before it has been written Relative Files Addition : Basic Files

28. File StructureSNU-OOPSLA Lab28 Basic Files : Ordered Sequential File & Ordered Relative File Addition : Basic Files

29. File StructureSNU-OOPSLA Lab29 Let’s Review !!! 5.1 Record Access 5.2 More about Record Structures 5.3 Encapsulating Record I/O Ops in a Single Class 5.4 File Access and File Organization 5.5 Beyond Record Structures 5.6 Portability and Standardization