Adv. UNIX: lowIO/161 Advanced UNIX v Objectives –look at low-level operations for handling files Special Topics in Comp. Eng. 2 Semester 2, Low-level File I/O

Adv. UNIX: lowIO/162 Overview 1. Basic Operations 2. Why use Low-level Operations? 3. A Simple Example 4. File Descriptors 5. File Permissions Again continued

Adv. UNIX: lowIO/ open() 12. lseek() 7. creat() 13. Hotel Guests 8. close() 14. unlink() 9. read() 15. fcntl() 10. write() 16. Duplicating FDs 11. copyfile 17. File Pointers

Adv. UNIX: lowIO/ Basic Operations  NameMeaning open() Opens a file for reading or writing. creat() Creates an empty file for writing. close() Closes an open file. read() Reads data from a file. write() Writes data to a file. lseek() Moves to a specified byte in file. unlink() Removes a file.

Adv. UNIX: lowIO/ Why use Low-level Operations? v No buffering of I/O –useful for network programming and reading/writing to certain peripherals (e.g. tape drives) v No conversion of I/O –no transformation of input to integers, floats, etc.; output can be any byte sequence v Building blocks for more complex I/O.

Adv. UNIX: lowIO/ A Simple Example: oprd.c #include #include #include #include #include void main() { int fd, nread; char buf[1024]; if ((fd = open("data-file", O_RDONLY)) < 0) { perror("open"); exit(-1); } nread = read(fd, buf, 1024); close(fd); buf[nread] = '\0'; /* so can print */ printf("buf: %s\nnread: %d\n", buf, nread); }

Adv. UNIX: lowIO/ File Descriptors v A file descriptor is a small, non-negative integer. v A file descriptor identifies an open file to the low-level operations. v Standard descriptors: 0represents stdin 1 stdout 2 stderr

Adv. UNIX: lowIO/ File Permissions Again  Earlier chmod examples used letters to represent permissions: –e.g. chmod a+r file  Most low-level operations (and chmod ) represent permissions as octal values: –e.g. chmod 0644 file

Adv. UNIX: lowIO/169 File Permissions as Octals OctalSymbolic Meaning of Permission 0400r w x r w x r w x

Adv. UNIX: lowIO/1610 Example  Add octal values together to get the complete file permission: OctalMeaning 0400r w r r-- =============== 0644rw- r-- r--  Use in chmod : chmod 0644 file

Adv. UNIX: lowIO/ open()   #include #include #include int open(char *pathname, int flag /*, mode_t mode */ ); v Return file descriptor if ok, -1 for error an octal value (as in chmod), used when creating a file

Adv. UNIX: lowIO/1612 Some open() Flags O_RDONLY Open file for reading only. O_WRONLY Open file for writing only. O_RDWR Open file for reading & writing. Possibly combined with: O_APPEND Append to file when writing. O_CREAT Create file if is does not exist (requires the 3rd mode argument). O_EXCL Return -1 error if file is to be created and already exists.

Adv. UNIX: lowIO/1613 Using Bitwise OR  open() flags can be combined with the bitwise OR operator ‘|”: fd = open(file, O_WRONLY | O_APPEND);  Now each write(fd, buf, BUFSIZE) means append to the end of the file.

Adv. UNIX: lowIO/ creat()  #include #include #include int creat(char *pathname, mode_t mode); v Creates a new file, or truncates an old one. v Return file descriptor if ok, -1 on error.

Adv. UNIX: lowIO/1615  mode gives access permission of resulting new file: 0644(rw- r-- r--)  mode only has meaning if the file is new –so it cannot be used to change the mode of an existing file

Adv. UNIX: lowIO/1616 Create a new file #include #include #include #include void main() { int fd; if ((fd = creat(“new-file”,0644)) #include #include #include void main() { int fd; if ((fd = creat(“new-file”,0644)) < 0){ printf(“cannot create new-file\n”); exit(1); } /* rest of program */

Adv. UNIX: lowIO/1617 Replacing creat() with open()  creat() only opens a file for writing; to read it the file must be closed and opened for reading.  creat() can be replaced by open() : open(“new-file”, O_WRONLY | O_CREAT | O_TRUNC, 0644)

Adv. UNIX: lowIO/1618 Other Versions  creat() version which can read and write: open(“new-file”, O_RDWR | O_CREAT | O_TRUNC, 0644)  Avoid truncation of old file (append instead): open(“new-file”, O_WRONLY|O_CREAT|O_APPEND, 0644)

Adv. UNIX: lowIO/1619 One User at a Time  Create lock if it doesn’t already exist, otherwise fail (and return -1): fd = open(“lock”, O_WRONLY|O_CREAT|O_EXCL, 0644);  Use lock to protect access to another file –open “ accounts ” only if lock can be opened –after processing “ accounts ”, delete lock

Adv. UNIX: lowIO/1620 Diagram of Locking "lock" file "accounts" file : if ((fd = open("lock", O_WRONLY | O_CREAT | O_EXCL, 0644)) < 0)) do nothing else { open "accounts" : close "accounts" delete "lock" } program 1 : if ((fd = open("lock", O_WRONLY | O_CREAT | O_EXCL, 0644)) < 0)) do nothing else { open "accounts" : close "accounts" delete "lock" } program 2

Adv. UNIX: lowIO/ close()  #include int close(int fd); v Close a file: return 0 if ok, -1 on error. v Useful since the number of open files is limited (~20)

Adv. UNIX: lowIO/ read()  #include int read(int fd, void *buffer, unsigned int nbytes);  Attempts to read nbytes into the buffer array –no conversion of bytes (characters) –may read less than specified amount (e.g. at end) v Returns the number of bytes read, 0 if end of file, -1 on error.

Adv. UNIX: lowIO/1623 The Read-Write Pointer v The R-W Pointer records the position of the next byte in the file to be read (or written).  It is implicit (hidden) in each use of read() and write() 11+n read(..., n)

Adv. UNIX: lowIO/1624 Count Characters: countchars.c #include #include #include #include #include #define SIZE 512 /* block of chars read at a time */ int main() { char buffer[SIZE]; int fd, j; : continued

Adv. UNIX: lowIO/1625 long total = 0; if ((fd = open(“foo”, O_RDONLY)) 0) total += j; printf(“total chars: %ld\n”, total); close(fd); return 0; }

Adv. UNIX: lowIO/1626 v UNIX systems are often configured to read/write in blocks of 512 or 1024 bytes (the disk blocking factor).  Can use BUFSIZ from stdio.h which contains a default disk blocking factor. Notes

Adv. UNIX: lowIO/ write()  #include int write(int fd, void *buffer, unsigned int nbytes); v Returns number of bytes written if ok, -1 on error.  write() starts at the current R-W pointer position, and pointer is moved as file is written.

Adv. UNIX: lowIO/1628 Example : int fd; char header1[512], header2[512]; : if ((fd = creat(“foo”, 0644)) != -1) : write(fd, header1, 512); write(fd, header2, 512); :

Adv. UNIX: lowIO/ copyfile #include #include #include #include #include int copyfile(char *nm1, char *nm2); int main() { int result; result = copyfile(“test.in”, “test.out”); printf("Result code of copy: %d\n", result); return 0; } continued

Adv. UNIX: lowIO/1630 int copyfile(char *nm1, char *nm2) { int infd, outfd, nread; char buffer[BUFSIZ]; if ((infd = open(nm1,O_RDONLY))< 0) return -1; if ((outfd = creat(nm2,0644)) < 0){ close(infd); return -2; } : continued

Adv. UNIX: lowIO/1631 while ((nread = read(infd, buffer,BUFSIZ)) > 0) { if (write(outfd, buffer, nread) < nread) { close(infd); close(outfd); return -3; /* write error */ } } close(infd); close(outfd); return 0; }

Adv. UNIX: lowIO/1632 copyfile() & BUFSIZ v BUFSIZ Real timeUser timeSystem time 13: : : : : : : : : : : : : :01.1 v Most saving is made by reducing no. of system calls. $ time copyfile

Adv. UNIX: lowIO/ lseek() v #include #include long int lseek(int fd, long int offset, int start); v Change position of R-W pointer: return new file offset if ok, -1 on error. v Start names (and numbers): –SEEK_SET 0start of file –SEEK_CUR 1current R-W pointer posn –SEEK_END 2end of file

Adv. UNIX: lowIO/1634 Examples   : fd = open(filename, O_RDWR); lseek(fd, 0L, SEEK_END); write(fd, buffer, BUFSIZ); : v v /* get filesize in bytes */ long int filesize; filesize = lseek(fd, 0L, SEEK_END);

Adv. UNIX: lowIO/ Hotel Guests  The guests file has the format:..... name of guest, ending in '\n'; always 41 chars

Adv. UNIX: lowIO/1636 Task v Write the function: int get_guest(int fd, int room_num, char *name);  Return 1 if the guest in room_num is found ( name is also bound); 0 otherwise.

Adv. UNIX: lowIO/1637 Code #include #include #include #include #include #define NAMELEN 41 #define NROOMS 5000 : continued

Adv. UNIX: lowIO/1638 int main() { int fd, i; char name[NAMELEN]; if ((fd = open(“guests”,O_RDONLY))== -1){ fprintf(stderr,“Cannot read guests\n”); exit(1); } for (i=1; i <= NROOMS; i++) if (get_guest(fd, i, name) == 1) printf(“Room %d, %s\n”, i, name); else printf(“Error in room %d\n”, i); return 0; } continued

Adv. UNIX: lowIO/1639 int get_guest(int fd, int rnum, char *nm) { long int offset; int nread; offset = (rnum-1)*NAMELEN; if (lseek(fd, offset, SEEK_SET) < 0) return 0; nread = read(fd, nm, NAMELEN); if (nread < NAMELEN) return 0; /* read() error */ else nm[nread-1] = ‘\0’; /* overwrite \n */ return 1; }

Adv. UNIX: lowIO/ unlink()  #include int unlink(char *pathname); v Remove a file: return 0 of ok, -1 on error. –e.g. unlink(“/tmp/tmpfile”); v Must have permissions to execute and write in the file’s directory –for the cd and change to the contents

Adv. UNIX: lowIO/ fcntl()  #include #include #include int fcntl(int fd, int cmd, int arg); v Used to view/alter the behaviour of the file.  Two (of 10) cmd values: F_GETFL get current status flags F_SETFL set a status flag

Adv. UNIX: lowIO/1642 Some File Status Flags for fcntl() FlagDescription O_RDONLY Open for reading only (0). O_WRONLY Open for writing only (1). O_RDWR Open for reading and writing (2). O_APPEND Append on each write. :

Adv. UNIX: lowIO/1643 Using O_ACCMODE  O_RDONLY, O_WRONLY and O_RDWR are not separate bits (e.g. not 0, 2, 4), and so their values can be hard to test.  One solution is to use the O_ACCMODE mask.

Adv. UNIX: lowIO/1644 : int accmode, val; if ((val = fcntl(fd, F_GETFL, 0)) < 0) printf(“Some error\n”); else { accmode = val & O_ACCMODE; if (accmode == O_RDONLY) printf(“read”); else if (accmode == O_WRONLY) printf(“write”); else if (accmode == O_RDWR) printf(“read write”); else printf(“access mode error”); if (val & O_APPEND) printf(“, append”); :

Adv. UNIX: lowIO/1645 Using F_SETFL  Main use is to change O_APPEND : if (fcntl(fd, F_SETFL, O_APPEND) < 0) printf(“Setting append failed\n”); else...

Adv. UNIX: lowIO/ Duplicating File Descriptors   #include int dup(int fd);  Make a new file descriptor ( fd2 ) refer to the same file as fd : fd2 = dup(fd) v Useful for concurrent programming, where two processes refer to the same file.

Adv. UNIX: lowIO/1647 Duplicating Diagram file fd 3 fd2 = dup(fd); file fd 3 fd2 4

Adv. UNIX: lowIO/1648 Another Approach   #include int dup2(int fd, int fd2);  Make the existing fd2 descriptor refer to the same file as fd.  If fd2 is already open, close its file before redirecting it.

Adv. UNIX: lowIO/1649  Main use is for redirecting stdio : dup2(fd, 0) v Same as the ‘hack’: close(0); dup(fd);

Adv. UNIX: lowIO/1650 dup2(fd,0) Diagram file fd program Before After file fd program stdin 0

Adv. UNIX: lowIO/ File Pointers  Connect a file pointer to fd ’s file with: #include FILE *fdopen(int fd, char *mode);  mode is usual thing: “ r ”, “ w”, “ rb ”, etc. v Useful for doing formatted I/O on top of pipes and network comunication.