Unix Directories unix etc home pro dev motd snt unix ... slide1 slide2

Slides:



Advertisements
Similar presentations
Operating system services Program execution I/O operations File-system manipulation Communications Error detection Resource allocation Accounting Protection.
Advertisements

January 13, Csci 2111: Data and File Structures Week1, Lecture 2 Basic File Processing Operations.
Read vs. mmap Tan Li. Man mmap #include void *mmap(void *start, size_t length, int prot, int flags, int fd, off_t offset); int munmap(void *start, size_t.
Today’s topic: –File operations –I/O redirection –Inter-process communication through pipes.
Files. System Calls for File System Accessing files –Open, read, write, lseek, close Creating files –Create, mknod.
Introduction to Kernel
1 Advanced programming in UNIX 1 File I/O Hua LiSystems ProgrammingCS2690File I/O.
CS 311 – Lecture 10 Outline Review open() and close() Difference between fopen() and open() File management system calls – read() – write() – lseek() –
CS Lecture 16 Outline Inter-process Communication (IPC) – Pipes – Signals Lecture 161CS Operating Systems 1.
CSE 451 Section 4 Project 2 Design Considerations.
POSIX: Files Introduction to Operating Systems: Discussion 1 Read Solaris System Interface Guide: Ch. 5.1 Basic File I/O.
Recitation 11: I/O Problems Andrew Faulring Section A 18 November 2002.
Adv. UNIX: lowIO/161 Advanced UNIX v Objectives –look at low-level operations for handling files Special Topics in Comp. Eng. 2 Semester.
Fundamentals CIS 552. Fundamentals Low-level I/O (read/write using system calls)  Opening/Creating files  Reading & Writing files  Moving around in.
SIMULATED UNIX FILE SYSTEM Implementation in C Tarek Youssef Bipanjit Sihra.
Cli/Serv.: procs/51 Client/Server Distributed Systems v Objectives –look at how to program UNIX processes , Semester 1, Processes.
Chapter 4. INTERNAL REPRESENTATION OF FILES
File System Review bottomupcs.com J. Kubiatowicz, UC Berkeley.
Operating Systems Recitation 1, March th, 2002.
Shell (Part 2). Example r What if we want to support something like this: m ps –le | sort r One process should execute ps –le and another should execute.
Recitation 11: 11/18/02 Outline Robust I/O Chapter 11 Practice Problems Annie Luo Office Hours: Thursday 6:00 – 7:00 Wean.
Pipes A pipe is a simple, synchronized way of passing information between processes A pipe is a special file/buffer that stores a limited amount of data.
CS252: Systems Programming Ninghui Li Based on Slides by Prof. Gustavo Rodriguez-Rivera Topic 8: Opening Files and Starting Processes.
CS162B: Pipes Jacob T. Chan. Pipes  These allow output of one process to be the input of another process  One of the oldest and most basic forms of.
UNIX Files File organization and a few primitives.
Operating Systems Process Creation
CSCI 330 UNIX and Network Programming Unit VII: I/O Management I.
Recitation: Signaling S04, Recitation, Section A Debug Multiple Processes using GDB Debug Multiple Processes using GDB Dup2 Dup2 Signaling Signaling.
File Systems cs550 Operating Systems David Monismith.
Laface 2007 File system 2.1 Operating System Design Filesystem system calls buffer allocation algorithms getblk brelse bread breada bwrite iget iput bmap.
January 7, 2003Serguei Mokhov, 1 File I/O System Calls Reference COMP 229, 444, 5201 Revision 1.2 Date: July 21, 2004.
Copyright ©: Nahrstedt, Angrave, Abdelzaher1 Tarek Abdelzaher Vikram Adve CS241 Systems Programming System Calls and I/O.
File I/O open close lseek read and write – unbuffered I/O dup and dup2.
File Systems - Part I CS Introduction to Operating Systems.
The Process CIS 370, Fall 2009 CIS UMassD. The notion of a process In UNIX a process is an instance of a program in execution A job or a task Each process.
OS interface: file and I/O system calls File operations in C/C++? –fopen(), fread(), fwrite(), fclose(), fseek() in C f.open(…), f.close(…) in C++ I/O.
C Programming Day 2. 2 Copyright © 2005, Infosys Technologies Ltd ER/CORP/CRS/LA07/003 Version No. 1.0 Union –mechanism to create user defined data types.
Process Related System Calls By Neha Hulkoti & Kavya Bhat.
1 COMP 3500 Introduction to Operating Systems Project 4 – Processes and System Calls Overview Dr. Xiao Qin Auburn University
File table: a list of opened files Each entry contains: – Index: file descriptors – Pointer to the file in memory – Access mode File descriptor is a positive.
Using System Calls (Unix) Have to tell compiler (if C/C++) where to find the headers, etc. – i.e., the “include” files May have to tell compiler where.
Files in UNIX u UNIX deals with two different classes of files:  Special Files  Regular Files u Regular files are just ordinary data files on disk -
File System Design David E. Culler CS162 – Operating Systems and Systems Programming Lecture 23 October 22, 2014 Reading: A&D a HW 4 out Proj 2 out.
Andrew Hanushevsky: Basic I/O API’s
Lecture 5 Systems Programming: Unix Processes: Orphans and Zombies
Introduction to Kernel
Today topics: File System Implementation
Operating Systems Moti Geva
Lecture 31: Introduction to File System
Advanced Unix Programming
UNIX PROCESSES.
Chapter 4: System calls for the file system
File Systems Directories Revisited Shared Files
Unix Directories unix etc home pro dev motd snt unix ... slide1 slide2
LINUX System Programming with Processes (additional)
CSE 333 – Section 3 POSIX I/O Functions.
File I/O in C Lecture 7 Narrator: Lecture 7: File I/O in C.
Programming Assignment # 2 – Supplementary Discussion
File I/O (1) Prof. Ikjun Yeom TA – Mugyo
CSE 333 – Section 3 POSIX I/O Functions.
IPC Prof. Ikjun Yeom TA – Hoyoun
Advanced UNIX progamming
CSE 333 – Section 3 POSIX I/O Functions.
Lecture 6: Multiprogramming and Context Switching
CSCI 380: Operating Systems William Killian
FILE I/O File Descriptors I/O Efficiency File Sharing
Low-Level I/O – the POSIX Layer CSE 333 Winter 2019
Internal Representation of Files
Structure of Processes
Lecture Topics: 11/20 HW 7 What happens on a memory reference Traps
Presentation transcript:

Unix Directories unix etc home pro dev motd snt unix ... slide1 slide2 passwd motd snt unix ... Here is a portion of a Unix directory tree. The ovals represent files, the rectangles represent directories (which are really just special cases of files). slide1 slide2

Directory Representation Component Name Inode Number directory entry . 1 .. 1 unix 117 etc 4 A directory consists of an array of pairs of component name and inode number, where the latter identifies the target file’s inode to the operating system (an inode is data structure maintained by the operating system that represents a file). Note that every directory contains two special entries, “.” and “..”. The former refers to the directory itself, the latter to the directory’s parent (in the case of the slide, the directory is the root directory and has no parent, thus its “..” entry is a special case that refers to the directory itself). home 18 pro 36 dev 93

Standard File Descriptors main( ) { char buf[100]; int n; const char* note = "Write failed\n"; while ((n = read(0, buf, sizeof(buf))) > 0) if (write(1, buf, n) != n) { (void)write(2, note, strlen(note)); exit(EXIT_FAILURE); } return(EXIT_SUCCESS); File descriptor 0: standard input File descriptor 1: standard output File descriptor 2: standard error output The file descriptors 0, 1, and 2 are opened to access your terminal when you log in, and are preserved across forks, unless redirected.

File operations int open(const char *pathname, int flags, mode_t mode); Returns a file descriptor ssize_t read(int fildes, void *buf, size_t count); Use the file descriptor ssize_t write(int fildes, const void *buf, size_t count); Use file descriptor off_t lseek(int fildes, off_t offset, int whence); Delete, read attributes, set attributes, etc

Representing an Open File (1) file descriptor table system file table active inode table 1 2 3 . n–1 ref count f pointer access inode A process’s set of open files is represented by a data structure known as the file descriptor table, which is used to map file descriptors, representing open files, to system file table entries, each representing an open file. Each process has a separate file descriptor table, but there is exactly one system file table for the entire system. Each active file (i.e., a file that is open or otherwise being used) is represented by an inode that is entered in the active inode table. Recently accessed blocks of data from the file are kept in the kernel’s data section in what is known as the buffer cache—an area of memory reserved for buffering data from files. Thus data from these blocks can be accessed again and again without wasting time going to the disk. When a disk block is modified, the modifications are written in the buffer cache and only later are written to disk. This allows the process doing the modification to proceed without having to wait for the data to be written to disk. disk buffer cache

Allocation of File Descriptors Whenever a process requests a new file descriptor, the lowest numbered file descriptor not already associated with an open file is selected. One can depend on always getting the lowest available file descriptor.

Representing an Open File (2) file descriptor table system file table active inode table 1 2 3 fdrw 1 rw . . . n–1 ref count f pointer access inode This slide shows what happens when a file is opened. The lowest-numbered available file descriptor is allocated from the file descriptor table. Next, an entry in the system file table is allocated, and the file descriptor table entry is set to point to the system file table entry. An inode-table entry for the file is allocated (or found if it already exists) and the system file table entry is set to point to it. Additional fields of the system file table entry are initialized, including: a reference count each file-descriptor-table entry pointing to it contribute one to the reference count the entry is freed when the reference count goes to zero the allowed access (i.e., how the file was opened—read-only, read-write, etc.) the file pointer (i.e., the location within the file at which the next transfer will start) after every read and write system call the file pointer is incremented by the number of bytes actually transferred, thus facilitating sequential I/O fdrw = open("x", O_RDWR); disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

Reading From a File #include <sys/types.h> #include <unistd.h> ssize_t read(int fd, void *buffer, size_t n) read up to n bytes of data into buffer returns number of bytes transferred returns 0 on end of file returns –1 on error When will “read” transfer fewer bytes than specified ? The read system call tells the system to read up to n bytes of data into the area of memory pointed to by buffer from the file referred to by fd. It returns the number of bytes actually transferred, or, as usual, –1 if there was an error. Read will transfer fewer bytes than specified if: the number of bytes left in the file was less than n the read request was interrupted by a signal after some of the bytes were transferred the file is a pipe, FIFO, or special device with less than n bytes immediately available for reading It transfers as many bytes as are present up to the given maximum; if it returns a zero, that means the end of the file has been reached.

Writing To a File #include <sys/types.h> #include <unistd.h> ssize_t write(int fd, void *buffer, size_t n) write up to n bytes of data from buffer returns number of bytes transferred returns –1 on error The write system call tells the system to write up to n bytes of data from the area of memory pointed to by buffer into the file referred to by fd. It returns the number of bytes actually transferred, or, as usual, –1 if there was an error. If write transfers fewer bytes than specified, then something caused the transfer to stop prematurely: a signal interrupted the system call this is pretty complicated and we’ll discuss it in more detail later. However, this would happen if the requested size was large enough so that the write was split by the kernel into a number of segments, at least one segment was written (otherwise the call would have returned with an error and errno set to EINTR), and the signal occurred while a subsequent segment was being written a filesize limit was reached an I/O error occurred The response in all these cases should be to attempt to rewrite those bytes that were not transferred; if a signal had interrupted the previous try, then the next try will succeed (unless again interrupted by a signal); if a filesize limit had been reached or an I/O error has occurred, this next write will yield the appropriate error code.

Example main( ) { char buf[BUFSIZE]; int nread; const char* note = "Write failed\n"; while ((nread = read(0, buf, sizeof(buf))) > 0) { int bytes_left = nread; int bpos = 0; while ((n = write(1, &buf[bpos], bytes_left)) < bytes_left) { if (n == –1) { write(2, note, strlen(note)); exit(EXIT_FAILURE); } bytes_left  n; bpos += n; return(EXIT_SUCCESS); Here we copy from the program’s standard input to its standard output, but take advantage of what we’ve learned about the behavior of write. In particular, since it’s not guaranteed that write will transfer all the bytes requested, we must supply code to make sure that all the data does get transferred. Thus, after each call to write (except, for reasons of space on the slide, when we write to file descriptor 2), we check to see how much data was written and call write again, if necessary, to handle the unwritten data.

Random Access #include <sys/types.h> #include <unistd.h> off_t lseek(int fd, off_t offset, int whence) sets the file pointer for fd: if whence is SEEK_SET, the pointer is set to offset bytes; if whence is SEEK_CUR, the pointer is set to its current value plus offset bytes; if whence is SEEK_END, the pointer is set to the size of the file plus offset bytes it returns the (possibly) updated value of the file pointer relative to the beginning of the file. Thus, n = lseek(fd, (off_t)0, SEEK_CUR); returns the current value of the file pointer for fd To effect random access to files (i.e., access files other than sequentially), you first set the file pointer, then perform a read or write. Setting the file pointer is done with the lseek system call.

lseek Example What does this piece of code do? fd = open("textfile", O_RDONLY); fptr = lseek(fd, (off_t)–1, SEEK_END); while (fptr != –1) { read(fd, buf, 1); write(1, buf, 1); fptr = lseek(fd, (off_t)–1, SEEK_CUR); } This example prints the contents of a file backwards. Note what’s happening in the while statement: it continues as long as fptr does not return –1, a value it will return if the call to lseek fails. Our intent is that the last successful call to lseek sets the file pointer to 0. The next call, which attempts to set it to –1, will fail, thus causing lseek to return –1 (recall that failed system calls always return –1).

Representing an Open File (3) file descriptor table system file table active inode table 1 2 3 1 r 10 fdrw 1 rw 20 . . fdr . n–1 ref count f pointer access inode In this slide we see the effect of two opens of the same file within the same process. fdrw = open("x", O_RDWR); fdr = open("x", O_RDONLY); write(fdrw, buf, 20); read(fdr, buf2, 10); disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

Multiple Descriptors; One File How are standard file descriptors set up? suppose 1 and 2 are opened separately while ((n = read(0, buf, sizeof(buf))) > 0) if (write(1, buf, n) != n) { (void)write(2, note, strlen(note)); exit(EXIT_FAILURE); } error message clobbers data bytes! By convention, file descriptors 1 and 2 are used for processes’ normal and diagnostic output. Normally they both refer to the display, and thus diagnostic output is intermingled with normal output. Suppose, however, one wanted to redirect both file descriptors so that all output, normal and diagnostic, was sent to a file. One might open this file twice, once as file descriptor 1 and again as file descriptor 2, thereby creating two system file table entries. As file descriptor 1 receives output, the offset field of its file table entry advances with each write. After 1000 bytes have been written (sequentially), the offset field is set to 1000, representing the current end-of-file. If at this point a diagnostic message is written to file descriptor 2, it will start at the beginning of the file, overwriting the data already there, since file descriptor 2’s file table entry’s offset is still at 0. This outcome is certainly not desirable.

dup System Calls dup returns a new file descriptor referring to the same file as its argument int dup(int fd) dup2 is similar, but it allows you to specify the new file descriptor int dup2(int oldfd, int newfd) We can use one of the dup system calls to solve this problem. dup obeys our rule of always allocating the lowest available file descriptor. However, with dup2, one can specify, via the second argument, which file descriptor is allocated. If the second argument is the file descriptor of an open file, the file is first closed, then associated with the file of the first argument.

dup Example /* redirect stdout and stderr to same file */ /* assumes file descriptor 0 is in use */ close(1); open("file", O_WRONLY|O_CREAT, 0666); close(2); dup(1); /* alternatively, replace last two lines with: */ dup2(1, 2); Here we see how to use dup and dup2 to set file descriptors 1 and 2 to refer to the same system file-table entry. Note the extra argument to open. We’ve given open the O_CREAT flag, which tells the system that if the file does not exist, it should create it. The third argument helps to specify the access permissions assigned to the file if it’s created by this call. We discuss this in detail in a few slides.

Representing an Open File (4) file descriptor table system file table active inode table 1 2 3 fdrw 2 rw 20 . fdrw2 . . n–1 ref count f pointer access inode The dup system call causes two file descriptors to refer to the same file table entry and hence share the offset. fdrw = open("x", O_RDWR); fdrw2 = dup(fdrw); write(fdrw, buf, 20); disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

Representing an Open File (5) file descriptor table system file table active inode table 1 2 3 2 r 10 fdrw 4 rw 20 . fdrw2 . fdr . n–1 ref count f pointer access inode If our process executes a fork system call, creating a child process, the child is given a file-descriptor table that’s a copy of its parent’s. Of course, the reference counts on the system-file-table entries are increased appropriately. fork( ) disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

I/O Redirection % who > file & if (fork( ) == 0) { } char *args[ ] = {"who", 0}; close(1); open("file", O_WRONLY|O_TRUNC, 0666); execv("who", args); printf("you screwed up\n"); exit(1); } This is an example of what a shell might do to handle I/O redirection: it first creates a new process in which to run a command (“who”, in this case). In the new process it closes file descriptor 1 (standard output—to which normal output is written). It then opens “file” (the arguments indicate that “file” is only to be written to, that any prior contents are erased, and that if “file” didn’t already exist, it will be created with read and write permission for all; assuming that file descriptor 0 is not available (it’s assigned to standard input), file descriptor 1 will be assigned to “file”. Assuming that execv succeeds, when “who” runs, its output is written to “file”. Note that the parent process does not wait for its child to terminate; it goes on to execute further commands. (This behavior occurs because we’ve placed an “&” at the end of the command line.) Note the args argument to execv: By convention, the first argument to each command is the name of the command (“who” in this case). To indicate that there are no further arguments, a zero is supplied. Note that we aren’t checking for errors: this is only because doing so would cause the resulting code not to fit in the slide. You should always check for errors.