Presentation is loading. Please wait.

Presentation is loading. Please wait.

TDC368 UNIX and Network Programming

Similar presentations


Presentation on theme: "TDC368 UNIX and Network Programming"— Presentation transcript:

1 TDC368 UNIX and Network Programming
Week 3-4: UNIX File System File Representation File Handles System Calls (APIs) System Calls: dup dup2 pipe Library functions: popen(); pclose(); Camelia Zlatea, PhD

2 Outline UNIX File System: files, catalogs (inodes; types,size, permissions). UNIX File I/O: unbuffered I/O functions (open ,close,seek, umask,read,write). Standard I/O buffering (fopen, fclose, fflush). File Sharing Communication between related processes via pipe (pipe, dup,dup2) Appendix: UNIX system calls: C library functions, C++ classes, POSIX API's.

3 UNIX File System Hierarchical organization Device independence
compatibility among files or catalogs, devices and IPC Types of files regular files (byte-stream, no format) directory files (groups of files mapped on user’s projects) special files (to access peripheral devices, /dev/...) pipe files (inter process communication mechanisms)

4 UNIX Type of Files Regular Files Directory Files Special Files
Shell commands: cp, rm, mv, cat, ls System Calls: <fcntl.h>, <unistd.h> open, creat, read, write, close, link, unkink Directory Files Shell commands: mkdir, rmdir, mv System Calls: <dirent.h> Special Files mknod ( root privileges) Pipe Files mkfifo

5 File Attributes File Type (-,d, b,c,p) ls -l
Access Permission for owner, group and others (r,w,x) File Owner ID and GID Number of hard links of a file File Size Last access, modify and change permission time INODE number File system ID

6 File Attributes Constant Attributes Changeable Attributes
file type file inode number File System ID Changeable Attributes chmod (command and system call) chown, chgrp touch (command), utime (sys call) ln (command ), link(syscall) rm (command), unlink(syscall) Attributes query by: system calls stat or fstat

7 File Permissions all UNIX files have associated a set of owner permissions bits, that are used by the OS to determine the access classes of users: owner, group, other permissions: owner group other rwx rwx rwx octal 777 octal 755 - default permissions = creation_mask XOR umask_value - creation mask (set at system initialization) is usually 0777

8 File Permissions Command shell:
umask new_mask_value (display/change umask value) System call: mode_t umask(mode_t mask); - to modify umask for all your processes edit your startup file , .login (.profile) and insert the line umask nnn (where nnn is the new umask value) - change permissions - command shell: chmod guorwx file

9 File Permissions System calls: <sys/types.h> <sys/stat.h>
int chmod(const char *path, mode_t mode); int fchmod(int fid, mode_t mode); Example: Use system call stat and determine the inode information Name of the file is passed as program’s argument.

10 File Permissions #include <sys/types.h>
#include <sys/stat.h> void main(int argc, char * argv[]) struct stat statbuf; if (stat(argv[1], &statbuf)==-1) { /* print error message */ exit(1); } else { if (statbuf.st_mode & S_IFDIR) { printf(“ %s is a directory \n”, argv[1]); printf(“user ID is %s \n”, stat_buf.st_uid); } Shell Command umask: %umask 022 % cat > test1 hello ^d %ls -l test1 %umask 011 % mkdir test2 % cat > test3 …..

11 File Owner Information
real user ID (UID) real group ID (GID) - information is obtained from the password file entry (/etc/passwd) group ID (GID) numbers should map to group names from /etc/group effective user ID (EUID) effective group ID (EGID) - usually UID identical with EUID

12 File Owner Information
Example: command passwd (change password) need to have access to a system file (as a root) a non-privileged user is able to run a command temporarily as root if suid permission is set: rws r-x r-x root /etc/passwd

13 File Owner Information
Command shell id is used to display owner information (UID, GUID, EUID, EGID) System Calls: <sys/types.h> <unistd.h> uid_t getuid(void); uid_t geteuid(void); gid_t getgid(void); gid_t getegid(void);

14 File Representation UNIX File System Organization: BOOT BLOCK
SUPERBLOCK INODE TABLE DATA BLOCK Boot Block contains a program that loads UNIX kernel file from the file system transfer control to the start address of the kernel code if the system is already running and the disk is accessed by mount, boot block is ignored  SUPERBLOCK - contains all information that defines the current state of the file system - size of i-list Iinode list table) - size in blocks of the files system - number of free blocks available in the file system - the address of the free inodes list - number of total free blocks in the system - files system name and file system disk name - superblock is read when mounting the device  File System ID + Inode Number identify uniquely the File Why File name is not an attrbute???

15 File Representation INODE - identifies a UNIX file Inode Structure
file attributes physical disk address where file date is located Inode Table (UNIX System V) an inode entry for each file in the file system Directory File - maps file name to inodes numbers File System ID + Inode Number identify uniquely the File Why File name is not an attrbute???

16 File Representation Inode Table inode structure File Information
- size (bytes) - owner information (UID, GID, EUID, EGID) - relevant times - number of links - number of blocks - permissions Data Blocks - direct pointers to beginning of file blocks - indirect pointers - double indirect pointers - triple indirect pointers Data Block  ……………. File System ID + Inode Number identify uniquely the File Why File name is not an attrbute???

17 Directory Structure /root /bin /etc unix /dev /usr /tmp
Absolute path name: /usr/project/bin/a.out Relative path name: ./a.out ../../a.out File name: NAME_MAX in <limits.h> Path name: PATH_MAX -’’- Hard Link - the path name of a file ln /usr/myproj/a.out /usr/project/bin/a.out

18 Directory Representation
UNIX directory - a file containing a correspondence between filenames and inodes the inode does not contain file name data blocks of the directory file are directory entries a directory entry contains: inode number + file name command as rename/move (mv) affect the name filed from directory entry, only one physical copy of the file exists on disk (file may have several names on different directories or the same name) for a newly created directory (ex. mkdir dir_name) OS makes a new directory entry for it and assign an inode to it File System ID + Inode Number identify uniquely the File Why File name is not an attrbute???

19 Accessing Files Example of Directory file: Inode Number File Name 78 .
78 . 115 sample 240 test.c 300 a.out 115 sample_data_l File System ID + Inode Number identify uniquely the File Why File name is not an attrbute???

20 Directory Files Record-oriented files
file name inode number Record data type is the structure dirent in SV/POSIX <dirent.h> direct in BSD <sys/dir.h>

21 Directory Functions Opens a directory file, return a handle of type DIR* opendir Reads next record from directory file readdir Closes a directory file closedir Sets file pointer at the beginning rewinddir Random directory access telldir, seekdir

22 Hard vs. Symbolic Links Hard Links: ln fileA fileB
pathnames of a file (directory entries) NO new inode is created cannot link files across file systems cannot link directories (as reg.user) increase the hardlink counter per inode ln /dirA/name1 /dirB/name2 Effect: creates a hard link, both names point to the same inode structure) Symbolic Links: ln -s fileA fileB Symbolic link is a file containing the name of another file(directory) a new inode is created can link files across file systems can link directories (as reg.user) ln -s /dirB/name2 /dirA.name1 Effect: if there is an entry – name1 in /dirA creates new inode 56789 new entry – name2 in /dirB the block of date from inode contains “/dirA/name1”

23 UNIX File System Calls (APIs)
Header file: <sys/types.h> <fcntl.h> open - opens a file for data access <unistd.h> creat - creates regular file read - reads data from a file write - writes data to a file close - terminate the access to a file

24 UNIX File System Calls (APIs)
Header files: <sys/types.h> <unistd.h> lseek - random access to a file link - creates a hard link to a file unlink - removes a hard link of a file chown - changes UID or GID of a file chmod - changes access perm. of a file Header files: <sys/types.h> <sys/stat.h> stat, fstat - queries attributes of s file

25 File Handles C stream pointers (FILE *)
Conversions, header file <stdio.h> int fileno(FILE *stream_pointer) FILE *fdopen(inf fd, char *open_mode); C Library Func. UNIX System Call fopen open fread, fget, fgets, fscanf read fwrite,fput,fputs,fprintf write fseek, ftell, frewind lseek fclose close

26 FILE handle Process System File Descriptor Table (FDT)
#include <stdio.h> FILE *fp; if (( fp = fopen(“test.dat”, “w”)== NULL) fprintf(stderr, “cannot open file\n”); else fprintf(fp, “This is a text”); Process File Descriptor Table (FDT) 0 - stdin 1 - stdout 2 - stderr 3 4 5 6 System File Descriptor Table (SDT) 0 - stdin 1 - stdout 2 - stderr 3 4 5 6 fp “This is a text”); 3 User area Kernel area

27 Buffering Minimize # of write/read calls Types of Buffering
Fully / Line / No Buffering Change Buffering Type #include <stdio.h> void setbuf(FILE *fp, char *buf) void setbuf(FILE *fp, char *buf, int mode, size_t size) mode: _IOFBF, _IOLBF, _IONBF size: default is BUFSIZ Flush a buffer int fflush(FILE *fp);

28 Parent Process opens file test.dat before fork
Child FDT 1 2 3 FD test.dat 4 5 6 Parent FDT 1 2 3 FD test.dat 4 5 6 SFT

29 Parent Process opens file test.dat after fork
Child FDT 1 2 3 FD test.dat 4 5 6 Parent FDT 1 2 3 FD test.dat 4 5 6 SFT

30 Pipe Think at a pipe as being a special file that can store a limited amount of data in a FIFO manner Size: 512 K on most Unix systems (<limits.h> or <sys/param.h>) Interaction rule: one process writes to the pipe (as it were a file) and other process reads from the pipe Data are written at one end of the pipe and are read at the other end System keeps track of the current location of last read/write

31 Process synchronization using pipes
If a writing process attempts to write to a full pipe the system blocks it until pipe is able to receive data If a process attempts to read from an empty pipe the system blocks it until data is available A process will block if it tries to read from a pipe that has been opened for reading but no process has opened it for writing If a write is made to a pipe not open for reading SIGPIPE is generated and errno is set to ``broken pipe''

32 write() System Call Data are written to a pipe using unbuffered I/O write Include files: <unistd.h> Calling pattern: size t write (int fileds, void *buf, size t nbytes); fileds is a file descriptor Return values: On success: #bytes written On failure: ­1 Set errno? Yes

33 Writing to a pipe Each write request is appended to the end of the pipe Write requests of P IPE BUF size or less are guaranteed not to be interleaved with other write requests to the same pipe Write request may cause the process to block; the constants O NONBLOCK, O NDELAY in !sys/fcntl.h? can be set by fcntl() to non­blocking strategy When O NONBLOCK, O NDELAY are set and the request to write P IPE BUF size bytes or less fails the value returned by write is shown in the next slide Table ==>

34 Setting the constants that control piping strategy
The table shows what write returns in function of constants Blocking constants for write() If a write is made to a pipe not open for reading SIGPIPE is generated and errno is set to ``broken pipe'' O NONBLOCK O NDELAY Value returned set clear -1 process blocks until space available process does not block

35 read() system call Data are read from a pipe using unbuffered I/O read
Include files: <sys/types.h>, <sys/uio.h>, <unistd.h> Calling pattern: size t read (int fileds, void *buf, size t nbyte); fileds is a file descriptor; Return values: On success: #bytes read On failure: ­1 Set errno? Yes

36 Reading from a pipe Read actions are initiated from the current position If pipe is open for writing by another process but it is empty read returns as shown bellow Blocking constants for read If the pipe is not opened for writing by another process read returns 0 (as in case of O NDELAY set) O NONBLOCK O NDELAY Value returned set clear -1 process blocks until data available process does not block

37 Types of pipes unnamed pipes, used with related processes (example: parent/child or child/child) named pipes, used with unrelated process

38 Unnamed pipes ; pipe() system call
An unnamed pipe is constructed with pipe() system call, Include files <unistd.h> Calling pattern int pipe ( int filedes[2] ); Return values: On success: 0 On failure: ­1 Set errno? Yes Functionality: pipe() operates on a full­duplex or half­duplex basis Writing/reading from a pipe filedes[1] )  filedes[0] filedes[0] )  filedes[1] full­duplex half­duplex

39 Actions performed by pipe()
If successful, it returns two file descriptors, filedes[0], filedes[1] that reference two data streams Historically pipes were non­duplex (i.e., unidirectional); if two­way communication was needed two pipes were opened, one for reading and the other for writing Currently file descriptors returned by pipe() are full­duplex (i.e., bi­directional), both opened for reading and writing. In a full duplex if the process writes to filedes[0] then filedes[1] is used for reading, and vice­versa In a half­duplex, filedes[0] is used for reading and filedes[1] for writing an attempt to use filedes[1] for reading or filedes[0] for writing produces an error

40 Example:. a parent process sends to a child the first argument
Example: a parent process sends to a child the first argument from its command line and child acknowledges the receipt /* Using a pipe to send data from parent to child */ #include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <string.h> #include <errno.h> int main (int argc, char *argv[]) { int fdes[2]; static char message[BUFSIZ]; if (argc != 2) {fprintf(stderr,''Usage: %s message``n'', *argv); exit(1); } if (pipe(fdes) == ­1) { perror(''Pipe''); exit(2); } switch (fork()) case ­1: { perror(''Fork''); exit(3); }

41 Example:. a parent process sends to a child the first argument
Example: a parent process sends to a child the first argument from its command line and child acknowledges the receipt /* Child process: closes fdes[1] and read fdes[0] to obtain the message */ case 0: { close(fdes[1]); if (read(fdes[0], message, BUFSIZ) != ­1) printf(''Message received by child is: [%s]``n'', message); fflush(stdout); } else { perror(''Read''); exit(4); } break;

42 Example:. a parent process sends to a child the first argument
Example: a parent process sends to a child the first argument from its command line and child acknowledges the receipt /* Parent closes fdes[0] and write message to fdes[1] */ default: { close(fdes[0]); if (write(fdes[1], argv[1], strlen(argv[1])) != ­1) printf(''Message sent by parent is: [%s]``n'', argv[1]); fflush(stdout); } else { perror(''Write''); exit(5); } break; exit(0);

43 Redirecting stdin and stdout
Examples: cat > test.dat “2” “2” cat cat “0” “0” “1” “1” test.dat 0 stdin 1 stdout 2 stderr 0 stdin 1 test.dat 2 stderr

44 Redirecting stdin and stdout; dup() system call
Create a pipe Associate stdin and stdout with the created pipe using dup() system call Include files <unistd.h> Calling pattern int dup ( int fdes ); Return values: On success: next available file descriptor On failure: ­1 Set errno? Yes or No

45 dup() functionality dup() duplicates an original open file descriptor;
new descriptor references SFT (System File Table) entry for the next available non­negative file descriptor new descriptor shares the same file pointer (offset), has the same access mode as original and remain open across exec. since actions necessary for redirection are as in the code: int fdes[2]; pipe(fdes); close(fileno(stdout));dup(fdes[1]); there is a chance that descriptor returned by dup() is not the one closed;

46 dup2() system call Using dup2() system call for file redirection
Include files <unistd.h> Calling pattern int dup2 ( int fdes1, int fdes2 ); Return values: On success: next available file descriptor >=0 On failure: ­1 Set errno? Yes dup2() functionality: dup2() closes the file and duplicates the file descriptor atomically therefore dup2() avoid the limitation of dup() both dup() and dup2() can be implemented with fcntl()

47 Redirecting stdin and stdout
Example: int fd; mode_t fd_mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH; fd = open(“test.dat”, O_WRONLY | O_CREAT, fd_mode); dup2(fd, STDOUT_FILENO); ….. close(fd); 0 stdin 1 stdout 2 stderr 3 Write to test.dat 0 stdin 1 Write to test.dat 2 stderr 3 Write to 0 stdin 1 Write to test.dat 2 stderr Process FDT after close Process FDT after open Process FDT after dup2

48 Example: implementing ''last | sort '' command
/* A home grown last | sort command */ #include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <errno.h> void main (void) { int fdes[2]; if (pipe(fdes) == ­1) { perror(''Pipe''); exit(1); } switch (fork()) case ­1: { perror(''Fork''); exit(2); } case 0: { dup2(fdes[1], fileno(stdout)); close(fdes[0]); close(fdes[1]); execlp(''last'', ''last'', 0); exit(2); } default: { dup2(fdes[0],fileno(stdin)); close(fdes[0]); close(fdes[1]); execlp(''sort'', ''sort'', 0); exit(4); }

49 Per Process FDT and SFT before dup2()
Child FDT 0 - stdin 1 - stdout 2 - stderr 3 fdes[0] 4 fdes[1] 5 6 Parent FDT 0 - stdin 1 - stdout 2 - stderr 3 fdes[0] 4 fdes[1] 5 6 SFT

50 Per Process FDT and SFT after dup2()
Child FDT 0 - stdin 1 – fdes[1] 2 - stderr 3 fdes[0] 4 fdes[1] 5 6 Parent FDT 0 – fdes[0] 1 – stdout 2 - stderr 3 fdes[0] 4 fdes[1] 5 6 SFT

51 Per Process FDT and SFT after close()
Child FDT 0 - stdin 1 – fdes[1] 2 - stderr 3 4 5 6 Parent FDT 0 – fdes[0] 1 – stdout 2 - stderr 3 4 5 6 SFT

52 Summary for communication via unnamed pipes
Create the pipe(s) needed Generate the child process(es) Close/duplicate file descriptors to properly associate the ends of the pipe Close the unneeded ends of the pipe Perform the communication activities Close any remaining open file descriptors If appropriate, wait for child processes to terminate

53 popen() library function
popen() encapsulates the sequence of activities, Generate a pipe; Fork a child process Duplicate file descriptors created by pipe; Passes command execution information from one process to another Include files <stdio.h> Calling pattern FILE *popen (char *command, char *type ); Return values: On success: ptr to a FILE On failure: NULL pointer Set errno? no

54 popen() functionality
create a child process; child process exec's a Bourne shell which will execute the shell command passed to popen() Input to and output from the child process is accomplished via pipe: if mode type in popen is ``w'' parent process can write to the standard input of the shell command run by the child and child process can read the data at its standard input; if mode type in popen is ``r'' parent process can read from the standard output of the shell command run by the child and child process can write the data at its standard output;

55 pclose() library function
pclose(): closes a data stream opened by popen(), Include files <stdio.h> Calling pattern int pclose (FILE *stream ); Return values: On success: Exit status of command On failure: ­1 Set errno? no

56 Example: using popen and pclose
#include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <limits.h> int main (int argc, char *argv[]) { FILE *fin, *fout; char buffer[PIPE_BUF]; int n; if (argc ! 3) { fprintf(stderr, ''Usage: %s cmd1 cmd2\n'', argv[0]); exit(1); } fin = popen(argv[1], ''r''); fout = popen(argv[2], ''w''); while ((n = read(fileno(fin), buffer, PIPE_BUF)) > 0) write (fileno(fout), buffer, n); pclose (fin); pclose (fout); exit(0); }

57 Example: using popen and pclose
Observations: This program requires two command line arguments, i.e., two shell commands whose standard output/input will be redirected via pipes generated by popen() The first popen() call with option ``r'' directs the system to fork a child that will execute first shell command whose output will be redirected so it can be read by the parent using fin. The second popen() call with option ``w'' directs the system to fork a new child that will execute the second shell command whose input will be the data written by parent using fout. The while() loop is used to copy data from the output end of one pipe to the input end of the other pipe.


Download ppt "TDC368 UNIX and Network Programming"

Similar presentations


Ads by Google