1. 1 Chapter 10 : Case Study - UNIX History Overview Processes Memory management Input/output in The unix file system Security Note: This case study covers only UNIX. Please read chapter 10 of the text book for LINUX.

2. 2 History of UNIX Originated from the MULTICS (Multiplexed Information and Computing Service) operating system by M.I.T, Bell Labs and GE Originated from the MULTICS (Multiplexed Information and Computing Service) operating system by M.I.T, Bell Labs and GE There are two main versions: There are two main versions: AT&T System V Release 4 (SVR4) AT&T System V Release 4 (SVR4) Originally developed by AT&T, now SCO Originally developed by AT&T, now SCO BSD (Berkeley Software Distribution) BSD (Berkeley Software Distribution)

3. 3 Overview of UNIX Supports various architectures Supports various architectures Structure varies Structure varies Supports preemptive multitasking Supports preemptive multitasking Multiuser environment - generally secure Multiuser environment - generally secure Supports multithreaded applications Supports multithreaded applications Protection/Security is high on modern versions Protection/Security is high on modern versions Supports symmetric multiprocessing Supports symmetric multiprocessing Highly scalable/portable to various systems Highly scalable/portable to various systems Many types/flavours of UNIX exist Many types/flavours of UNIX exist

4. 4 UNIX Layers The layers of a UNIX system. User Interface

5. 5 UNIX Utility Programs A few of the more common UNIX utility programs required by POSIX

6. 6 UNIX Kernel (1) Approximate structure of generic UNIX kernel

7. 7 UNIX Kernel (2) Bottom layer Bottom layer Device drivers for character and block devices Device drivers for character and block devices Process dispatcher which stops the current process, saves its state and starts the appropriate driver when an interrupt occurs Process dispatcher which stops the current process, saves its state and starts the appropriate driver when an interrupt occurs

8. 8 Process Creation in UNIX - fork Process creation in UNIX.

9. 9 POSIX Shell A highly simplified shell

10. 10 The ls Command Steps in executing the command ls type to the shell

11. 11 Some UNIX Process Concepts Daemons (background processes) Daemons (background processes) Cron daemon which wakes up once a minute to check scheduled events (eg., disk backup) Cron daemon which wakes up once a minute to check scheduled events (eg., disk backup) Pipes - syncronized channels between processes to pass byte streams Pipes - syncronized channels between processes to pass byte streams Signals – software interrupts used for interprocess communication. Choices: catch, ignore, or kill process Signals – software interrupts used for interprocess communication. Choices: catch, ignore, or kill process

12. 12 Signals Required By POSIX The signals required by POSIX.

13. 13 System Calls for Process Management s is an error code pid is a process ID residual is the remaining time from the previous alarm

14. 14 Thread Calls in POSIX The principal POSIX thread calls.

15. 15 Thread Calls in POSIX Threads were not in the first versions of UNIX Threads were not in the first versions of UNIX There are many thread packages in use which are standardized in POSIX There are many thread packages in use which are standardized in POSIX Thread calls are the same for user-space or kernel- space Thread calls are the same for user-space or kernel- space In kernel-space implementation calls are system calls In kernel-space implementation calls are system calls In user-space implementation calls are to a run-time library In user-space implementation calls are to a run-time library

16. 16 Thread Communication - mutexes Threads use locks called mutexes for short-time locking a resource (say a shared buffer) Threads use locks called mutexes for short-time locking a resource (say a shared buffer) A mutex must be first created (and finally destroyed) A mutex must be first created (and finally destroyed) Mutual exclusion is implemented by locking a mutex before accessing a resource and unlock it when they are done (like binary semaphores which is “0” or “1”, a mutex is either locked or unlocked) Mutual exclusion is implemented by locking a mutex before accessing a resource and unlock it when they are done (like binary semaphores which is “0” or “1”, a mutex is either locked or unlocked)

17. 17 Thread Communication – condition variables For long-term synchronization (such as waiting for a tape to become free) condition variables are used For long-term synchronization (such as waiting for a tape to become free) condition variables are used Condition variables have to be created first and later destroyed like mutexes Condition variables have to be created first and later destroyed like mutexes A condition variable is used by having one thread wait on it, and another thread signal it. If no thread is waiting when a signal is sent, the signal is lost A condition variable is used by having one thread wait on it, and another thread signal it. If no thread is waiting when a signal is sent, the signal is lost

18. 18 UNIX Scheduler (1) The UNIX scheduler is based on a multilevel queue structure (highest priority queue first, round-robin in each queue) The UNIX scheduler is based on a multilevel queue structure (highest priority queue first, round-robin in each queue) In this scheme, a process which was blocked and waiting for an event joins the appropriate queue when blocking is over (a process whose disk I/O is finished joins,say, queue –4) In this scheme, a process which was blocked and waiting for an event joins the appropriate queue when blocking is over (a process whose disk I/O is finished joins,say, queue –4)

19. 19 UNIX Scheduler (2) Once a second the priority of all processes are recalculated to avoid starvation using Once a second the priority of all processes are recalculated to avoid starvation using priority = CPU_usage + nice + base CPU_usage, represents the average number of clock ticks per second that the process has had during the past few seconds CPU_usage, represents the average number of clock ticks per second that the process has had during the past few seconds Nice is a value between –20 to 20 (default 0). Nice system call can be used to set this value 0-20 Nice is a value between –20 to 20 (default 0). Nice system call can be used to set this value 0-20 Base is a system parameter in UNIX source code Base is a system parameter in UNIX source code The scheduler forces CPU bound (on positive queues) get any service that is left over when all I/O bound and interactive processes are blocked The scheduler forces CPU bound (on positive queues) get any service that is left over when all I/O bound and interactive processes are blocked

20. 20 Booting UNIX (1) The first sector of the boot disk (master boot record) is read in and executed The first sector of the boot disk (master boot record) is read in and executed This sector loads the boot program This sector loads the boot program Boot reads root directory, loads kernel and starts its execution Boot reads root directory, loads kernel and starts its execution Kernel reads the rest of the operating system (main C-code section) Kernel reads the rest of the operating system (main C-code section) C code does some initialization, allocates system data structures, loads device drivers and handcrafts the first process, process 0 C code does some initialization, allocates system data structures, loads device drivers and handcrafts the first process, process 0

21. 21 Booting UNIX (2) The sequences of processes used to boot some systems cp

22. 22 Handling Memory Each process has three segments (shown as one segment in the figure, but if hardware supports they can be separate): Each process has three segments (shown as one segment in the figure, but if hardware supports they can be separate): Text : executable code (which is shared in the figure) Text : executable code (which is shared in the figure) Data : variables, strings, arrays etc. Data : variables, strings, arrays etc. initialized data – variables which must be initialized to some value when program starts initialized data – variables which must be initialized to some value when program starts Uninitialized data (BSS) – not initialized but has value 0 as default Uninitialized data (BSS) – not initialized but has value 0 as default Stack Stack Text is fixed in length, data and stack can grow and shrink Text is fixed in length, data and stack can grow and shrink Process B Process A

23. 23 Paging in UNIX (1) Prior to 3BSD, UNIX systems used swapping (if memory is full, swap processes to disk) Prior to 3BSD, UNIX systems used swapping (if memory is full, swap processes to disk) To run a process, all that is needed is the user structure and page table. The pages of the text, data and stack segments are brought in on demand To run a process, all that is needed is the user structure and page table. The pages of the text, data and stack segments are brought in on demand

24. 24 Paging in UNIX (2) – Core Map Main memory: kernel, core map, pages Main memory: kernel, core map, pages Core map has an entry for each page and contains information about the contents of the page frames Core map has an entry for each page and contains information about the contents of the page frames Page on Disk Process table entry

25. 25 Page Replacement Algorithm (1) The page replacement algorithm is executed by the page daemon (process 2) The page replacement algorithm is executed by the page daemon (process 2) Page daemon wakes up every 250 msec and transfers pages to disk if the amount of free memory is less than the system parameter lotsfree (typically set to ¼ of memory) Page daemon wakes up every 250 msec and transfers pages to disk if the amount of free memory is less than the system parameter lotsfree (typically set to ¼ of memory) Page daemon uses the two-handed clock algorithm Page daemon uses the two-handed clock algorithm

26. 26 Basic Clock Algorithm (1) The pointer (hand) points to the oldest page When a page fault occurs, if the R bit of the pointed page is 0 (page not referred), this page is evicted (the new page replaces this page - written to disk first if it is dirty) if the R bit is 1 (page accessed), R bit is cleared and the hand is advanced to the next page

27. 27 Two-handed Clock Algorithm (2) Page daemon has to do two passes with one core map pointer. Pass 1 clears all R bits, second pass removes pages (R bits are set between pass 1 and 2) Page daemon has to do two passes with one core map pointer. Pass 1 clears all R bits, second pass removes pages (R bits are set between pass 1 and 2) Page daemon maintains two pointers into the core map to speed up the process (one pass instead of two) for large memories Page daemon maintains two pointers into the core map to speed up the process (one pass instead of two) for large memories When page daemon runs, it first clears the R bit at the front hand, and then checks the R bit at the back hand, after which it advances both hands When page daemon runs, it first clears the R bit at the front hand, and then checks the R bit at the back hand, after which it advances both hands Each time the page daemon runs, the hands rotate less than a full revolution, the amount depending on the number of pages needed to reach lotsfree Each time the page daemon runs, the hands rotate less than a full revolution, the amount depending on the number of pages needed to reach lotsfree

28. 28 I/O in UNIX All I/O devices are integrated into the file system as special files All I/O devices are integrated into the file system as special files These special files are accessed like ordinary files (ie., file operations such as read, write, open are the same for special files These special files are accessed like ordinary files (ie., file operations such as read, write, open are the same for special files

29. 29 Networking in UNIX (1) Sockets are used to establish a connection between network nodes Sockets are used to establish a connection between network nodes

30. 30 Networking in UNIX (2) Sockets are created and destroyed dynamically Sockets are created and destroyed dynamically Creating a socket returns a file descriptor, which is needed for establishing a connection, reading data, writing data, and releasing the connection Creating a socket returns a file descriptor, which is needed for establishing a connection, reading data, writing data, and releasing the connection One party makes a listen call on a local socket, which creates a buffer and blocks until data arrive One party makes a listen call on a local socket, which creates a buffer and blocks until data arrive The other party makes a connect call giving as parameters the file descriptor of the local socket and the address of a remote socket (a sockets has an address in the network like the internet) The other party makes a connect call giving as parameters the file descriptor of the local socket and the address of a remote socket (a sockets has an address in the network like the internet) Once a connection is established, a socket functions like a pipe Once a connection is established, a socket functions like a pipe

31. 31 UNIX I/O (1) When a user accesses a special file, the file system determines the major and minor device numbers and whether it is a block or character special file When a user accesses a special file, the file system determines the major and minor device numbers and whether it is a block or character special file Major device number is used to index into either bdevsw array for block special or cdevsw for character special files Major device number is used to index into either bdevsw array for block special or cdevsw for character special files These structures contain pointers to the procedures to open the device, read, write etc., Some of the fields of a typical cdevsw table are shown below These structures contain pointers to the procedures to open the device, read, write etc., Some of the fields of a typical cdevsw table are shown below

32. 32 UNIX I/O (2) The UNIX I/O system in BSD C-list

33. 33 UNIX I/O (3) For block special files (eg., disks) the blocks are cached in a buffer cache For block special files (eg., disks) the blocks are cached in a buffer cache The buffer cache works for both reads and writes The buffer cache works for both reads and writes Usually dirty (modified) blocks are written to the disk in every 30 seconds Usually dirty (modified) blocks are written to the disk in every 30 seconds For character special devices, data is buffered in a chain of C-lists. A C-list block is 64 characters long, plus a count and a pointer to the next block (BSD method of character buffering) For character special devices, data is buffered in a chain of C-lists. A C-list block is 64 characters long, plus a count and a pointer to the next block (BSD method of character buffering)

34. 34 The UNIX File System (1) Some important directories found in most UNIX systems

35. 35 The UNIX File System (2) Before linking. Before linking. After linking. After linking. (a) Before linking. (b) After linking

36. 36 The UNIX File System (3) Separate file systems Separate file systems After mounting After mounting (a) Before mounting(b) After Mounting

37. 37 Locking Files in UNIX (1) Accessing a file by several processes need some critical section management Accessing a file by several processes need some critical section management This is done by locks This is done by locks A lock is defined by a file name, the starting byte and the number of bytes A lock is defined by a file name, the starting byte and the number of bytes When placing a lock, the process specifies to When placing a lock, the process specifies to Block: when the existing block is removed, the process is unblocked and the lock is placed Block: when the existing block is removed, the process is unblocked and the lock is placed Not to block: the system call returns with a status code telling whether the lock succeeded or not Not to block: the system call returns with a status code telling whether the lock succeeded or not

38. 38 Locking Files (2) ( a) File with one lock (b) Addition of a second lock (c) A third lock

39. 39 System Calls for File Management s is an error code (-1 if an error has occured) s is an error code (-1 if an error has occured) fd is a file descriptor (a positive number: 0 standard input) fd is a file descriptor (a positive number: 0 standard input) position is a file offset position is a file offset

40. 40 The stat System Call Fields returned by the stat system call

41. 41 System Calls for Directory Management s is an error code s is an error code dir identifies a directory stream dir identifies a directory stream dirent is a directory entry dirent is a directory entry

42. 42 UNIX File System (1) Disk layout in classical UNIX systems Block 0 is the boot block Block 0 is the boot block Block 1 is the superblock which contains information about the layout of the file system, including the number of i-nodes, number of disk blocks, and start of the list of free disk blocks Block 1 is the superblock which contains information about the layout of the file system, including the number of i-nodes, number of disk blocks, and start of the list of free disk blocks

43. 43 UNIX File System (2) Directory entry fields. Structure of the i-node in System V

44. 44 UNIX File System (3) The relation between the file descriptor table, the open file description The relation between the file descriptor table, the open file description File descriptor table is indexed by the fd parameter and has one entry for each file

45. 45 UNIX File System (4) A BSD directory with three files.The same directory after the file voluminous has been removed A BSD directory with three files.The same directory after the file voluminous has been removed File name can be 255 characters long File name can be 255 characters long The first 4 fields are fixed length The first 4 fields are fixed length

46. 46 Security in UNIX (1) Each UNIX user has a UID (User ID). A UID is an integer between 0 and 65536. Files, processes and other resources are marked with the UID of their owner Each UNIX user has a UID (User ID). A UID is an integer between 0 and 65536. Files, processes and other resources are marked with the UID of their owner The user with UID 0 is the superuser The user with UID 0 is the superuser Users can be organized in groups, which are also numbered with 16-bit GID’s (Group ID) Users can be organized in groups, which are also numbered with 16-bit GID’s (Group ID)

47. 47 Security in UNIX (2) Some examples of file protection modes

48. 48 System Calls for File Protection s is an error code s is an error code uid and gid are the UID and GID, respectively uid and gid are the UID and GID, respectively