1. University of Maryland, CMSC 412, Fall 2002 1 CMSC 412 Operating Systems Fall 2002 Liviu Iftode iftode@cs.umd.edu

2. University of Maryland, CMSC 412, Fall 2002 2 Course overview Goals Understand how an operating system works as a mediator between the computer architecture and user programs Learn how OS concepts are implemented in a real operating system Introduce to systems programming Learn about performance evaluation Learn about current trends in OS research

3. University of Maryland, CMSC 412, Fall 2002 3 OS Learning OS Concepts Real OSOS Programming OS Implementation (lectures, textbooks) recitations homeworks project (Unix/Linux textbooks) (man pages) (source code, project doc)

4. University of Maryland, CMSC 412, Fall 2002 4 Course Timeline Lecture Lab Homework Project W Apply A HW A Due W W Project AB Due W Apply B W Apply C TuTh Concept A TuTh Concept B TuTh Concept C HW BC Due W

5. University of Maryland, CMSC 412, Fall 2002 5 Suggested Approach Read the assigned chapter from the textbook before the lecture to understand the basic idea and become familiar with the terminology Attend the recitation Start homeworks and project right away, systems programming is not easy ! Ask questions during lecture, recitation. Use the mailing list/newsgroup for discussions, do not be afraid to answer a question posted by your colleague even if you are not sure. This is a way to validate your understanding of the material. Do not forget, questions and discussions are not graded !

6. University of Maryland, CMSC 412, Fall 2002 6 Course Outline Processes and process management Threads and thread programming Synchronization and Deadlock Memory management and Virtual Memory CPU Scheduling File systems and I/O Management Networking and Distributed Systems Security

7. University of Maryland, CMSC 412, Fall 2002 7 Course requirements Prerequisites computer architecture good programming skills (C!!, C++, Java) Expected work substantial readings (textbooks and papers) challenging project extended over the entire semester homeworks (require programming) midterm and final exams

8. University of Maryland, CMSC 412, Fall 2002 8 Work Evaluation midterm exam 25% homework 25% project 25% final exam 25%

9. University of Maryland, CMSC 412, Fall 2002 9 Homeworks Goals Deepen the understanding of OS concepts Develop systems programming skills: virtual memory, threads, synchronization, sockets Learn to design, implement, debug and evaluate the performance of an OS-bound program Structure 4-5 homeworks Both theoretical and C-programming problems

10. University of Maryland, CMSC 412, Fall 2002 10 Project Goals learn to design, implement and evaluate basic OS mechanisms and policies Structure individual project multiple phases project report for each phase

11. University of Maryland, CMSC 412, Fall 2002 11 Textbooks Stallings Operating Systems. Internals and Design Principles, 4th Edition, Prentice-Hall, 2001. Silberschatz, Galvin and Gagne, Operating System Concepts, 6th Edition, John Wiley & Sons, 2001. Papers will be made available on the course homepage

12. University of Maryland, CMSC 412, Fall 2002 12 Logistics TAs: Chunyuan Liao Iulian Neamtiu (to be confirmed) Course homepage: http://www.cs.umd.edu/~iftode/cs412/cs412syllabus.htm Preliminary schedule and course notes are already available for the entire semester but they may be updated before each class. Homeworks every other week. A project phase every other week. A mailing list/newsgroup will be announced shortly.

13. University of Maryland, CMSC 412, Fall 2002 13 What is an operating system a software layer between the hardware and the application programs/users which provides a virtual machine interface: easy and safe a resource manager that allows programs/users to share the hardware resources: fair and efficient hardware operating system application (user)

14. University of Maryland, CMSC 412, Fall 2002 14 How does an OS work receives requests from the application: system calls satisfies the requests: may issue commands to hardware handles hardware interrupts: may upcall the application OS complexity: synchronous calls + asynchronous events hardware OS application (user)system callsupcalls commands interrupts hardware independent hardware dependent

15. University of Maryland, CMSC 412, Fall 2002 15 Files traditional approach: OS does disk block allocation and caching (buffer cache), disk operation scheduling and replacement in the buffer cache new approaches: application-controlled cache replacement, log-based allocation (makes writes fast) hardware: disk operating system: files, directories A file is a storage abstraction application/user: copy file1 file2 naming, protection, operations on files operations on disk blocks...

16. University of Maryland, CMSC 412, Fall 2002 16 Traditional file system application: read/write files OS: translate file to disk blocks...buffer cache... maintains controls disk accesses: read/write blocks hardware:

17. University of Maryland, CMSC 412, Fall 2002 17 Mechanism vs Policy mechanism: data structures and operations that implement the abstraction (e.g. the buffer cache) policy: the procedure that guides the selection of a certain course of action from among alternatives (e.g. the replacement policy for the buffer cache) traditional OS is rigid: mechanism together with policy hardware operating system: mechanism+policy application (user)

18. University of Maryland, CMSC 412, Fall 2002 18 traditional OS cannot provide the best policy in all cases new OS approaches separate mechanisms from policies: OS provides the mechanism + some policy applications contribute to the policy flexibility+efficiency require new OS structures and/or new OS interfaces Mechanism-policy split

19. University of Maryland, CMSC 412, Fall 2002 19 Application-controlled caching application: read/write files replacement policy OS: translate file to disk blocks...buffer cache... maintains controls disk accesses: read/write blocks hardware:

20. University of Maryland, CMSC 412, Fall 2002 20 Processes traditional approach: OS switches processes on the processor (process scheduling), provides inter-process communication and handles exceptions new approaches: application-controlled scheduling, reservation-based scheduling, agile applications hardware: processor operating system: processes A process is a processor abstraction user: run application create, kill processes, inter-process comm context switch

21. University of Maryland, CMSC 412, Fall 2002 21 Traditional approach OS mediates inter-process communication (IPC) OS schedules processes on the processor application provides hints: priorities processor OS processes IPC active

22. University of Maryland, CMSC 412, Fall 2002 22 Hierarchical scheduling OS schedules schedulers which schedule dependent processes processor OS processes schedulers

23. University of Maryland, CMSC 412, Fall 2002 23 Virtual memory traditional approach: OS provides a sufficiently large virtual address space for each running application, does memory allocation and replacement and may ensure protection new approaches: external memory management, huge (64- bit) address space, global memory hardware: physical memory operating system: virtual memory Virtual memory is a memory abstraction application: address space virtual addresses physical addresses

24. University of Maryland, CMSC 412, Fall 2002 24 VM: mechanism and policy processes can run being partially loaded in memory illusion of more memory than physically available: swapping processes can share physical memory if permitted replacement policy can be exposed to the application physical memory: v-to-p memory mappings processes: virtual address spaces p1 p2

25. University of Maryland, CMSC 412, Fall 2002 25 Communication traditional approach: OS provides naming schemes, reliable transport of messages, packet routing to destination new approaches: zero-copy protocols, active messages, memory-mapped communication hardware: network interface operating system: TCP/IP protocols Message passing is a communication abstraction application: sockets naming, messages network packets

26. University of Maryland, CMSC 412, Fall 2002 26 Traditional OS structure monolithic/layered systems one/N layers all executed in “kernel-mode” good performance but rigid OS kernel hardware user process file system memory system user system calls

27. University of Maryland, CMSC 412, Fall 2002 27 Micro-kernel OS micro-kernel hardware client process file server memory server IPC user mode client-server model, IPC between clients and servers the micro-kernel provides protected communication OS functions implemented as user-level servers flexible but efficiency is the problem easy to extend for distributed systems

28. University of Maryland, CMSC 412, Fall 2002 28 Extensible OS kernel extensible kernel hardware process default memory service user mode process my memory service user processes can load customized OS services into the kernel good performance but protection and scalability become problems

29. University of Maryland, CMSC 412, Fall 2002 29 Virtual Machines old concept which is heavily revived today the real hardware in “cloned” in several identical virtual machines OS functionality built on top of the virtual machine hardware user exokernel allocate resource OS on virtual machine

30. University of Maryland, CMSC 412, Fall 2002 30 Computer System Processor: performs data processing Main memory: stores both data and programs, typically volatile Disks: secondary memory devices which provide persistent storage Network interfaces: inter-machine communication Buses: intra-machine communication memory bus (processor-memory) I/O bus (disks, network interfaces, other I/O devices, memory-bus)

31. University of Maryland, CMSC 412, Fall 2002 31 Basic computer structure CPU Memory memory bus I/O bus diskNet interface

32. University of Maryland, CMSC 412, Fall 2002 32 Memory caches motivated by the mismatch between processor and memory speed closer to the processor than the main memory smaller and faster than the main memory act as “attraction memory”: contains the value of main memory locations which were recently accessed (temporal locality) transfer between caches and main memory is performed in units called cache blocks/lines caches contain also the value of memory locations which are close to locations which were recently accessed (spatial locality)

33. University of Maryland, CMSC 412, Fall 2002 33 Cache design issues cache size and cache block size mapping: physical/virtual caches, associativity replacement algorithm: LRU write policy: write through/write back cpu cache main memory word transfer block transfer

34. University of Maryland, CMSC 412, Fall 2002 34 Memory Hierarchy cpu cache main memory word transfer block transfer disks page transfer decrease cost per bit decrease frequency of access increase capacity increase access time increase size of transfer unit

35. University of Maryland, CMSC 412, Fall 2002 35 Data transfer on the bus CPU Memory memory bus I/O bus diskNet interface cache cache-memory: cache misses, write-through/write-back memory-disk: swapping, paging, file accesses memory-Network Interface : packet send/receive I/O devices to the processor: interrupts

36. University of Maryland, CMSC 412, Fall 2002 36 Direct Memory Access (DMA) bulk data transfer between memory and an I/O device (disk, network interface) initiated by the processor address of the I/O device starting location in memory number of bytes direction of transfer (read/write from/to memory) processor interrupted when the operation completes bus arbitration between cache-memory and DMA transfers memory cache must be consistent with DMA

37. University of Maryland, CMSC 412, Fall 2002 37 Multiprocessors CPU Memory memory bus I/O bus diskNet interface cache simple scheme: more than one processor on the same bus memory is shared among processors-- cache consistency bus contention increases -- does not scale alternative (non-bus) system interconnect -- expensive single-image operating systems CPU cache

38. University of Maryland, CMSC 412, Fall 2002 38 Multicomputers network of computers: “share-nothing” -- cheap communication through message-passing: difficult to program challenge: build efficient shared memory abstraction in software each system runs its own operating system CPU Memory memory bus I/O bus diskNet interface cache CPU Memory memory bus I/O bus disk Net interface cache network