1. Introduction to Operating Systems CS-2301, B-Term 20091 Introduction to Operating Systems CS-2301, System Programming for Non-Majors (Slides include materials from The C Programming Language, 2 nd edition, by Kernighan and Ritchie and from C: How to Program, 5 th and 6 th editions, by Deitel and Deitel)

2. Introduction to Operating Systems CS-2301, B-Term 20092 Why an Intro to Operating Systems? This is a System Programming Course For people who are not CS majors (Nearly) every programming task in real- life includes working with an OS Inevitably will have to deal with principle OS features Even if not knowledgeable in their designs

3. Introduction to Operating Systems CS-2301, B-Term 20093 Class Discussion What is an Operating System? (Laptops closed, please!)

4. Introduction to Operating Systems CS-2301, B-Term 20094 What Operating Systems have you Used? (Other than Windows, Linux, Mac-OS, Unix)

5. Introduction to Operating Systems CS-2301, B-Term 20095 What is an Operating System? Characteristics –Large, complex set of programs –Long-lived, evolving –Worked on by many people for many years Functions –Creates abstractions –Multiplexes concurrent activities –Manages resources –Mediates access to hardware devices –Provides a variety of services to users and applications –… Large = 10 8 –10 9 lines of code (Windows and Linux) 10 7 line of code for a real-time OS. Some systems smaller

6. Introduction to Operating Systems CS-2301, B-Term 20096 Definition – Abstraction The distillation of a complex mechanism into a simple, conceptual model User of abstraction does not need to worry about details Implementer of abstraction does not need to worry about how user will use it within limits We have already had this notion in the definition of function in C Abstraction is a collection of functions and data for creating a simple model

7. Introduction to Operating Systems CS-2301, B-Term 20097 What is an Operating System? Characteristics –Large, complex set of programs –Long-lived, evolving –Worked on by many people for many years Functions –Creates abstractions –Multiplexes concurrent activities –Manages resources –Mediates access to hardware devices –Provides a variety of services to users and applications –… What operating system services have we used already in this course?

8. Introduction to Operating Systems CS-2301, B-Term 20098 Operating Systems Typically –Long-lived –Frequently extended and updated –Worked on by many developers –Used and, maybe, abused by a variety of users with varying expertise and expectations Essential to create an acceptable computing environment to create and execute other programs that achieve business or personal goals

9. Introduction to Operating Systems CS-2301, B-Term 20099 Important new category E.g.:– Windows Linux Unix Kinds of operating systems Mainframe Operating Systems Server Operating Systems Multiprocessor Operating Systems Personal Computer Operating Systems Mobile Phone Operating Systems Handheld Computer Operating Systems Embedded Operating Systems Sensor Node Operating Systems Real-time Operating Systems Smart-card Operating Systems …

10. Introduction to Operating Systems CS-2301, B-Term 200910 Some operating systems you may (or may not) have heard about z/OS VMS/Open VMS VxWorks RT Linux QNX Neutrino eCOS BrickOS/LeJos TinyOS Arduino … iPhone OS Android Symbian Blackberry OS PalmOS/Garnet Windows Mobile …

11. Introduction to Operating Systems CS-2301, B-Term 200911 OS and Hardware OS mediates programs’ access to hardware –Computation – CPU –Storage – volatile (memory) and persistent (disk) –Networks – NIC, protocols –I/O devices – sound cards, keyboards, displays

12. Introduction to Operating Systems CS-2301, B-Term 200912 Four Fundamental Abstractions Processes & threads Multiplexing of processor(s) to create the illusion of many of them Virtual memory Multiplexing of physical memory and disk blocks to create illusion of own memory per process Files – i.e., persistent storage Organizing principles about long-term data storage Sockets & Connections Organizing principles about network communication

13. Introduction to Operating Systems CS-2301, B-Term 200913 Four Fundamental Abstractions Processes & threads Multiplexing of processor(s) to create the illusion of many of them Virtual memory Multiplexing of physical memory and disk blocks to create illusion of own memory per process Files – i.e., persistent storage Organizing principles about long-term data storage Sockets & Connections Organizing principles about network communication

14. Introduction to Operating Systems CS-2301, B-Term 200914 Definition – Process A particular execution of a program Different from all other executions of that program Different from executions of other programs The OS uses one or more CPUs to make it seem like each process has its own CPU Can execute at same time! Uses interrupts to manage and enforce multiplexing of CPU

15. Introduction to Operating Systems CS-2301, B-Term 200915 Definition – Interrupt A mechanism by which the processor suspends execution of the current, running program and gives control to the OS OS saves the state of the interrupted program so that it can be restarted later OS then takes appropriate action

16. Introduction to Operating Systems CS-2301, B-Term 200916 Why Processes? Enables programmers –to completely disengage from issues of concurrent execution of independent programs –to build applications with more than one concurrent activity Enables independent applications to share a computing system –Safely!

17. Introduction to Operating Systems CS-2301, B-Term 200917 Why Processes (continued)? Exploit modern processors –Capable of executing multiple, simultaneous, program executions –Interleaved at instruction level or even memory access level Moore’s Law:– –Integrated circuit components shrink in size by 50% every 18 months –Double in speed every 18 months Modern limitation due to power dissipation.

18. Introduction to Operating Systems CS-2301, B-Term 200918 Resources Assigned to a Process Memory Virtual or real Processor time Priorities Deadlines for real-time activities Privileges Security, authentication, etc. Files and file space For long-term storage, temporary storage Devices For input and output activity, sensors, etc.

19. Introduction to Operating Systems CS-2301, B-Term 200919 Resources (continued) Managed by OS Protection and isolation from other processes Allocation according to defined policies Enforcement of limits, etc. …

20. Introduction to Operating Systems CS-2301, B-Term 200920 Shell / Command Prompt Linux Shell is a process Windows Command Prompt is a process Created when you log on or connect to system (e.g., via PuTTY) Open Command Prompt, konsole, xterm, etc., window –Reads what you type (and displays it in your window) –Interprets lines as commands and arguments –Creates a process for each command, passes args –(Typically) waits for process to complete before interpreting next line

21. Introduction to Operating Systems CS-2301, B-Term 200921 Window Manager Window Manager is a process Tracks mouse movements, key clicks, menu actions “Open” an application means … Create process for that application; give it a window “Open” a document means … If application is not open, create process for it Pass document as argument to application

22. Introduction to Operating Systems CS-2301, B-Term 200922 Creating and Deleting Processes A process is created … –… at system boot time The first process Built-in processes – e.g., in embedded systems OR –… by another process Possibly in response to an action by a (human) user A process is deleted … –When its program exits –By another process – killed or paused (by debugger) –When system crashes or shuts down

23. Introduction to Operating Systems CS-2301, B-Term 200923 Questions about Processes?

24. Introduction to Operating Systems CS-2301, B-Term 200924 Four Fundamental Abstractions Processes & threads Multiplexing of processor(s) to create the illusion of many of them Virtual memory Multiplexing of physical memory and disk blocks to create illusion of separate memory per process Files – i.e., persistent storage Organizing principles about long-term data storage Sockets & Connections Organizing principles about network communication

25. Introduction to Operating Systems CS-2301, B-Term 200925 Virtual Memory Definition:– the illusion that a process has its own, independent memory (Often) more memory than machine has installed May be implemented using interrupts, pages, and disk blocks Swapping fast enough so process is unaware May be implemented by partitioning Swapping not necessary for real-time activities

26. Introduction to Operating Systems CS-2301, B-Term 200926 Independence of Virtual Memories A process cannot even see the virtual memory of another process A process cannot even see the memory used by the OS I.e., no possible pointer value can point to something in a different virtual memory Separate, parallel universes Except where explicitly linked together

27. Introduction to Operating Systems CS-2301, B-Term 200927 Typical Virtual Memory for Process (Windows & Linux) 0x00000000 0xFFFFFFFF address space program code (text) static data heap (dynamically allocated) stack (dynamically allocated) PC SP

28. Introduction to Operating Systems CS-2301, B-Term 200928 Typical Virtual Memory for Process (Windows & Linux) 0x00000000 0xFFFFFFFF address space program code (text) static data heap (dynamically allocated) stack (dynamically allocated) PC SP Every process has one of these. Independent of all others.

29. Introduction to Operating Systems CS-2301, B-Term 200929 Virtual Memory (continued) Typical virtual memory size – 4 GBytes Per process — even in a 1 GByte computer! “Inactive” pages are not resident in RAM Reference results in interrupt called a page fault OS brings in page from disk, (maybe) swaps one out Unused pages not filled in by OS Dereferencing pointer results in Segmentation Fault New pages created by OS as needed When stack or heap grows or programs are loaded

30. Introduction to Operating Systems CS-2301, B-Term 200930 Virtual Memory in Embedded Systems Implemented by partitioning RAM No swapping in or out May not even have a disk! Also in smart-phone operating systems

31. Introduction to Operating Systems CS-2301, B-Term 200931 Virtual Memories in Embedded System OS Kernel stack Process 1 stack Process 2 0x00000000 0x0000FFFF Physical memory stack Process 3

32. Introduction to Operating Systems CS-2301, B-Term 200932 Questions about Processes or Virtual Memory? Not in Kernighan & Ritchie

33. Introduction to Operating Systems CS-2301, B-Term 200933 Threads A refinement of concept of process Short for “thread of control” Concurrent execution of a function within the context of a process Needs own stack in order to call other functions Shares heap, static data, and code with other threads of same process Reason Application may need to manage concurrency of its own computation with external events

34. Introduction to Operating Systems CS-2301, B-Term 200934 Virtual Memory for Multiple Threads 0x00000000 0xFFFFFFFF Virtual address space code (text) static data heap thread 1 stack PC (T2) SP (T2) thread 2 stack thread 3 stack SP (T1) SP (T3) PC (T1) PC (T3)

35. Introduction to Operating Systems CS-2301, B-Term 200935 Why Threads? To enable development of applications with concurrent activities inside them Need to share data (difficult with separate processes) Examples Web server over common data pages Transaction processor over common data base Applications within a mobile phone or PDA Applications with different speeds of devices …

36. Introduction to Operating Systems CS-2301, B-Term 200936 Thread Interface – POSIX standard #include int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine) (void), void *arg) –creates a new thread of control –new thread begins executing at start_routine pthread_exit(void *value_ptr) –terminates the calling thread pthread_join(pthread_t thread, void **value_ptr) –blocks the calling thread until the thread specified terminates pthread_t pthread_self() –Returns the calling thread's identifier

37. Introduction to Operating Systems CS-2301, B-Term 200937 Thread Interface – POSIX standard #include int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine) (void), void *arg) –creates a new thread of control –new thread begins executing at start_routine pthread_exit(void *value_ptr) –terminates the calling thread pthread_join(pthread_t thread, void **value_ptr) –blocks the calling thread until the thread specified terminates pthread_t pthread_self() –Returns the calling thread's identifier Pointer to a function– see §5.7 of K & R

38. Introduction to Operating Systems CS-2301, B-Term 200938 Thread Interface – POSIX standard #include int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine) (void), void *arg) –creates a new thread of control –new thread begins executing at start_routine pthread_exit(void *value_ptr) –terminates the calling thread pthread_join(pthread_t thread, void **value_ptr) –blocks the calling thread until the thread specified terminates pthread_t pthread_self() –Returns the calling thread's identifier Arguments to that function

39. Introduction to Operating Systems CS-2301, B-Term 200939 Thread Example pthread_create(tp, &f, &args) pthread_join(tp) main function f g f h f

40. Introduction to Operating Systems CS-2301, B-Term 200940 Thread Example pthread_create(tp, &f, &args) pthread_join(tp) main function f g f h f Two threads run at the same time

41. Introduction to Operating Systems CS-2301, B-Term 200941 Why Threads? To enable development of applications with concurrent activities inside them Need to share data (difficult with separate virtual memories) Examples Web server over common data pages Transaction processor over common data base Applications within a mobile phone or PDA Applications with different speeds of devices …

42. Introduction to Operating Systems CS-2301, B-Term 200942 Example Thread Application One or more threads to get data from sensor 1000 times per second; cannot afford to miss a reading Places data on queue Another thread processes and displays data Removes and processes each data item from queue Displays system state on control panel User thread Allows operator to adjust system parameters While other threads are still running

43. Introduction to Operating Systems CS-2301, B-Term 200943 Additional Functions & Data Required pthread_mutex_t — mutual exclusion lock –Enables a thread to “lock” some data so that other threads do not touch in mid-operation pthread_cond_t — condition variable –Enables a thread to wait for an event to happen –Signaled by another thread See man pages for details

44. Introduction to Operating Systems CS-2301, B-Term 200944 Questions about Threads? Not in Kernighan & Ritchie

45. Introduction to Operating Systems CS-2301, B-Term 200945 Four fundamental Abstractions Processes & threads Multiplexing of processor(s) to create the illusion of many of them Virtual memory Multiplexing of physical memory and disk blocks to create illusion of own memory per process Files & persistent storage Organizing principles about long-term data storage Sockets & connections Organizing principles about network communication

46. Introduction to Operating Systems CS-2301, B-Term 200946 Definition – File A (potentially) large amount of information or data that lives a (potentially) very long time Often much larger than the memory of the computer Often much longer than any computation Sometimes longer than life of machine itself (Usually) organized as a linear array of bytes or blocks Internal structure is imposed by application (Occasionally) blocks may be variable length (Often) requiring concurrent access by multiple threads or processes Even by processes on different machines!

47. Introduction to Operating Systems CS-2301, B-Term 200947 Implementations of Files Usually on disks (or devices that mimic disks) Magnetic – hard drive or floppy Optical – CD, DVD Flash drives – electronic memory, organized as disks Requirement Preserve data contents during power-off or disasters Directory / Folder Special kind of file that contains links pointing to other files Associates names with files

48. Introduction to Operating Systems CS-2301, B-Term 200948 Implementations of Files Usually on disks (or devices that mimic disks) Magnetic – hard drive or floppy Optical – CD, DVD Flash drives – electronic memory, organized as disks Requirement Preserve data contents during power-off or disasters Directory / Folder Special kind of file that contains links pointing to other files Associates names with files Older systems also used magnetic tape, paper tape, trays of punched cards, etc.

49. Introduction to Operating Systems CS-2301, B-Term 200949 File Access in C See Kernighan & Ritchie, Chapter 8 Raw file access Without simplifying stream functions – e.g., –scanf, fscanf, printf, fprintf, fgetc, etc. read and write raw disk blocks Seek to a file position –lseek, fseek — sets file pointer to specified location –Subsequent read, write, etc., start there –ftell – returns file pointer

50. Introduction to Operating Systems CS-2301, B-Term 200950 Organizations of Files Contiguous Blocks stored contiguously on storage medium E.g., CD, DVD, some large database systems Access time to any block is O(1) Linked Blocks linked together – File Allocation Table (FAT) Access time is O(n) Indexed Blocks accessed via tree of index blocks (i-nodes) Access time is O(log n) However, base of logarithm may be very large (>100)

51. Introduction to Operating Systems CS-2301, B-Term 200951 Organizations of Files Contiguous Blocks stored contiguously on storage medium E.g., CD, DVD, some large database systems Access time to any block is O(1) Linked Blocks linked together – File Allocation Table (FAT) Access time is O(n) Indexed Blocks accessed via tree of index blocks (i-nodes) Access time is O(log n) However, base of logarithm may be very large (>100) NTFS and Linux file systems on hard drives

52. Introduction to Operating Systems CS-2301, B-Term 200952 Organizations of Files Contiguous Blocks stored contiguously on storage medium E.g., CD, DVD, some large database systems Access time to any block is O(1) Linked Blocks linked together – File Allocation Table (FAT) Access time is O(n) Indexed Blocks accessed via tree of index blocks (i-nodes) Access time is O(log n) However, base of logarithm may be very large (>100) Typical camera chips, flash drives, floppies

53. Introduction to Operating Systems CS-2301, B-Term 200953 Questions about Files?

54. Introduction to Operating Systems CS-2301, B-Term 200954 Four fundamental Abstractions Processes & threads Multiplexing of processor(s) to create the illusion of many of them Virtual memory Multiplexing of physical memory and disk blocks to create illusion of own memory per process Files & persistent storage Organizing principles about long-term data storage Sockets & connections Organizing principles about network communication

55. Introduction to Operating Systems CS-2301, B-Term 200955 Sockets and Connections Connection: –a serial conversation over a network between two end points e.g., processes, threads, tasks on different computers –organized as a sequence of messages or datagrams –distinct from all other connections Socket: –An end point of a connection –An abstraction that allows a process to send or receive only the information of that connection –Multiplexed on network with all other connections

56. Introduction to Operating Systems CS-2301, B-Term 200956 Sockets and Connections Defer to another course –CS-4513, Distributed Systems –CS-4514, Computer Networks

57. Introduction to Operating Systems CS-2301, B-Term 200957 General Questions?