Presentation on theme: "SYSC3001 Operating Systems and Data Bases"— Presentation transcript:
1 SYSC3001 Operating Systems and Data Bases From calendar:Management of CPU (scheduling) processes, memory, disk, and files.Implications of concurrency.Concurrent programming, including interprocess communications in distributed systems.Data base models and query languages.
2 Computer Systems Overview Basically Chapter 1 in textbookNeed some basic understanding of computer organization and hardware:processormemoryI/O devices“Left as an exercise”
3 Chapter 2: Operating Systems Overview What is an Operating System?Services provided by an OSEvolution of the OSDesired Hardware Features to support the OSMultiprogrammingTime Sharing SystemsDifficulties of the OS DesignAchievements of the OSSystem Structure of the OSModern OS
4 Operating SystemIs a program that controls the execution of application programsTells the CPU when to execute other programsUses the hardware efficientlyIs an interface between the user and hardwareMasks the details of the hardware to application programsHence OS must deal with hardware detailsMake the system convenient to use
5 Layers and Views The OS typically is further divided into multiple layers.
6 Operating Systems Definitions Resource manager: manages and allocates resources (CPU, memory, I/O, software processes)Control program: control the execution of user programsKernel: the one program running at all times (all else being application programs)
7 Services Provided by the OS Facilities for Program creationeditors, compilers, linkers, and debuggersProgram executionloading in memory, I/O and file initializationAccess to I/O and filesdeals with the specifics of I/O and file formatsSystem accessProtection in access to resources and dataResolves conflicts for resource contention
8 Services Provided by the OS Error Detectioninternal and external hardware errorsmemory errordevice failuresoftware errorsarithmetic overflowaccess forbidden memory locationsInability of OS to grant request of applicationError Responsesimply report error to the applicationRetry the operationAbort the application
9 Services Provided by the OS Accountingcollect statistics on resource usagemonitor performance (e.g., response time)used for system parameter tuning to improve performanceuseful for anticipating future enhancementsused for billing users (on multiuser systems)
10 Quiz 2-1 What is an OS? Name two OS services. Application program Interface between user programsResource managerKernelName two OS services.
11 Evolution of the Operating System Must adapt to hardware upgrades and new types of hardware. Examples:Character vs. graphic terminalsIntroduction of paging hardwareMust offer new services, e.g., Internet supportThe need to change the OS software on a regular basismodular construction with clean interfacesobject oriented methodology
12 Simple Batch Systems First generation OS (mid-50s) Main objective is to maximize utilizationThe user submits a job (written on card or tape) to a computer operatorThe computer operator places a batch of several jobs on a input deviceThe OS, the monitor, manages the execution of each program in the batchMonitor mostly is in main memory and available for execution, Resident MonitorMonitor utilities are loaded when needed
13 The Monitor Monitor reads in jobs one at a time from the input device Monitor places a job in the user program areaA monitor instruction branches to the start of the user programExecution of user program continues until:end-of-program occurserror occursCauses the CPU to fetch its next instruction from MonitorMonitor is like a babysitter taking care of several kids (processes) who share one or more toys (CPU, memory, …).What are some policies in this case?
14 Job Control Language (JCL) Is the special type language to provide instructions to the monitorwhat compiler to usewhat data to useExample of job format:$JOB shows a new job$FTN loads the compiler and transfers control to it$LOAD loads the object code (in place of compiler)$RUN transfers control to user program$JOB$FTN...FORTRANprogram$LOAD$RUNData$END
15 Job Control Language (JCL) During user program execution:Each read (or input) instruction (in user program) causes one line of input data to be readCauses an input routine to be invokedchecks for not reading a JCL lineskips to the next JCL line at completion of user program
16 Simple Batch OSAlternates execution between a user program and the monitor programRelies on available hardware to effectively alternate execution from various parts of memory
17 Desirable Hardware Features Memory protectiondo not allow the memory area containing the monitor to be altered by user programsTimerprevents a job from monopolizing the systeman interrupt occurs when time expires
18 Desirable Hardware Features Privileged instructionscan be executed only by the monitoran interrupt occurs if a program tries these instructionsInterruptsprovides flexibility for relinquishing control to and regaining control from user programs
20 Main Problem with Batch Systems - System Utilization Example The system works. However, the product cost millions of dollars, but the utilization is only single digit. Customers are not happy, managers/sales are pressured. Engineers (both system/software engineers and hardware engineers) have to do something to improve it.But HOW?
21 Multiprogrammed Batch Systems I/O operations are exceedingly slow (compared to instruction execution)A program containing even a very small number of I/O operations will spend most of its time waiting for themHence: poor CPU usage when only one program is present in memory
22 Multiprogrammed Batch Systems If memory can hold several programs, then CPU can switch to another one whenever a program is awaiting for an I/O to completeThis is multitasking (multiprogramming)
23 Requirements for Multiprogramming Hardware support:I/O interrupts and (possibly) DMAin order to execute instructions while I/O device is busyMemory managementseveral ready-to-run jobs must be kept in memoryMemory protection (data and programs)Software support from the OS:Scheduling (which program is to be run next)To manage resource contention and concurrency
24 Example: three jobs are submitted JOB JOB2 JOB3Type of job Heavy compute Heavy I/O Heavy I/ODuration 5 min. 15 min. 10 min.Memory req. 50M 100 M 80 MNeed disk? No No YesNeed terminal No Yes NoNeed printer? No No YesAlmost no contention for resourcesAll 3 can run in minimum time in a multitasking environment (assuming JOB2/3 have enough CPU time to keep their I/O operations active)Total memory size is 256M bytes.
25 Advantages of Multiprogramming Uniprogramming MultiprogrammingProcessor use 17% 33%Memory use 33% 67%Disk use 33% 67%Printer use 33% 67%Elapsed time 30 min. 15 min.Throughput rate 6 jobs/hr 12 jobs/hrMean response time 18 min. 10 min.Elapsed time:- uniprogramming: job 1, job2, job3 in sequence -> = 30- multiprogramming: job2 and job3 use different I/O devices, they can be overlapped with Job1 in an ideal case -> 15 (the longest time for all three jobs)Processor utilization:- Uni: 5 (for job1) / 30 (total) ~ 17%- Multi: 5 / 15 ~ 33%Disk/printer use:- Uni: 10 (job3) / 30 ~33%- Multi: 10/15 ~67%Memory use:- uni: (50x5 +100x15 +80x10)/(256x30)- multi” (( )x5 + (100+80)x x5) / (256x15)
26 Time Sharing Systems (TSS) Batch multiprogramming does not support interaction with usersTSS extends multiprogramming to handle multiple interactive jobsProcessor’s time is shared among multiple usersMultiple users simultaneously access the system through terminals
27 Time Sharing Systems (TSS) Because of slow human reaction time, a typical user needs 2 sec of processing time per minuteThen (about) 30 users should be able to share the same system without noticeable delay in the computer reaction timeThe file system must be protected (multiple users…)
28 Difficulties with OS Design Improper synchronizationensure a program waiting for an I/O device receives the signalFailed mutual exclusionmust permit only one program at a time to perform a transaction on a portion of dataDeadlockIt might happen that 2 or more programs wait endlessly after each other to perform an operation.
29 An example of deadlockProgram A wants to copy from disk1 to disk2 and takes control of disk1Program B wants to copy from disk2 to disk1 and takes control of disk2Program A must wait that program B releases disk2 and program B must wait that program A releases disk1Program A and B may wait forever
30 Quiz 2-2 Batch multiprogramming vs. TSS Key objective What is multiprogramming?Multiprogramming vs. uniprogrammingMain advantage?Challenges?True of FalseTime sharing systems (TSS) are multiprogrammingMultiprogramming systems are TSSBatch multiprogramming vs. TSSKey objectiveSource of directives to OSBatch multiprogramming vs. TSSKey objective: processor utilization vs. response timeSource of directives to OS: JCL vs. interactive commands entered at the terminal
31 Major Achievements of OS Operating Systems are among the most complex pieces of software ever developedTo meet the difficult requirements of multiprogramming and time sharing, there have been 5 major achievements by OS:Processes and threadsMemory managementInformation protection and securityScheduling and resource managementSystem structure
32 ProcessIntroduced to obtain a systematic way of monitoring and controlling pgm executionA process is an executable program with:associated data (variables, buffers…)execution context: i.e., all the information thatthe CPU needs to execute the processcontent of the processor registersthe OS needs to manage the process:priority of the processthe event (if any) after which the process is waitingother data (that we will introduce later)
33 Process Fundamental to the structure of operating systems A process can be defined as:a program in executionan instance of a running programthe entity that can be assigned to, and executed on, a processora unit of activity characterized by a single sequential thread of execution, a current state, and an associated set of system resourcesCentral to the design of operating systems is the concept of process. This term wasfirst used by the designers of Multics in the 1960s [DALE68]. It is a somewhatmore general term than job. Many definitions have been given for the term process ,including• A program in execution• An instance of a program running on a computer• The entity that can be assigned to and executed on a processor• A unit of activity characterized by a single sequential thread of execution, acurrent state, and an associated set of system resources
34 A simple implementation of processes The process index register contains the index into the process list of the currently executing process (B)A process switch from B to A consist of storing (in memory) B’s context and loading (in CPU registers) A’s contextA data structure that provides flexibility (to add new features)
35 Memory ManagementThe OS has five principal storage management responsibilities:process isolationautomatic allocation and managementsupport of modular programmingprotection and access controllong-term storageThe needs of users can be met best by a computing environment that supportsmodular programming and the flexible use of data. System managers need efficientand orderly control of storage allocation. The OS, to satisfy these requirements, hasfive principal storage management responsibilities:• Process isolation: The OS must prevent independent processes from interferingwith each other’s memory, both data and instructions.• Automatic allocation and management: Programs should be dynamicallyallocated across the memory hierarchy as required. Allocation should betransparent to the programmer. Thus, the programmer is relieved of concernsrelating to memory limitations, and the OS can achieve efficiency by assigningmemory to jobs only as needed.• Support of modular programming: Programmers should be able to define programmodules, and to create, destroy, and alter the size of modules dynamically.• Protection and access control: Sharing of memory, at any level of the memoryhierarchy, creates the potential for one program to address the memory spaceof another. This is desirable when sharing is needed by particular applications.At other times, it threatens the integrity of programs and even of the OS itself.The OS must allow portions of memory to be accessible in various ways byvarious users.• Long-term storage: Many application programs require means for storinginformation for extended periods of time, after the computer has beenpowered down.
36 Memory Management The key contribution is virtual memory It allows programs to address memory from a logical point of view without regard to the amount that is physically availableWhile a program is running only a portion of the program and data is kept in (real) memoryOther portions are kept in blocks on diskthe user has access to a memory space that is larger than real memory
38 Virtual MemoryAll memory references made by a program are to virtual memory which can be eithera linear address spacea collection of segments (variable-length blocks)The hardware (mapper) must map virtual memory address to real memory addressIf a reference is made to a virtual address not in memory, then(1) a portion of the content of real memory is swapped out to disk(2) the desired block of data is swapped in
39 File System Implements long-term store (often on disk) Information stored in named objects called filesa convenient unit of access and protection for OSFiles (and portions) may be copied into virtual memory for manipulation by programs
40 Security and Protection Access control to resourcesforbid intruders (unauthorized users) to enter the systemforbid user processes to access resources which they are not authorized to
41 Scheduling and Resource Management Differential responsivenessdiscriminate between different classes of jobsFairnessgive equal and fair access to all processes of the same classEfficiencymaximize throughput, minimize response time, and accommodate as many users as possible
42 Key Elements for Scheduling OS maintains queues of processes waiting for some resourceShort term queue of processes in memory ready to executeThe dispatcher (short term scheduler) decides who goes nextLong term queue of new jobs waiting to use the systemOS must not admit too many processesA queue for each I/O device consisting of processes that want to use that I/O device
44 System StructureBecause of it’s enormous complexity, we view the OS system as a series of levelsEach level performs a related subset of functionsEach level relies on the next lower level to perform more primitive functionsWell defined interfaces: one level can be modified without affecting other levelsThis decomposes a problem into a number of more manageable sub problems
45 Characteristics of Modern Operating Systems New design elements were introduced recentlyNew hardware developmentmultiprocessor machineshigh-speed networksfaster processors and larger memoryNew software / application needsmultimedia applicationsInternet and Web accessClient/Server applications
46 Microkernel architecture Only a few essential functions in the kernelprimitive memory management (address space)Interprocess communication (IPC)basic schedulingOther OS services are provided by processes running in user mode (servers)device drivers, file system, virtual memory…More flexibility, extensibility, portability…A performance penalty by replacing service calls with message exchanges between process...
47 MultithreadingA process is a collection of one or more threads that can run simultaneouslyUseful when the application consists of several tasks that do not need to be serializedGives the programmer a greater control over the timing of application-related eventsAll threads within the same process share the same data and resources and a part of the process’s execution contextIt is easier to create or destroy a thread or switch among threads (of the same process) than to do these with processes
48 Distributed Systems Distributed operating systems Provides the illusion of a single main memory space and single secondary memory space48
49 Symmetric Multiprocessing (SMP) A computer with multiple processorsEach processor can perform the same functions and share same main memory and I/O facilities (symmetric)The OS schedule processes/threads across all the processors (real parallelism)Existence of multiple processors is transparent to the user.Incremental growth: just add another CPU (at least in theory).Robustness: a single CPU failure does not halt the system, only the performance is reduced.
51 Quiz 2-3Name the main difference between multiprogramming and multiprocessingTrue or FalseMultiprocessing generally implies multiprogrammingMultiprogramming generally implies multiprocessing
52 Quiz 2-3 Micorkernel systems: What extra “weight” (features) do they usually remove?What are the main advantages?What is the main cost?
53 Virtual Machines and Virtualization enables a single PC or server to simultaneously run multiple operating systems or multiple sessions of a single OSa machine can host numerous applications, including those that run on different operating systems, on a single platformhost operating system can support a number of virtual machines (VM)each has the characteristics of a particular OS and, in some versions of virtualization, the characteristics of a particular hardware platformTraditionally, applications have run directly on an OS on a PC or a server. Each PCor server would run only one OS at a time. Thus, the vendor had to rewrite partsof its applications for each OS/platform they would run on. An effective strategyfor dealing with this problem is known as virtualization . Virtualization technologyenables a single PC or server to simultaneously run multiple operating systems ormultiple sessions of a single OS. A machine with virtualization can host numerousapplications, including those that run on different operating systems, on a singleplatform. In essence, the host operating system can support a number of virtualmachines (VM) , each of which has the characteristics of a particular OS and, in someversions of virtualization, the characteristics of a particular hardware platform.The VM approach is becoming a common way for businesses and individualsto deal with legacy applications and to optimize their hardware usage by maximizingthe number of kinds of applications that a single computer can handle [GEER09].Commercial VM offerings by companies such as VMware and Microsoft are widelyused, with millions of copies having been sold. In addition to their use in serverenvironments, these VM technologies also are used in desktop environments to runmultiple operating systems, typically Windows and Linux.
54 Virtual Memory Concept The specific architecture of the VM approach varies among vendors.Figure 2.13 shows a typical arrangement. The virtual machine monitor (VMM) , orhypervisor , runs on top of (or is incorporated into) the host OS. The VMM supportsVMs, which are emulated hardware devices. Each VM runs a separate OS.The VMM handles each operating system’s communications with the processor,the storage medium, and the network. To execute programs, the VMM hands offthe processor control to a virtual OS on a VM. Most VMs use virtualized networkconnections to communicate with one another, when such communication is needed.Key to the success of this approach is that the VMM provides a layer between softwareenvironments and the underlying hardware and host OS that is programmable,transparent to the software above it, and makes efficient use of the hardwarebelow it.