1. More Multiprogramming Issues ECEN5043 Software Engineering of Multi-program Systems University of Colorado, Boulder

2. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 2 Quote-of-the-week award “Memory is an important resource that must be carefully managed.” A. S. Tanenbaum :-))

3. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 3 Overview -- Resource Management Memory management –If certain techniques are not relevant now, they may be in the future as they were once before -- historical cycles Input/Output –Understand impact of choices –Device independence; process independence –Shared resources –Spooling File Systems Multimedia (high demand, real time) file issues

4. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 4 Growing pains In a given environment, programs get bigger faster than memories In other words, software expands to “max out” available hardware resources -- memory is one of those

5. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 5 Memory Management Concerns Which parts of a memory are in use Allocate memory to processes when needed De-allocate memory when processes do not need it Manage swapping between hierarchies of memory as needed –onboard cache –separate cache (possibly multi-layer) –main memory –disk

6. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 6 Simple is as simple does... There are simple schemes and complicated ones. Depending on the program’s environment, –the simpler ones are in use or obsolete –the more complicated ones are in use not thought of yet or not possible to apply... yet –What environments for which? –Why?

7. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 7 Monoprogramming Run one program at a time Share memory between the program and the operating system (may be primitive) First used on mainframes, then minis, then PCs, then palmtops and simple embedded systems... What conditions make it appropriate? User program Op. Sys. in RAM Op. Sys. in ROM User program Device drivers in ROM User program Op. Sys. in RAM

8. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 8 Multiprogramming Multiple processes ready to run simultaneously means –When one process is blocked waiting for I/O to finish, another one can use the CPU –Increased CPU utilization Expectations of this capability are increasing –Network servers -- expected to run (serve) multiple processes –Client machines also have this ability

9. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 9 Some ways to achieve multiprogramming Easiest: divide memory into n partitions, not necessarily equal –Each process has a queue Put a new process into input queue for smallest partition large enough to hold it (problem? remedy?) –One queue fits all Scan jobs in an order-of-arrival queue; select first fit for available partition (problem? remedy?) Scan all; select best fit (problem? remedy?)

10. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 10 Modeling Multiprogramming Probabilistic viewpoint –Process spends fraction p of its time waiting for I/O to complete –n processes in memory at once -- probability that all n processes are waiting for I/O is p n –Prob. that CPU is being used: CPU utilization = 1 - p n Interactive systems, batch systems with disk I/O, (others?) can have 80% wait time Assumptions don’t quite fit -- makes this an approximation, not an accurate calculation

11. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 11 Analysis of Multiprogramming System Performance Analysis examples are in the online handout in 4.1.4, p. 194 You have a homework problem, too. If you designed a system of programs and those programs will be the only ones running in a certain environment... –What information do you need to know to be able to do an analysis of performance? –How can you get that information?

12. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 12 Multiprogramming Introduces 2 Problems Protection -- the other problem A malicious program can construct a new instruction and jump to it –If programs use absolute addresses rather than relative addresses, difficult to stop it Protection-code solution Base/limit approach solved both the relocation and protection problems –base has address of partition start; limit has length –disadvantage?

13. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 13 When is “simple” not enough? Simplest memory management to implement –fixed partitions –process/job loaded into partition when it gets to the head of the queue –stays in memory until done What questions do you want to know the answers to re user-threads or user-spawned copies of processes? Why do anything different?

14. Whenever you open a can of worms, it always requires a bigger can to put them back. from Murphy’s Law and Other Reasons Why Things Go Wrong wrong

15. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 15 Pandora’s box syndrome... Swapping –Bring in a process in its entirety, run it, put back on disk if it has changed –Fixed partitions -- simplest –variable partitions -- much more complicated (see next slide) Virtual memory –Bring in a portion of the process that includes what is needed –Once it is understood that only a portion needs to be in main memory, multiple portions can be present without being contiguous

16. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 16 Swap Meet -- Variable Partitions In (g), A’s addresses must be relocated by software when it is swapped in or, more likely, by hardware during execution.

17. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 17 Swapping with variable partitions –Number, location, and size of partitions vary dynamically –Improved memory utilization because not tied to fixed partitions that waste space or are too small –Complicates allocating and deallocating memory - why? –Complicates keeping track of available space - why?

18. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 18 Managing a Growing Process IF processes have fixed sizes, allocate what is needed. IF data segment can grow –Process can grow into adjacent hole, if any –Process can be swapped to new location large enough –One or more other processes can be swapped out to create a large enough hole –Process can wait –Process can be killed

19. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 19 How much is enough? If most processes will grow while running, design o.s. so that it –Allocates space in main memory a little more than is apparently needed –When swapping out, swap only memory actually in use OR If supported, allow stack and heap to grow toward each other where the available memory in between is used by whichever needs it.

20. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 20 Managed by the operating system if there is one You may be designing the technique if your company’s product is a very small computer (palm) or embedded system with no o.s. or a limited one. Bitmap Linked list -- first, next, best, worst, quick fit Dynamic memory assignment

21. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 21 Linked Lists... “Tables” Linked list of allocated and free memory segments containing either processes or holes First fit -- take first hole big enough, even if tooooo big –allocate the part that is needed; leaving new smaller hole Next fit -- Keeps track of location when it finds a suitable hole -- searches from that point next time. Best fit -- Searches all of list; takes the closest that is larger Worst fit -- leaves the biggest hole possible Quick fit -- separate lists of commonly requested sizes -- fast to find -- slow to merge remaining holes

22. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 22 Overlays Split the program into pieces called overlays; scheme for how this was done was designed by the programmer When one overlay done, it called another (or other schemes) Overlays kept on disk, swapped in and out of memory as needed Eventually, operating systems took over the whole job with a technique called virtual memory In the absence of an operating system, it is still a viable concept in specialized environments

23. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 23 Virtual Memory O.S. (or equivalent) keeps –parts of the program currently in use in main memory, not necessarily contiguously –parts of the program not currently in use (or blocked) may be on disk if the space is needed –may do this simultaneously for several programs Paging, segmentation, paged-segmentation, etc.

24. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 24 Simple Paging page # page offset virtual address page table page table pointer page frame number physical address main memory... frame n frame 1 frame 2 present modified page table index

25. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 25 Managing Page Tables Page tables usually kept in memory -- they’re big Execution speed is usually limited by the rate the CPU can get instructions and data out of memory For each memory address reference, there is (theoretically) a page table memory reference plus the resulting memory reference Locality of reference -- a small number of page table entries are heavily used; others rarely Translation Lookaside Buffer is inside MMU with the frequently referenced page table entries’ information

26. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 26 Software TLB Manager RISC machines (Reduced Instruction Set Computer) –Small number of simple instructions –Programs are longer in number of lines but usually execute faster on simpler architectures –Burden is on the programmer (compiler or assembly language writer) to express the thought with a more restricted set of instructions Tend to handle the TLB faults in the o.s. (software) instead of the MMU (hardware)

27. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 27 Inverted Page Tables When the virtual address space is too huge, the page table is too huge -- one entry for each page. Instead, use a table with an entry for each physical frame –Keeps track of which process and virtual page is located in the page frame Plus: save lots of space when the virtual address space is much larger than the physical address space Minus: Virtual-to-physical translation becomes much harder –Use TLB for the inverted page table

28. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 28 Other techniques for dealing with large virtual memories – pdf excerpt from book gives references Page replacement algorithms –When a page fault occurs, which page should be removed to make room for the needed one? Page Replacement - a kind of prioritization –LRU is a good approximation to the optimal algorithm Hard to implement in software NFU -- not frequently used -- is a software way to simulate LRU –NRU -- not recently used; FIFO; clock page; LRU

29. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 29 Deja Vu all over again Working set –notion regarding pages in interacting but distinct programs –If you can influence this -- improve performance? –Prefer an operating system that keeps track of which pages are in the working set. –When page fault occurs, evict a page not in the working set

30. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 30 Design Issues for Paging Systems We’ll come back to this slide in the week that we look at Performance patterns and anti-patterns. Reminder: –Fast path“god” class –First Things FirstExcessive dynamic alloc –CouplingCircuitous Treasure Hunt –BatchingOne-Lane Bridge –Alternate RoutesTraffic Jam –Flex Time –Slender Cyclic Functions

31. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 31 pdf handout from Tanenbaum Chapter 5 - Input/Output

32. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 32 Principles of I/O Software Device independence –“... should be possible to write programs that can access any I/O device without having to specify the device in advance” –“A program that reads a file as input should be able to read a file on a floppy disk, on a hard disk, or on a CD-ROM, without having to modify the program for each different device.” –sort output should work with input coming from various types of disks or the keyboard and the output going to any disk or the screen

33. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 33 What’s that got to do with multi-program systems? It is up to the o.s. to take care of the problems caused by the fact that these devices really are different and require different command sequences to read or write. You may be responsible for that in the absence of an o.s. or with an inadequate one for the embedded system But In multi-program communication, the data flow can be between programs and many of the same techniques can be used.

34. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 34 buffered synchronous vs. asynchronous dedicated vs. shared devices other Program- med I/O Interrupt- driven I/O I/O using DMA Fill in for device I/O managed by CPU - 5.2

35. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 35 buffered synchronous vs. asynchronous dedicated vs. shared devices other Program- med I/O = ? Interrupt- driven I/O = ? I/O using DMA = ? Fill in analogous concept for inter-program communication

36. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 36 I/O Software System Layers User-level I/O software Device-independent operating system software Device drivers Interrupt handlers Hardware

37. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 37 How might a program communicate with another? Driver starting an I/O operation blocks until the I/O has completed and the interrupt occurs. –driver blocks itself by setting a semaphore, a wait on a condition variable, a receive on a message or something similar When the interrupt happens, the procedure –does whatever it has to do in order to handle the interrupt –then unblocks the driver (reset the semaphore, execute an interrupt return to the previous process)

38. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 38 Role for “device” driver An exercise in layered software architecture Standard interface procedures that the rest of the o.s. can call Functions: –accept abstract read and write requests from the device- independent software and see that they are carried out –initialize the device –manage power requirements and log events –check input parameters to see if valid (Meyer: Filter-methods in Design by Contract programming) –may need to translate parameters from abstract to concrete terms –check if device is currently in use; if so, queue the request –list continued on next slide...

39. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 39 continued –timing: check if request can be handled now or if something else must be done like start the device –control the device -- issue sequence of commands, may need to interact sequentially –driver blocks itself until interrupt indicates completion or it doesn’t need to –check for errors –may pass data –returns some status information for error reporting –Select and start another request or blocks, waiting for next request must be reentrant cannot make system calls; may need to make certain kernel calls

40. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 40 Think of the device driver as a shared resource.. Look in Modern Operating Systems, Chapter 2 on Processes and Threads, Section 2.3 on Interprocess Communication Which of these forms of communication does a program typically use with a device driver? Why are some techniques in Section 2.3 appropriate for program-program communication but not appropriate (or not necessary) for program-device-driver communication? Analyze the interprocess comm techniques with respect to their suitability to accomplish “device driver” functions in program-program communication.

41. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 41 User Space I/O Software -- (parallels to inter-program communication?) Library procedures Spooling –A way of dealing with dedicated I/O devices in a multiprogramming system –special process, daemon –special directory, spooling directory –process generates entire file to be output –process puts it in the spooling directory –the daemon extracts files from that directory and uses them as data

42. May 5, 2002ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder 42 Can spooling be useful in multi-program interactions?