1. Outline for Today’s Lecture Administrative: –Enjoy Fall Break Objective for today: –Replacement policies

2. Policies for Paged Virtual Memory The OS tries to minimize page fault costs incurred by all processes, balancing fairness, system throughput, etc. (1) fetch policy: When are pages brought into memory? prepaging: reduce page faults by bring pages in before needed on demand: in direct response to a page fault. (2) replacement policy: How and when does the system select victim pages to be evicted/discarded from memory? (3) placement policy: Where are incoming pages placed? Which frame? (4) backing storage policy: Where does the system store evicted pages? When is the backing storage allocated? When does the system write modified pages to backing store? Clustering: reduce seeks on backing storage

3. Paged Virtual Memory General notion of a cache: –copies of data temporarily moved into storage of faster, higher cost, lower capacity technology –Achieving a high hit ratio gives performance equivalent to having most of memory built using this $$ technology The paging system manages physical memory as a cache over a larger virtual address space. –Data “lives” on disk, and a copy is in physical memory only while in active use. –Hardware and OS software cooperate to maintain the illusion that the machine’s memory “looks like” the virtual memory. –The OS controls data placement/movement and establishes the set of translations in effect at any time. –The VM abstraction is (mostly) transparent to user code.

4. Demand Paging Missing pages are loaded from disk into memory at time of reference (on demand). The alternative would be to prefetch into memory in anticipation of future accesses (need good predictions). Page fault occurs because valid bit in page table entry (PTE) is off. The OS: –allocates an empty frame * –reads the page from disk –updates the PTE when I/O is complete –restarts faulting process

5. *Page Replacement When there are no free frames available, the OS must replace a page (victim), removing it from memory to reside only on disk (backing store), writing the contents back if they have been modified since fetched (dirty). Replacement algorithm - goal to choose the best victim, with the metric for “best” (usually) being to reduce the fault rate.

6. Terminology Each thread/process/job utters a stream of page references. –Model execution as a page reference string: e.g., “abcabcdabce..” The OS tries to minimize the number of faults incurred. –The set of pages (the working set) actively used by each job changes relatively slowly. –Try to arrange for the resident set of pages for each active job to closely approximate its working set. Replacement policy is the key. –Determines the resident subset of pages..

7. A Example (Artificially Small Pagesize) #define dim int A[dim] [dim], B[dim] [dim]; int main() { int i, j, k; for (i= 0; i<dim; i++) for (j=0; j<dim; j++) B[i][j] = A[i][j]; exit; }. sw $0,16($fp) $L10: lw $2,16($fp) slt $3,$2,dim bne $3,$0,$L13 j $L11 $L13: sw $0,20($fp). B i j

8. Example Reference String …0 { 1 2 { 2 3 4 5 15 5 20 5 6} m 2 6 { 2 3 4 5 16 5 21 5 6} m 2 6 { 2 3 4 5 17 5 22 5 6} m 2 6 … { 2 3 4 5 19 5 24 5 6} m 2 6 7 } n A B i j

9. Assessing Replacement Algs Model program execution as a reference string Metric of algorithm performance is fault rate Comparison to base line of Optimal Algorithm. For a specific algorithm: What is the information needed? How is that information gathered? When is it acted upon? At each memory reference At fault time At periodic intervals “What did he know and when did he know it?” (history lesson - 1972-74)

10. Managing the VM Page Cache 1. Pages are typically referenced by page table entries. Must invalidate before reusing the frame. 2. Reads and writes are implicit; the TLB hides them from the OS. How can we tell if a page is dirty? How can we tell if a page is referenced? 3. Cache manager must run policies periodically, sampling page state. Continuously push dirty pages to disk to “launder” them. Continuously check references to judge how “hot” each page is. Balance accuracy with sampling overhead.

11. Replacement Algorithms Assume fixed number of frames in memory assigned to this process: Optimal - baseline for comparison - future references known in advance - replace page used furthest in future. FIFO Least Recently Used (LRU) stack algorithm - don’t do worse with more memory. LRU approximations for implementation Clock, Aging register

12. Example: Optimal …0 { 1 2 { 2 3 4 5 15 5 20 5 6} m 2 6 { 2 3 4 5 16 5 21 5 6} m 2 6 { 2 3 4 5 17 5 22 5 6} m 2 6 … { 2 3 4 5 19 5 24 5 6} m 2 6 7 } n A B i j VAS 0 1 2 3 4 5 1520 6 3 4 15 202

13. FIFO No extra hardware assistance needed, No per-reference overhead (we have no information about actual access pattern) At fault time: maintain a first-in first-out queue of pages resident in physical memory Replace oldest resident page Why it might make sense - straight-line code, sequential scans of data Belady’s anomaly - fault rate can increase with more memory

14. Example Reference String …0 { 1 2 { 2 3 4 5 15 5 20 5 6} m 2 6 { 2 3 4 5 16 5 21 5 6} m 2 6 { 2 3 4 5 17 5 22 5 6} m 2 6 … { 2 3 4 5 19 5 24 5 6} m 2 6 7 } n A B i j 012 34 5 1520 6 2

15. LRU At fault time: replace the resident page that was last used the longest time ago Idea is to track the program’s temporal locality To implement exactly: we need to order the pages by time of most recent reference (per-reference information needed  HW, too $$) –timestamp pages at each ref, stack operations at each ref Stack algorithm - doesn’t suffer from Belady’s anomaly -- if i > j then set of pages with j frames is a subset of set of pages with i frames.

16. …0 { 1 2 { 2 3 4 5 15 5 20 5 6} m 2 6 { 2 3 4 5 16 5 21 5 6} m 2 6 { 2 3 4 5 17 5 22 5 6} m 2 6 … { 2 3 4 5 19 5 24 5 6} m 2 6 7 } n 5 Example Reference String 0 1 2 0 1 2 3 3 4 4 515 20 15 520 5 6 6

17. LRU Approximations for Paging Pure LRU and LFU are prohibitively expensive to implement. –most references are hidden by the TLB –OS typically sees less than 10% of all references –can’t tweak your ordered page list on every reference Most systems rely on an approximation to LRU for paging. –Implementable at reasonable cost –periodically sample the reference bit on each page visit page and set reference bit to zero run the process for a while (the reference window) come back and check the bit again –reorder the list of eviction candidates based on sampling

18. Approximations to LRU HW support: usage/reference bit in every PTE - set on each reference. –Set in TLB entry –Written back to PTE on TLB replacement We now know whether or not a page has been used since the last time the bits were cleared. –Algorithms differ on when & how that’s done. –Unless it’s on every reference, there will be “ties”

19. A Page Table Entry (PTE) PFN valid bit: OS sets this to tell MMU that the translation is valid. write-enable: OS touches this to enable or disable write access for this mapping. reference bit: set when a reference is made through the mapping. dirty bit: set when a store is completed to the page (page is modified).

20. Clock Algorithm Maintain a circular queue with a pointer to the next candidate (clock hand). At fault time: scan around the clock, looking for page with usage bit of zero (that’s your victim), clearing usage bits as they are passed. We now know whether or not a page has been used since the last time the bits were cleared Newest 1st Candidate

21. …0 { 1 2 { 2 3 4 5 15 5 20 5 6} m 2 6 { 2 3 4 5 16 5 21 5 6} m 2 6 { 2 3 4 5 17 5 22 5 6} m 2 6 … Example Reference String 0 21 0* 1* 2* 1 2 3* 3 4* 4 03*

22. Approximating a Timestamp Maintain a supplemental data structure (a counter) for each page Periodically (on a regular timer interrupt) gather info from the usage bits and zero them. for each page i {if (used i ) counter i = 0; else counter i ++; used i = 0;} At fault time, replace page with largest counter value (time intervals since last use)

23. Practical Considerations Dirty bit - modified pages require a writeback to secondary storage before frame is free to use again. Variation on Clock tries to maintain a healthy pool of clean, free frames –on timer interrupt, scan for unused pages, move to free pool, initiate writeback on dirty pages –at fault time, if page is still in frame in pool, reclaim; else take free, clean frame.

24. A Page Table Entry (PTE) PFN valid bit: OS sets this to tell MMU that the translation is valid. write-enable: OS touches this to enable or disable write access for this mapping. reference bit: set when a reference is made through the mapping. dirty bit: set when a store is completed to the page (page is modified).

25. The Paging Daemon Most OS have one or more system processes responsible for implementing the VM page cache replacement policy. –A daemon is an autonomous system process that periodically performs some housekeeping task. The paging daemon prepares for page eviction before the need arises. –Wake up when free memory becomes low. –Clean dirty pages by pushing to backing store. prewrite or pageout –Maintain ordered lists of eviction candidates. –Decide how much memory to allocate to UBC, VM, etc.

26. FIFO with Second Chance (Mach) Idea: do simple FIFO replacement, but give pages a “second chance” to prove their value before they are replaced. –Every frame is on one of three FIFO lists: active, inactive and free –Page fault handler installs new pages on tail of active list. –“Old” pages are moved to the tail of the inactive list. Paging daemon moves pages from head of active list to tail of inactive list when demands for free frames is high. Clear the refbit and protect the inactive page to “monitor” it. –Pages on the inactive list get a “second chance”. If referenced while inactive, reactivate to the tail of active list.

27. Illustrating FIFO-2C active list inactive list free list Consume frames from the head of the free list. If free shrinks below threshold, kick the paging daemon to start a scan. 2. Page at head of inactive list has not been referenced? page_protect and place on tail of free list. 3. Dirty page on inactive list? Push to disk and return to inactive list tail. Restock inactive list by pulling pages from the head of the active list: knock off the reference bit and inactivate. Inactive list scan: 1. Page on inactive list has been referenced? Return to tail of active list (reactivation). Paging daemon scans a few times per second, even if not needed to restock free list.

28. MMU Games Vanilla Demand Paging –Valid bit in PTE means non-resident page. Resulting page fault causes OS to initiate page transfer from disk. –Protection bits in PTE means page should not be accessed in that mode (usually means non-writable) What else can you do with them?

29. A Page Table Entry (PTE) PFN valid bit: OS sets this to tell MMU that the translation is valid. write-enable: OS touches this to enable or disable write access for this mapping. reference bit: set when a reference is made through the mapping. dirty bit: set when a store is completed to the page (page is modified). Useful in forcing an exception! Allows OS to regain control.

30. Simulating Usage Bits Turn off both valid bit and write-protect bit On first reference - fault allows recording the reference bit information by OS in an auxiliary data structure. Set it valid for subsequent accesses to go through HW. On first write attempt - protection fault allows recording the dirty bit information by OS in aux. data structure. PFN valid bit: OS sets this to tell MMU that the translation is valid. write-enable: OS touches this to enable or disable write access for this mapping.

31. Copy-on-Write Operating systems spend a lot of their time copying data. –particularly Unix operating systems, e.g., fork() –cross-address space copies are common and expensive Idea: defer big copy operations as long as possible, and hope they can be avoided completed. –create a new shadow object backed by an existing object –shared pages are mapped read-only in participating spaces read faults are satisfied from the original object (typically) write faults trap to the kernel –make a (real) copy of the faulted page –install it in the shadow object with writes enabled

32. Variable / Global Algorithms Not requiring each process to live within a fixed number of frames, replacing only its own pages. Can apply previously mentioned algorithms globally to victimize any process’s pages Algorithms that make number of frames explicit. size of locality set time transitions stable

33. Variable Space Algorithms Working Set Tries to capture what the set of active pages currently is. The whole working set should be resident in memory for the process to bother running. WS is set of pages referenced during window of time (now-t, now). –Working Set Clock - a hybrid approximation Page Fault Frequency Monitor fault rate, if pff > high threshold, grow # frames allocated to this process, if pff < low threshold, reduce # frames. Idea is to determine the right amount of memory to allocate.

34. Working Set Model Working set at time t is the set of pages referenced in the interval of time (t-w, t) where w is the working set window. –Implies per-reference information captured. –How to choose w? Identifies the “active” pages. Any page that is resident in memory but not in any process’s working set is a candidate for replacement. Size of the working set can vary with locality changes

35. Example Reference String A B i j …0 { 1 2 { 2 3 4 5 15 5 20 5 6} m 2 6 { 2 3 4 5 16 5 21 5 6} m 2 6 { 2 3 4 5 17 5 22 5 6} m 2 6 … { 2 3 4 5 19 5 24 5 6} m 2 6 7 } n

36. Clock Algorithm Maintain a circular queue with a pointer to the next candidate (clock hand). At fault time: scan around the clock, looking for page with usage bit of zero (that’s your victim), clearing usage bits as they are passed. We now know whether or not a page has been used since the last time the bits were cleared Newest 1st Candidate

37. Working Set Model Working set at time t is the set of pages referenced in the interval of time (t-w, t) where w is the working set window. –Implies per-reference information captured. –How to choose w? Identifies the “active” pages. Any page that is resident in memory but not in any process’s working set is a candidate for replacement. Size of the working set can vary with locality changes

38. WSClock The implementable approximation At fault time: scan usage bits of resident pages. For each page i {if (used i ) {time_of_ref i = vt owner[i] /*virtual time of owning process*/; used i = 0;} else if ( | vt owner[i]   time_of_ref i | >= w ) replaceable; //else still in “working set”}

39. WSClock {a,b,c} {x,y} {a,f} {y,z} vt=3 vt=7vt=3vt=6 rt=3rt=6rt=10rt=13rt=0 W=2

40. WSClock (the picture) VT 0 = 8 VT 1 = 6 P1P1 P0P0 x x xx xx ref = 2ref = 5ref = 8 ref = 3 ref = 6 Let w = 5 Then this is no longer in working set of P 0

41. Thrashing Page faulting is dominating performance Causes: –Memory is overcommited - not enough to hold locality sets of all processes at this level of multiprogramming –Lousy locality of their programs –Positive feedback loops reacting to paging I/O rates # frames page fault freq. 1 N Load control is important (how many slices in frame pie?) –LT/RT strategy (limit # processes in load phase) enough frames too few frames difference between repl. algs

42. Pros and Cons of VM Demand paging gives the OS flexibility to manage memory... –programs may run with pages missing unused or “cold” pages do not consume real memory improves degree of multiprogramming –program size is not limited by physical memory program size may grow (e.g., stack and heap) …but VM takes control away from the application. –With traditional interfaces, the application cannot tell how much memory it has or how much a given reference costs. –Fetching pages on demand may force the application to incur I/O stalls for many of its references.

43. Policies for Paged Virtual Memory The OS tries to minimize page fault costs incurred by all processes, balancing fairness, system throughput, etc. (1) fetch policy: When are pages brought into memory? prepaging: reduce page faults by bring pages in before needed on demand: in direct response to a page fault. (2) replacement policy: How and when does the system select victim pages to be evicted/discarded from memory? (3) placement policy: Where are incoming pages placed? Which frame? (4) backing storage policy: Where does the system store evicted pages? When is the backing storage allocated? When does the system write modified pages to backing store? Clustering: reduce seeks on backing storage

44. Placement Policy Which free frame to chose? Are all frames in physical memory created equal? Yes, only considering size. Fixed size. No, if considering –Cache performance, conflict misses –Access to multi-bank memories –Multiprocessors with distributed memories

45. Power Management (RAMBUS memory) active Powered down standby nap Read or write 300mW 180 mW 30mW 3mW 2 chips +60ns +6ns +6 l s Again, the v.a.  p.a mapping could exploit page coloring ideas 60ns

46. Power Management (RAMBUS memory) active Powered down standby nap Read or write 300mW 180 mW 30mW 3mW 2 chips +60ns +6ns +6 l s Again, the v.a.  p.a mapping could exploit page coloring ideas 60ns

47. Power Management (RAMBUS memory) active Powered down standby nap Read or write 300mW 180 mW 30mW 3mW 2 chips +60ns +6ns +6 l s Again, the v.a.  p.a mapping could exploit page coloring ideas 60ns

48. CPU/$ OS Page Mapping Allocation Chip 1 Chip 2 Chip n Power Down Standby Active ctrl Hardware control Software control Two Dimensions to Control Energy

49. Dual-state (Static) HW Power State Policies All chips in one base state Individual chip Active while pending requests Return to base power state if no pending access access No pending access Standby/Nap/Powerdown Active access Time Base Active Access

50. Quad-state (Dynamic) HW Policies Downgrade state if no access for threshold time Independent transitions based on access pattern to each chip Competitive Analysis –rent-to-buy –Active to nap 100’s of ns –Nap to PDN 10,000 ns no access for Ts-n no access for Ta-s no access for Tn-p access Active STBY NapPDN Time PDN Active STBY Nap Access

51. Page Allocation Polices Random Allocation –Pages spread across chips Sequential First-Touch Allocation –Consolidate pages into minimal number of chips –One shot Frequency-based Allocation –First-touch not always best –Allow movement after first-touch

52. Dual-state + Random Allocation*  Active to perform access, return to base state  Nap is best ~85% reduction in E*D over full power  Little change in run-time, most gains in energy/power 2 state model * NT Traces

53. Quad-state HW + Sequential Allocation* Base: Dual-state Nap Sequential Allocation Thresholds: 0ns A->S; 750ns S->N; 375,000 N->P Quad-state + Sequential 30% to 55% additional improvement over dual-state nap sequential Sophisticated HW not enough 1 * SPEC

54. The Design Space Quad-state Hardware (dynamic) Dual-state Hardware (static) Random Allocation Sequential Allocation Nap is best dual-state policy 60%-85% Additional 10% to 30% over Nap Thresholds not obvious, Could be equal to dual-state Best Approach: 6% to 55% over dual-nap-seq, 99% to 80% over all active.

55. Another problem: Sharing The indirection of the mapping mechanism in paging makes it tempting to consider sharing code or data - having page tables of two processes map to the same page, but… Interaction with caches, especially if virtually addressed cache. What if there are addresses embedded inside the page to be shared?

56. Paging and Sharing (difficulties with embedded addresses) Virtual address spaces still look contiguous. Virtual Address Space for Process 0 links foo into pink address region Virtual Address Space for Process 1 links bar into blue region Then along comes Process 2... VAS 0 VAS 1 VAS 2 proc foo( ) VAS 2 wants to share both foo and bar. proc foo( ) BR 42 42:proc bar( )

57. Segmentation A better basis for sharing. Naming is by logical unit (rather than arbitrary fixed size unit) and then offset within unit (e.g. procedure). Segments are variable size Segment table is like a bunch of base/limit registers. seg#offset virtual addr effective addr segment table + base

58. Sharing in Segmentation Naming is by logical objects. Offsets are relative to base of object. Address spaces may be sparse as well as being non-contiguous. proc foo( ) BR foo, 30 BR bar, 2 proc bar( ) segment table VAS 0 VAS 1 VAS 2 segment table

59. Combining Segmentation and Paging Sharing supported by segmentation. Programs name shared segments. Physical storage management simplified by paging of each segment. pageoffset virtual addr frameoffset effective addr page table segment table yet another page table for diff seg

60. Motivation: How to exploit remote memory? P $ Memory “I/O bottleneck” VM and file caching Remote Memory Low latency networks make retrieving data across the network faster than local disk

61. Distributed Shared Memory (DSM) Allows use of a shared memory programming model (shared address space) in a distributed system (processors with only local memory) network proc msg mmu mem

62. DSM Issues Can use the local memory management hardware to generate fault when desired page is not locally present or when write attempted on read-only copy. Locate the page remotely - current “owner” of page (last writer) or “home” for page. Page sent in message to requesting node (read access makes copy; write migrates) Consistency protocol - invalidations or broadcast of changes (update) –directory kept of “caches” holding copies

63. DSM States Forced faults are key to consistency operations Invalid local mapping, attempted read access - data flushed from most recent writer, set write-protect bit for all copies. Invalid local mapping, attempted write access - migrate data, invalidate all other copies. Local read-only copy, write-fault - invalidate all other copies

64. Consistency Models Sequential consistency –All memory operations appear to execute one at a time. A write is considered done only when invalidations or updates have propagated to all copies. Weaker forms of consistency –Guarantees associated with synchronization primitives; at other times, it doesn’t matter –For example: acquire lock - make sure others’ writes are done release lock - make sure all my writes are seen by others

65. Example - the Problem A = 0; A = 1; if (B  0) succ[0] = true; B = 0; B = 1; if (A  0) succ[1] = true; time (A  1 & B  1)

66. Example - Sequential Consistency A = 0; A = 1; if (B  0) succ[0] = true; B = 0; B = 1; if (A  0) succ[1] = true; time B = 1 delayed A = 1 delayed

67. Example - Weaker Consistency A = 0; A = 1; acquire (mutex0); if (B  0) succ[0] = true; release (mutex0); B = 0; B = 1; acquire (mutex1); if (A  0) succ[1] = true; release (mutex1);

68. False Sharing Page bouncing sets in when there are too frequent coherency operations. One cause: false sharing Half used by one proc. Other half by another Subpage management? Packaged & managed as one page