1. CENG334 Introduction to Operating Systems
13/03/07
CENG334 Introduction to Operating Systems
Memory Management and Virtual Memory -1
Topics:
Virtual Memory
MMU and TLB
Page fault
Demand paging
Copy-on-write
Erol Sahin
Dept of Computer Eng.
Middle East Technical University
Ankara, TURKEY

2. 13/03/07
Memory Management
Today we start a series of lectures on memory mangement
Goals of memory management
Convenient abstraction for programming
Provide isolation between different processes
Allocate scarce physical memory resources across processes
Especially important when memory is heavily contended for
Minimize overheads
Mechanisms
Virtual address translation
Paging and TLBs
Page table management
Policies
Page replacement policies
Adapted from Matt Welsh’s (Harvard University) slides.

3. 13/03/07
Virtual Memory
The basic abstraction provided by the OS for memory management
VM enables programs to execute without requiring their entire address space to be resident in physical memory
Program can run on machines with less physical RAM than it “needs”
Many programs don’t use all of their code or data
e.g., branches they never take, or variables never accessed
Observation: No need to allocate memory for it until it's used
OS should adjust amount allocated based on its run-time behavior
Virtual memory also isolates processes from each other
One process cannot access memory addresses in others
Each process has its own isolated address space
VM requires both hardware and OS support
Hardware support: memory management unit (MMU) and translation lookaside buffer (TLB)
OS support: virtual memory system to control the MMU and TLB
Adapted from Matt Welsh’s (Harvard University) slides.

4. Memory Management Requirements
13/03/07
Memory Management Requirements
Protection
Restrict which addresses processes can use, so they can't stomp on each other
Fast translation
Accessing memory must be fast, regardless of the protection scheme
(Would be a bad idea to have to call into the OS for every memory access.)
Fast context switching
Overhead of updating memory hardware on a context switch must be low
(For example, it would be a bad idea to copy all of a process's memory out to disk on every context switch.)
Adapted from Matt Welsh’s (Harvard University) slides.

5. 13/03/07
Virtual Addresses
A virtual address is a memory address that a process uses to access its own memory
The virtual address is not the same as the actual physical RAM address in which it is stored
When a process accesses a virtual address, the MMU hardware translates the virtual address into a physical address
The OS determines the mapping from virtual address to physical address
0xFFFFFFFF
(Reserved for OS)‏
Stack
Stack pointer
Address space
Heap
Uninitialized vars
(BSS segment)‏
Initialized vars
(data segment)‏
Code
(text segment)‏
Program counter 0x

6. 13/03/07
Virtual Addresses
A virtual address is a memory address that a process uses to access its own memory
The virtual address is not the same as the actual physical RAM address in which it is stored
When a process accesses a virtual address, the MMU hardware translates the virtual address into a physical address
The OS determines the mapping from virtual address to physical address
(Reserved for OS)
Stack
How does this thing work??
MMU
Heap
Uninitialized vars
(BSS segment)
Initialized vars
(data segment)
Code
(text segment)
Physical RAM

7. 13/03/07
Virtual Addresses
A virtual address is a memory address that a process uses to access its own memory
The virtual address is not the same as the actual physical RAM address in which it is stored
When a process accesses a virtual address, the MMU hardware translates the virtual address into a physical address
The OS determines the mapping from virtual address to physical address
Virtual addresses allow isolation
Virtual addresses in one process refer to different physical memory than virtual addresses in another
Exception: shared memory regions between processes (discussed later)‏
Virtual addresses allow relocation
A program does not need to know which physical addresses it will use when it's run
Compiler can generate relocatable code – code that is independent of physical location in memory
Adapted from Matt Welsh’s (Harvard University) slides.

8. MMU and TLB Memory Management Unit (MMU)
13/03/07
MMU and TLB
Memory Management Unit (MMU)
Hardware that translates a virtual address to a physical address
Each memory reference is passed through the MMU
Translate a virtual address to a physical address
Lots of ways of doing this!
Translation Lookaside Buffer (TLB)
Cache for MMU virtual-to-physical address translations
Just an optimization – but an important one!
Physical address
Memory
CPU
Translation
mapping
Virtual address
MMU
TLB
Cache of translations
Adapted from Matt Welsh’s (Harvard University) slides.

9. 13/03/07
Fixed Partitions
Original memory management technique: Break memory into fixed-size partitions
Hardware requirement: base register
Translation from virtual to physical address: simply add base register to vaddr
Advantages and disadvantages of this approach??
physical memory
partition 0 1K 3K
partition 1
base register 2K
MMU
partition 2 3K
partition 3 +
offset 4K
virtual address
partition 4 5K
partition 5
Adapted from Matt Welsh’s (Harvard University) slides.

10. Fixed Partitions Advantages: Disadvantages:
13/03/07
Fixed Partitions
Advantages:
Fast context switch – only need to update base register
Simple memory management code: Locate empty partition when running new process
Disadvantages:
Internal fragmentation
Must consume entire partition, rest of partition is “wasted”
Static partition sizes
No single size is appropriate for all programs!
Adapted from Matt Welsh’s (Harvard University) slides.

11. Variable Partitions Obvious next step: Allow variable-sized partitions
13/03/07
Variable Partitions
Obvious next step: Allow variable-sized partitions
Now requires both a base register and a limit register for performing memory access
Solves the internal fragmentation problem: size partition based on process needs
New problem: external fragmentation
As jobs run and complete, holes are left in physical memory
physical memory
partition 0
limit register
base register
P3’s size
P3’s base
partition 1
partition 2
yes
offset
<?
partition 3 +
virtual address
MMU no
partition 4
raise
protection fault
Adapted from Matt Welsh’s (Harvard University) slides.

12. Modern technique: paging
13/03/07
Modern technique: paging
Solve the external fragmentation problem by using fixed-size chunks of virtual and physical memory
Virtual memory unit called a page
Physical memory unit called a frame (or sometimes page frame)
virtual memory
(for one process)‏
physical memory
page 0
frame 0
page 1
frame 1
page 2
frame 2
page 3 …
... …
...
frame Y
page X
Adapted from Matt Welsh’s (Harvard University) slides.

13. Application Perspective
13/03/07
Application Perspective
Application believes it has a single, contiguous address space ranging from 0 to 2P – 1 bytes
Where P is the number of bits in a pointer (e.g., 32 bits)
In reality, virtual pages are scattered across physical memory
This mapping is invisible to the program, and not even under it's control!
(Reserved for OS)‏
Lots of separate processes
Stack
Heap
Uninitialized vars
(BSS segment)‏
(Reserved for OS)‏
Initialized vars
(data segment)‏
Stack
Code
(text segment)‏
MMU
Heap
(Reserved for OS)‏
Uninitialized vars
(BSS segment)‏
Initialized vars
(data segment)‏
Stack
Code
(text segment)‏
Heap
Uninitialized vars
(BSS segment)‏
Initialized vars
(data segment)‏
Code
(text segment)‏
Physical RAM

14. Virtual Address Translation
13/03/07
Virtual Address Translation
Virtual-to-physical address translation performed by MMU
Virtual address is broken into a virtual page number and an offset
Mapping from virtual page to physical frame provided by a page table
0xdeadb
0xeef
Virtual page number
Offset
0xdeadbeef =
offset
virtual address
virtual page #
0xeef
physical memory
page
frame 0
page table
0xdeadb
page
frame 1
physical address
page
frame 2
page frame #
page frame #
offset
page
frame 3
Page table entry …
...
page
frame Y

15. Page Table Entries (PTEs)
13/03/07
Page Table Entries (PTEs)
Typical PTE format (depends on CPU architecture!)
Various bits accessed by MMU on each page access:
Modify bit: Indicates whether a page is “dirty” (modified)
Reference bit: Indicates whether a page has been accessed (read or written)
Valid bit: Whether the PTE represents a real memory mapping
Protection bits: Specify if page is readable, writable, or executable
Page frame number: Physical location of page in RAM
Why is this 20 bits wide in the above example??? 1 1 1 2 20 M R V
prot
page frame number
Adapted from Matt Welsh’s (Harvard University) slides.

16. Page Table Entries (PTEs)
13/03/07
Page Table Entries (PTEs)
What are these bits useful for?
The R bit is used to decide which pages have been accessed recently.
Next lecture we will talk about swapping “old” pages out to disk.
Need some way to keep track of what counts as an “old” page.
“Valid” bit will not be set for a page that is currently swapped out!
The M bit is used to tell whether a page has been modified
Why might this be useful?
Protection bits used to prevent certain pages from being written.
How are these bits updated? 1 1 1 2 20 M R V
prot
page frame number
Adapted from Matt Welsh’s (Harvard University) slides.

17. Advantages of paging Simplifies physical memory management
13/03/07
Advantages of paging
Simplifies physical memory management
OS maintains a free list of physical page frames
To allocate a physical page, just remove an entry from this list
No external fragmentation!
Virtual pages from different processes can be interspersed in physical memory
No need to allocate pages in a contiguous fashion
Allocation of memory can be performed at a fine granularity
Only allocate physical memory to those parts of the address space that require it
Can swap unused pages out to disk when physical memory is running low
Idle programs won't use up a lot of memory (even if their address space is huge!)
Adapted from Matt Welsh’s (Harvard University) slides.

18. 13/03/07
Page Tables
Page Tables store the virtual-to-physical address mappings.
Where are they located? In memory!
OK, then. How does the MMU access them?
The MMU has a special register called the page table base pointer.
This points to the physical memory address of the top of the page table for the currently-running process.
Process A page tbl
Process B page tbl
MMU pgtbl base ptr
Physical RAM

19. The TLB Now we've introduced a high overhead for address translation
13/03/07
The TLB
Now we've introduced a high overhead for address translation
On every memory access, must have a separate access to consult the page tables!
Solution: Translation Lookaside Buffer (TLB)
Very fast (but small) cache directly on the CPU
Pentium 6 systems have separate data and instruction TLBs, 64 entries each
Caches most recent virtual to physical address translations
A TLB miss requires that the MMU actually try to do the address translation
Adapted from Matt Welsh’s (Harvard University) slides.

20. The TLB Now we've introduced a high overhead for address translation
13/03/07
The TLB
Now we've introduced a high overhead for address translation
On every memory access, must have a separate access to consult the page tables!
Solution: Translation Lookaside Buffer (TLB)‏
Very fast (but small) cache directly on the CPU
Pentium 6 systems have separate data and instruction TLBs, 64 entries each
Caches most recent virtual to physical address translations
Implemented as fully associative cache
Any address can be stored in any entry in the cache
All entries searched “in parallel” on every address translation
A TLB miss requires that the MMU actually try to do the address translation
Virtual Physical
0x49381
0x00200
Physical frame addr
Virtual page addr
0xab790
0x0025b
0xdeadb
0x002bb
0xdeadb
0x49200
0x00468
0x002bb
0xef455
0x004f8
0x978b2
0x0030f
0xef456
0x0020a
Adapted from Matt Welsh’s (Harvard University) slides.

21. Loading the TLB Two ways to load entries into the TLB.
13/03/07
Loading the TLB
Two ways to load entries into the TLB.
1) MMU does it automatically
MMU looks in TLB for an entry
If not there, MMU handles the TLB miss directly
MMU looks up virt->phys mapping in page tables and loads new entry into TLB
2) Software-managed TLB
TLB miss causes a trap to the OS
OS looks up page table entry and loads new TLB entry
Why might a software-managed TLB be a good thing?
Adapted from Matt Welsh’s (Harvard University) slides.

22. Loading the TLB Two ways to load entries into the TLB.
13/03/07
Loading the TLB
Two ways to load entries into the TLB.
1) MMU does it automatically
MMU looks in TLB for an entry
If not there, MMU handles the TLB miss directly
MMU looks up virt->phys mapping in page tables and loads new entry into TLB
2) Software-managed TLB
TLB miss causes a trap to the OS
OS looks up page table entry and loads new TLB entry
Why might a software-managed TLB be a good thing?
OS can dictate the page table format!
MMU does not directly consult or modify page tables.
Gives a lot of flexibility for OS designers to decide memory management policies
But ... requires TLB misses even for updating modified/referenced bits in PTE
OS must now handle many more TLB misses, with some performance impact.
Adapted from Matt Welsh’s (Harvard University) slides.

23. Page Table Size How big are the page tables for a process?
13/03/07
Page Table Size
How big are the page tables for a process?
Well ... we need one PTE per page.
Say we have a 32-bit address space, and the page size is 4KB
How many pages?
Adapted from Matt Welsh’s (Harvard University) slides.

24. Page Table Size How big are the page tables for a process?
13/03/07
Page Table Size
How big are the page tables for a process?
Well ... we need one PTE per page.
Say we have a 32-bit address space, and the page size is 4KB
How many pages?
2^32 == 4GB / 4KB per page == 1,048,576 (1 M pages)‏
How big is each PTE?
Depends on the CPU architecture ... on the x86, it's 4 bytes.
So, the total page table size is: 1 M pages * 4 bytes/PTE == 4 Mbytes
And that is per process
If we have 100 running processes, that's over 400 Mbytes of memory just for the page tables.
Solution: Swap the page tables out to disk!
Adapted from Matt Welsh’s (Harvard University) slides.

25. Application Perspective
13/03/07
Application Perspective
Remember our three requirements for memory management:
Isolation
One process cannot access another's pages. Why?
Process can only refer to its own virtual addresses.
O/S responsible for ensuring that each process uses disjoint physical pages
Fast Translation
Translation from virtual to physical is fast. Why?
MMU (on the CPU) translates each virtual address to a physical address.
TLB caches recent virtual->physical translations
Fast Context Switching
Context Switching is fast. Why?
Only need to swap pointer to current page tables when context switching!
(Though there is one more step ... what is it?)
Adapted from Matt Welsh’s (Harvard University) slides.

26. More Issues What happens when a page is not in memory?
13/03/07
More Issues
What happens when a page is not in memory?
How do we prevent having page tables take up a huge amount of memory themselves?
Adapted from Matt Welsh’s (Harvard University) slides.

27. Page Faults Page fault!! CPU
13/03/07
Page Faults
Page fault!!
CPU
MMU
Virtual address
Physical address
Memory
TLB
Translation
mapping
When a virtual address translation cannot be performed, it's called a page fault
Adapted from Matt Welsh’s (Harvard University) slides.

28. Page Faults Recall the PTE format:
13/03/07
Page Faults
Recall the PTE format:
Valid bit indicates whether a page translation is valid
If Valid bit is 0, then a page fault will occur
Page fault will also occur if attempt to write a read-only page (based on the Protection bits, not the valid bit)‏
This is sometimes called a protection fault M R V
prot
page frame number
Adapted from Matt Welsh’s (Harvard University) slides.

29. 13/03/07
Demand Paging
Does it make sense to read an entire program into memory at once?
No! Remember that only a small portion of a program's code may be used!
For example, if you never use the “print” capability of Powerpoint on this machine...
Virtual address space
Physical Memory
(Reserved for OS)‏
Stack
Heap
Uninitialized vars
(BSS segment)‏
Initialized vars
(data segment)‏
Code
(text segment)‏
Adapted from Matt Welsh’s (Harvard University) slides.

30. 13/03/07
Demand Paging
Does it make sense to read an entire program into memory at once?
No! Remember that only a small portion of a program's code may be used!
For example, if you never use the “print” capability of Powerpoint...
Virtual address space
Physical Memory
(Reserved for OS)‏
Where are the “holes” ??
Adapted from Matt Welsh’s (Harvard University) slides.

31. 13/03/07
Where are the “holes”?
Three kinds of “holes” in a process's page tables:
1. Pages that are on disk
Pages that were swapped out to disk to save memory
Also includes code pages in an executable file
When a page fault occurs, load the corresponding page from disk
2. Pages that have not been accessed yet
For example, newly-allocated memory
When a page fault occurs, allocate a new physical page
What are the contents of the newly-allocated page???
3. Pages that are invalid
For example, the “null page” at address 0x0
When a page fault occurs, kill the offending process
Adapted from Matt Welsh’s (Harvard University) slides.

32. 13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
Stack
Unmapped pages
Heap
Uninitialized vars
Initialized vars
Code
Code
Adapted from Matt Welsh’s (Harvard University) slides.

33. 13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
Reference next instruction
Adapted from Matt Welsh’s (Harvard University) slides.

34. 13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
Page fault!!!
Adapted from Matt Welsh’s (Harvard University) slides.

35. OS reads missing page from executable file on disk
13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
OS reads missing page from executable file on disk
Adapted from Matt Welsh’s (Harvard University) slides.

36. OS adds page to process's page table
13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
OS adds page to process's page table
Adapted from Matt Welsh’s (Harvard University) slides.

37. 13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
Process resumes at the next instruction
Adapted from Matt Welsh’s (Harvard University) slides.

38. 13/03/07
Starting up a process
What does a process's address space look like when it first starts up?
Over time, more pages are
brought in from the executable as needed
Adapted from Matt Welsh’s (Harvard University) slides.

39. Uninitialized variables and the heap
13/03/07
Uninitialized variables and the heap
Page faults bring in pages from the executable file for:
Code (text segment) pages
Initialized variables
What about uninitialized variables and the heap?
Say I have a global variable “int c” in the program ... what happens when the process first accesses it?
Adapted from Matt Welsh’s (Harvard University) slides.

40. Uninitialized variables and the heap
13/03/07
Uninitialized variables and the heap
Page faults bring in pages from the executable file for:
Code (text segment) pages
Initialized variables
What about uninitialized variables and the heap?
Say I have a global variable “int c” in the program ... what happens when the process first accesses it?
Page fault occurs
OS looks at the page and realizes it corresponds to a zero page
Allocates a new physical frame in memory and sets all bytes to zero
Why???
Maps the frame into the address space
What about the heap?
malloc() just asks the OS to map new zero pages into the address space
Page faults allocate new empty pages as above
Adapted from Matt Welsh’s (Harvard University) slides.

41. More Demand Paging Tricks
13/03/07
More Demand Paging Tricks
Paging can be used to allow processes to share memory
A significant portion of many process's address space is identical
For example, multiple copies of your shell all have the same exact code!
Physical Memory
Shell #1
(Reserved for OS)‏
(Reserved for OS)‏
Stack
Stack
Shell #2
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Heap
Heap
Code for shell
Same page
table mapping!
Uninitialized vars
Uninitialized vars
Initialized vars
Initialized vars
Code
Code
Adapted from Matt Welsh’s (Harvard University) slides.

42. More Demand Paging Tricks
13/03/07
More Demand Paging Tricks
This can be used to let different processes share memory
UNIX supports shared memory through the shmget/shmat/shmdt system calls
Allocates a region of memory that is shared across multiple processes
Some of the benefits of multiple threads per process, but the rest of the processes address space is protected
Why not just use multiple processes with shared memory regions?
Memory-mapped files
Idea: Make a file on disk look like a block of memory
Works just like faulting in pages from executable files
In fact, many OS's use the same code for both
One wrinkle: Writes to the memory region must be reflected in the file
How does this work?
When writing to the page, mark the “modified” bit in the PTE
When page is removed from memory, write back to original file
Adapted from Matt Welsh’s (Harvard University) slides.

43. Remember fork()? fork() creates an exact copy of a process
13/03/07
Remember fork()?
fork() creates an exact copy of a process
What does this imply about page tables?
When we fork a new process, does it make sense to make a copy of all of its memory?
Why or why not?
What if the child process doesn't end up touching most of the memory the parent was using?
Extreme example: What happens if a process does an exec() immediately after fork()?
Adapted from Matt Welsh’s (Harvard University) slides.

44. 13/03/07
Copy-on-write
Idea: Give the child process access to the same memory, but don't let it write to any of the pages directly!
1) Parent forks a child process
2) Child gets a copy of the parent's page tables
They point to the same physical frames!!!
Parent
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Child
Child's
page tbl
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Parent's
page tbl

45. Copy-on-write All pages (both parent and child) marked read-only
13/03/07
Copy-on-write
All pages (both parent and child) marked read-only
Why???
Parent
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Child
Child's
page tbl
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Parent's
page tbl RO RO RO RO RO RO RO RO RO RO RO RO RO RO

46. Copy-on-write What happens when the child reads the page?
13/03/07
Copy-on-write
What happens when the child reads the page?
Just accesses same memory as parent .... niiiiiice
What happens when the child writes the page?
Protection fault occurs (page is read-only!)‏
OS copies the page and maps it R/W into the child's addr space
Parent
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Child
Child's
page tbl
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Parent's
page tbl RO RO RO RO RO RO RO RO RO RO RO RO RO RO
Copy page

47. Copy-on-write What happens when the child reads the page?
13/03/07
Copy-on-write
What happens when the child reads the page?
Just accesses same memory as parent .... niiiiiice
What happens when the child writes the page?
Protection fault occurs (page is read-only!)‏
OS copies the page and maps it R/W into the child's addr space
Parent
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Child
Child's
page tbl
Stack
Heap
Initialized vars
Code
Uninitialized vars
(Reserved for OS)‏
Parent's
page tbl RO RO RO RO RO RO RO RO RW RO RW RO RO RO RO RO

48. Page Tables Remember how paging works:
13/03/07
Page Tables
Remember how paging works:
Recall that page tables for one process can be very large!
2^20 PTEs * 4 bytes per PTE = 4 Mbytes per process
offset
virtual address
virtual page #
physical memory
page
frame 0
page table
page
frame 1
physical address
page
frame 2
page frame #
page frame #
offset
page
frame 3
Page table entry …
...
page
frame Y
Adapted from Matt Welsh’s (Harvard University) slides.

49. Multilevel Page Tables
13/03/07
Multilevel Page Tables
Problem: Can't hold all of the page tables in memory
Solution: Page the page tables!
Allow portions of the page tables to be kept in memory at one time
offset
virtual address
primary page #
secondary page #
physical memory
page
frame 0
Primary page
table (1)‏
Secondary page
tables (N)‏
page
frame 1
physical address
page
frame 2
page table #
page frame #
offset
page
frame 3 …
...
page frame #
page frame #
page frame #
page
frame Y
page frame #
Adapted from Matt Welsh’s (Harvard University) slides.

50. Multilevel Page Tables
13/03/07
Multilevel Page Tables
Problem: Can't hold all of the page tables in memory
Solution: Page the page tables!
Allow portions of the page tables to be kept in memory at one time
offset
virtual address
primary page #
secondary page #
Secondary page
tables (N)‏
Primary page
table (1)‏
On disk
page table #
On disk
Adapted from Matt Welsh’s (Harvard University) slides.

51. Multilevel Page Tables
13/03/07
Multilevel Page Tables
Problem: Can't hold all of the page tables in memory
Solution: Page the page tables!
Allow portions of the page tables to be kept in memory at one time
offset
virtual address
primary page #
secondary page #
Secondary page
tables (N)‏
Primary page
table (1)‏
page table #
On disk
Adapted from Matt Welsh’s (Harvard University) slides.

52. Multilevel Page Tables
13/03/07
Multilevel Page Tables
Problem: Can't hold all of the page tables in memory
Solution: Page the page tables!
Allow portions of the page tables to be kept in memory at one time
offset
virtual address
primary page #
secondary page #
physical memory
Secondary page
tables (N)‏
page
frame 0
Primary page
table (1)‏
page
frame 1
physical address
page
frame 2
page table #
page frame #
offset
page
frame 3
On disk …
...
page
frame Y
Adapted from Matt Welsh’s (Harvard University) slides.

53. Multilevel page tables
13/03/07
Multilevel page tables
With two levels of page tables, how big is each table?
Say we allocate 10 bits to the primary page, 10 bits to the secondary page, 12 bits to the page offset
Primary page table is then 2^10 * 4 bytes per PTE == 4 KB
Secondary page table is also 4 KB
Hey ... that's exactly the size of a page on most systems ... cool
What happens on a page fault?
MMU looks up index in primary page table to get secondary page table
Assume this is “wired” to physical memory
MMU tries to access secondary page table
May result in another page fault to load the secondary table!
MMU looks up index in secondary page table to get PFN
CPU can then access physical memory address
Issues
Page translation has very high overhead
Up to three memory accesses plus two disk I/Os!!
TLB usage is clearly very important.

54. CENG334 Introduction to Operating Systems
13/03/07
CENG334 Introduction to Operating Systems
Memory Management and Virtual Memory - 2
Topics:
Page replacement
Page replacement policies
Swapping
Erol Sahin
Dept of Computer Eng.
Middle East Technical University
Ankara, TURKEY

55. 13/03/07
Page Replacement
How do we decide which pages to kick out of physical memory when memory is tight?
How do we decide how much memory to allocate to a process?
Adapted from Matt Welsh’s (Harvard University) slides.

56. Benefits of sharing pages
13/03/07
Benefits of sharing pages
How much memory savings do we get from sharing pages across identical processes?
A lot! Use the “top” command...
Adapted from Matt Welsh’s (Harvard University) slides.

57. cpu CPU usage. pid Process ID prt Number of Mach ports
cpu CPU usage. pid Process ID prt Number of Mach ports. reg Number of memory regions. rprvt Resident private address space size. rshrd Resident shared address space size. rsize Resident memory size. th Number of threads. time Execution time. uid User ID. username Username. vprvt Private address space size. vsize Total memory size.

58. 13/03/07
Paging and swapping
However, on heavily-loaded systems, memory can fill up
To achieve good system performance, must move “inactive” pages out to disk
If we didn't do this, what options would the system have if memory is full???
What constitutes an “inactive” page?
How do we choose the right set of pages to copy out to disk?
How do we decide when to move a page back into memory?
Swapping
Usually refers to moving the memory for an entire process out to disk
This effectively puts the process to sleep until OS decides to swap it back in
Paging
Refers to moving individual pages out to disk (and back)
We often use the terms “paging” and “swapping” interchangeably
Adapted from Matt Welsh’s (Harvard University) slides.

59. Page eviction When do we decide to evict a page from memory?
13/03/07
Page eviction
When do we decide to evict a page from memory?
Usually, at the same time that we are trying to allocate a new physical page
However, the OS keeps a pool of “free pages” around, even when memory is tight, so that allocating a new page can be done quickly
The process of evicting pages to disk is then performed in the background
Adapted from Matt Welsh’s (Harvard University) slides.

60. Basic Page Replacement
13/03/07
Basic Page Replacement
How do we replace pages?
Find the location of the desired page on disk
Find a free frame:
If there is a free frame, use it
If there is no free frame, use a page replacement algorithm to select a victim frame
Read the desired page into the (newly) free frame. Update the page and frame tables.
Restart the process

61. 13/03/07
Page Replacement

62. Locality Exploiting locality
13/03/07
Locality
Exploiting locality
Temporal locality: Memory accessed recently tends to be accessed again soon
Spatial locality: Memory locations near recently-accessed memory is likely to be referenced soon
Locality helps to reduce the frequency of paging
Once something is in memory, it should be used many times
This depends on many things:
The amount of locality and reference patterns in a program
The page replacement policy
The amount of physical memory and the application footprint
Adapted from Matt Welsh’s (Harvard University) slides.

63. Evicting the best page Goal of the page replacement algorithm:
13/03/07
Evicting the best page
Goal of the page replacement algorithm:
Reduce page fault rate by selecting the best page to evict
The “best” pages are those that will never be used again
However, it's impossible to know in general whether a page will be touched
If you have information on future access patterns, it is possible to prove that evicting those pages that will be used the furthest in the future will minimize the page fault rate
What is the best algorithm for deciding the order to evict pages?
Much attention has been paid to this problem.
Used to be a very hot research topic.
These days, widely considered solved (at least, solved well enough)
Adapted from Matt Welsh’s (Harvard University) slides.

64. Page Replacement Basics
13/03/07
Page Replacement Basics
Most page replacement algorithms operate on some data structure that represents physical memory:
Might consist of a bitmap, one bit per physical page
Might be more involved, e.g., a reference count for each page (more soon!!)
Free list consists of pages that are unallocated
Several ways of implementing this data structure
Scan all process PTEs that correspond to mapped pages (valid bit == 1)
Keep separate linked list of physical pages
Inverted page table: One entry per physical page, each entry points to PTE
Free list
Adapted from Matt Welsh’s (Harvard University) slides.

65. Algorithm: OPT (a.k.a MIN)
13/03/07
Algorithm: OPT (a.k.a MIN)
Evict page that won't be used for the longest time in the future
Of course, this requires that we can foresee the future...
So OPT cannot be implemented!
This algorithm has the provably optimal performance
Hence the name “OPT”
Also called “MIN” (for “minimal”)
OPT is useful as a “yardstick” to compare the performance of other (implementable) algorithms against
Adapted from Matt Welsh’s (Harvard University) slides.

66. Algorithms: Random and FIFO
13/03/07
Algorithms: Random and FIFO
Random: Throw out a random page
Obviously not the best scheme
Although very easy to implement!
FIFO: Throw out pages in the order that they were allocated
Maintain a list of allocated pages
When the length of the list grows to cover all of physical memory, pop first page off list and allocate it
Why might FIFO be good?
Why might FIFO not be so good?
Adapted from Matt Welsh’s (Harvard University) slides.

67. 13/03/07
Algorithms: FIFO
FIFO: Throw out pages in the order that they were allocated
Maintain a list of allocated pages
When the length of the list grows to cover all of physical memory, pop first page off list and allocate it
Why might FIFO be good?
Maybe the page allocated very long ago isn't being used anymore
Why might FIFO not be so good?
Doesn't consider locality of reference!
Suffers from Belady's Anomaly: Performance of an application might get worse as the size of physical memory increases!!!
Adapted from Matt Welsh’s (Harvard University) slides.

68. 13/03/07
Belady's Anomaly
time
Access pattern 1 1 2 1 2 3 2 3 3 1 1 4 1 4 1 4 1 4 2 4 2 3 4 2 3
Physical memory
(3 page frames)‏
9 page faults!
time
Access pattern 1 2 3 4
Physical memory
(4 page frames)‏
10 page faults!
Adapted from Matt Welsh’s (Harvard University) slides.

69. Algorithm: Least Recently Used (LRU)
13/03/07
Algorithm: Least Recently Used (LRU)
Evict the page that was used the longest time ago
Keep track of when pages are referenced to make a better decision
Use past behavior to predict future behavior
LRU uses past information, while MIN uses future information
When does LRU work well, and when does it not?
Implementation
Every time a page is accessed, record a timestamp of the access time
When choosing a page to evict, scan over all pages and throw out page with oldest timestamp
Problems with this implementation?
Adapted from Matt Welsh’s (Harvard University) slides.

70. Algorithm: Least Recently Used (LRU)
13/03/07
Algorithm: Least Recently Used (LRU)
Evict the page that was used the longest time ago
Keep track of when pages are referenced to make a better decision
Use past behavior to predict future behavior
LRU uses past information, while MIN uses future information
When does LRU work well, and when does it not?
Implementation
Every time a page is accessed, record a timestamp of the access time
When choosing a page to evict, scan over all pages and throw out page with oldest timestamp
Problems with this implementation?
32-bit timestamp for each page would double the size of every PTE
Scanning all of the PTEs for the lowest timestamp would be slow
Adapted from Matt Welsh’s (Harvard University) slides.

71. Approximating LRU: Additional-Reference-Bits
13/03/07
Approximating LRU: Additional-Reference-Bits
Use the PTE reference bit and a small counter per page
(Use a counter of, say, 2 or 3 bits in size, and store it in the PTE)
Periodically (say every 100 msec), scan all physical pages in the system
If the page has not been accessed (PTE reference bit == 0), increment (or shift right) the counter
If the page has been accessed (reference bit == 1), set counter to zero (or shift right)
Clear the PTE reference bit in either case!
Counter will contain the number of scans since the last reference to this page.
PTE that contains the highest counter value is the least recently used
So, evict the page with the highest counter
Adapted from Matt Welsh’s (Harvard University) slides.

72. LRU example time Accessed pages in blue 1 Increment counter
13/03/07
LRU example
time
Accessed pages
in blue 1
Increment counter
for untouched pages 1 2 1
These pages have
the highest counter
value and can be
evicted.
Adapted from Matt Welsh’s (Harvard University) slides.

73. Algorithm: LRU Second-chance (Clock)
13/03/07
Algorithm: LRU Second-chance (Clock)
LRU requires searching for the page with the highest last-ref count
Can do this with a sorted list or a second pass to look for the highest value
Simpler technique: Second-chance algorithm
“Clock hand” scans over all physical pages in the system
Clock hand loops around to beginning of memory when it gets to end
If PTE reference bit == 1, clear bit and advance hand to give it a second-chance
If PTE reference bit == 0, evict this page
No need for a counter in the PTE!
Evict!
Accessed pages
in blue
Clock hand
Adapted from Matt Welsh’s (Harvard University) slides.

74. Algorithm: LRU Second-chance (Clock)
13/03/07
Algorithm: LRU Second-chance (Clock)
LRU requires searching for the page with the highest last-ref count
Can do this with a sorted list or a second pass to look for the highest value
Simpler technique: Second-chance algorithm
“Clock hand” scans over all physical pages in the system
Clock hand loops around to beginning of memory when it gets to end
If PTE reference bit == 1, clear bit and advance hand to give it a second-chance
If PTE reference bit == 0, evict this page
No need for a counter in the PTE!
Clock hand
Evict!
Accessed pages
in blue
Adapted from Matt Welsh’s (Harvard University) slides.

75. Algorithm: LRU Second-chance (Clock)
13/03/07
Algorithm: LRU Second-chance (Clock)
This is a lot like LRU, but operates in an iterative fashion
To find a page to evict, just start scanning from current clock hand position
What happens if all pages have ref bits set to 1?
What is the minimum “age” of a page that has the ref bit set to 0?
Slight variant -- “nth chance clock”
Only evict page if hand has swept by N times
Increment per-page counter each time hand passes and ref bit is 0
Evict a page if counter >= N
Counter cleared to 0 each time page is used
Adapted from Matt Welsh’s (Harvard University) slides.

76. Algorithm: LRU Enhanced Second-chance (Clock)
13/03/07
Algorithm: LRU Enhanced Second-chance (Clock)
Be even smarter: Consider the R(eference) bit and the M(odified) bit as an ordered pair to classify pages into four classes
(0,0) : Neither recently used not modified – best page to replace
(0,1): Not recently used but modified – not quite as good, since the page has to be written out before replacement
(1,0): recently used but clean – probably will be used again
(1,1) recently used and modified – probably will be used again and the page will be need to be written out before it can be replaced
We may need to scan the circular queue several times.
The number of required I/O's reduced. to 0?

77. Swap Files What happens to the page that we choose to evict?
13/03/07
Swap Files
What happens to the page that we choose to evict?
Depends on what kind of page it is and what state it's in!
OS maintains one or more swap files or partitions on disk
Special data format for storing pages that have been swapped out
Virtual address space
Physical Memory
Swap files on disk
(Reserved for OS)‏
“Zero page”
Program
executable
Adapted from Matt Welsh’s (Harvard University) slides.

78. Swap Files How do we keep track of where things are on disk?
13/03/07
Swap Files
How do we keep track of where things are on disk?
Recall PTE format
When V bit is 0, can recycle the PFN field to remember something about the page.
But ... not all pages are swapped in from swap files!
What about executables?
Or “zero pages”?
How do we deal with these file types?
V bit
5 bits
24 bits
Swap file index
Swap file offset
Swap file (max 2^24 pages = 64 GB)‏
Swap file table
(max 32 entries)‏
Adapted from Matt Welsh’s (Harvard University) slides.

79. 13/03/07
VM map structure
OS keeps a “map” of the layout of the process address space.
This is separate from the page tables.
In fact, the VM map is used by the OS to lay out the page tables.
This map can indicate where to find pages that are not in memory
e.g., the disk file ID and the offset into the file.
(Reserved for OS)‏
Stack
Copy-on-write of zero page or
page in from swap (if page was previously modified)‏
Heap
Uninitialized vars
(BSS segment)‏
Initialized vars
(data segment)‏
Page in from executable or swap
(if page was previously modified)‏
Code
(text segment)‏
Page in from executable file
Adapted from Matt Welsh’s (Harvard University) slides.

80. Page Eviction How we evict a page depends on its type. Code page:
13/03/07
Page Eviction
How we evict a page depends on its type.
Code page:
Just remove it from memory – can recover it from the executable file on disk!
Unmodified (clean) data page:
If the page has previously been swapped to disk, just remove it from memory
Assuming that page's backing store on disk has not been overwritten
If the page has never been swapped to disk, allocate new swap space and write the page to it
Exception: unmodified zero page – no need to write out to swap at all!
Modified (dirty) data page:
If the page has previously been swapped to disk, write page out to the swap space
Adapted from Matt Welsh’s (Harvard University) slides.

81. Physical Frame Allocation
13/03/07
Physical Frame Allocation
How do we allocate physical memory across multiple processes?
What if Process A needs to evict a page from Process B?
How do we ensure fairness?
How do we avoid having one process hogging the entire memory of the system?
Local replacement algorithms
Per-process limit on the physical memory usage of each process
When a process reaches its limit, it evicts pages from itself
Global-replacement algorithms
Physical size of processes can grow and shrink over time
Allow processes to evict pages from other processes
Note that one process' paging can impact performance of entire system!
One process that does a lot of paging will induce more disk I/O
Adapted from Matt Welsh’s (Harvard University) slides.

82. 13/03/07
Working Set
A process's working set is the set of pages that it currently “needs”
Definition:
WS(P, t, w) = the set of pages that process P accessed in the time interval [t-w, t]
“w” is usually counted in terms of number of page references
A page is in WS if it was referenced in the last w page references
Working set changes over the lifetime of the process
Periods of high locality exhibit smaller working set
Periods of low locality exhibit larger working set
Basic idea: Give process enough memory for its working set
If WS is larger than physical memory allocated to process, it will tend to swap
If WS is smaller than memory allocated to process, it's wasteful
This amount of memory grows and shrinks over time
Adapted from Matt Welsh’s (Harvard University) slides.

83. Estimating the working set
13/03/07
Estimating the working set
How do we determine the working set?
Simple approach: modified clock algorithm
Sweep the clock hand at fixed time intervals
Record how many seconds since last page reference
All pages referenced in last T seconds are in the working set
Now that we know the working set, how do we allocate memory?
If working sets for all processes fit in physical memory, done!
Otherwise, reduce memory allocation of larger processes
Idea: Big processes will swap anyway, so let the small jobs run unencumbered
Very similar to shortest-job-first scheduling: give smaller processes better chance of fitting in memory
How do we decide the working set time limit T?
If T is too large, very few processes will fit in memory
If T is too small, system will spend more time swapping
Which is better?
Adapted from Matt Welsh’s (Harvard University) slides.

84. 13/03/07
Page Fault Frequency
Dynamically tune memory size of process based on # page faults
Monitor page fault rate for each process (faults per sec)‏
If page fault rate above threshold, give process more memory
Should cause process to fault less
Doesn't always work!
Recall Belady's Anomaly
If page fault rate below threshold, reduce memory allocaton
Adapted from Matt Welsh’s (Harvard University) slides.

85. 13/03/07
Thrashing
As system becomes more loaded, spends more of its time paging
Eventually, no useful work gets done!
System is overcommitted!
If the system has too little memory, the page replacement algorithm doesn't matter
Solutions?
Change scheduling priorities to “slow down” processes that are thrashing
Identify process that are hogging the system and kill them?
Is thrashing a problem on systems with only one user?
Thrashing
CPU utilization
Number of processes
Adapted from Matt Welsh’s (Harvard University) slides.