1. Memory Management and Virtual Memory
Fred Kuhns
Applied Research Laboratory
Department of Computer Science and Engineering
Washington University in St. Louis

2. Memory Management Central Component of any operating system
Hierarchical layering
registers
cache
primary (main) memory
secondary (backing store, local disk) memory
file servers (networked storage)
Policies related to memory requirements of processes (i.e. all or part resident)
goals of admitting new processes (long term), memory allocation (medium term) and processor scheduling (short term) must be considered together.
Common goal is to optimize the number of runnable process resident in memory
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

3. UNIX Memory Management
UNIX uses a demand paged virtual memory architecture
anticipatory paging is where the OS proactively pages
Memory is managed at the page level
page frame or simply frame == physical page
virtual page or simply page
Basic memory allocation responsibility of the page-level allocator
has two principle clients: paging system and kernel memory allocator
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

4. The Virtual Address Space
kernel memory
proc struct
Data
Stack
Text (shared)
kernel stack/u area
Data
Stack
Text (shared)
kernel stack/u area
Data
Stack
Text (shared)
kernel stack/u area
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

5. Process Address Space (one approach)
0xffffffff
Kernel stack
Kernel address space
0x7fffffff
Data
stack
Text (shared)
Process address space 0x
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

6. Virtual Memory Goals Run applications larger than physical memory.
Run partially loaded programs.
Multiprogramming: > one program simultaneously reside in memory.
Allow relocatable programs – anywhere, anytime
Application Portability:
Applications should not have to manage memory resources
Write machine independent code – program should not depend on memory architecture.
Permit sharing of memory segments or regions. For example, read-only code segments should be shared between program instances.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

7. Virtual Memory Costs
Space: Translation tables and other data used by VM system reduce available memory to programs
Time: Address translation time is added to the cost (execution time) of each instruction.
Async: Also page fault handling may result in page I/O operations increasing latency and possibly affecting unrelated processes.
Overhead: Memory management operations have been measured to consume up to 10% of the CPU time on a busy system.
Efficiency: Allocating memory in pages may result in fragmentation
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

8. Processes and Memory
Process runs on a virtual machine as defined by the underlying hardware.
Focus is on Hardware support for a virtual address space
virtual addresses independent of physical memory
Key hardware component is the Memory Management Unit (MMU)
address translation: virtual to physical memory
simplifies context switching
ensures virtual address space protection
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

9. Memory allocation physical page Page-level allocator Kernel memory
Paging
system
Network
buffers
Data
structures
temp
storage
process
Buffer cache
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

10. Page-level Allocation
Kernel maintains a list of free page frames (physical memory)
Since kernel and user space programs use virtual memory addresses, the physical location of a page is not important
Pages are allocated from the free list
Two principal clients:
paging system
kernel memory allocator
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

11. The Paging System
Responsible for allocating and managing the address space of processes
Primary goal is to allow processes to run in a virtual address space and to perform address translations transparently
In demand-paged system the page is the basic unit of memory allocation, protection and address translation. Virtual address is converted to physical page (frame) number and offset.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

12. The Paging System Requirements: Address space management
Address Translation – translation maps used by MMU, may result in a page fault exception
Physical memory management – physical memory used as a cache for useful data.
Memory Protection – HW support exploited
Memory Sharing
Monitoring system load
Other facilities – for example memory mapped files or shared libraries
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

13. Paged Virtual Memory Working set Physical address space P1 virtual
pagen
pagen
pagen
resident
Address
Translation
Page
frames
Non-resident
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

14. The Virtual Address Space
Address space along with processes register context reflects the current state
exec causes kernel to build new process image:
memory “regions”: text, initialized data, uninitialized data, modified data, stack, heap, shared memory and shared libraries.
These regions may differ in protection, initialization and sharing. Protections usually set at page level when allocated.
Process may start running before any of its pages are resident in memory.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

15. Initial Access to Pages
Text and initialized data are read in from executable file.
Uninitialized data are zero-filled pages
Shared libraries from library file
The u area and stacks are setup during process creation (copied from parent).
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

16. Swap Area: VM Backing Store
Swap Area: Pages are copied to the swap device to free up space for running programs.
Swapping plus paging for two-tiered scheme
Requires a swap map to locate swapped out pages
MMU set dirty bit for page if it has been modified
Text pages need not be backed by swap
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

17. Translation Maps
Hardware Translation Tables – each access to memory must have the virtual address translated to a physical memory location
Page tables provide the MMU with this mapping
MMU uses TLB to cache recent translations
Other maps used by the OS:
Address space map – describes a virtual address space for a process or kernel
Physical memory map – kernel uses to perform reverse maps and to describe a pages ownership, references and protections.
Backing store map – used to locate non-resident pages
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

18. Replacement Algorithms
Deciding when to reclaim a page: Defined in terms of criteria used for selecting pages to reclaim
Reference string: pages referenced over time
fault rate: page faults for some length of a reference string (i.e. over a period of time)
Algorithms evaluated based on effectiveness on collected (real) reference strings
Implementations usually require sample reference strings
Local versus global policies:
Most UNIX implementation use a global replacement policy but guarantee a minimum
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

19. Working set Model Assumes a slowing changing locality of reference
processes tend to localize references to a small set of pages
implies that if a page was recently accessed then it will be accessed again in the “near” future
if working set is in memory then few page faults
A simple model is a least recently used (LRU) policy:
if a page has been accessed “recently” then assume it will be need again else assume it will not be needed
else free pages not accessed “recently”
Implement using an approximate set:
number of pages held versus fault rate.
Set high and low water marks
Most kernels implement a scheme whereby pages are periodically freed and placed on a free pool.
Prepaging: working set resident before scheduling process
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

20. Working set model Reference pattern nonuniform but clustered
w(k,t) = pages (set) corresponding to the last k memory references, at time t.
w(1,t)  w(2,t)  …  w(n,t)
|w(k,t)| = size of w is monotonically increasing with k
Practical considerations lead to using a processes virtual time rather than k recent references
consider page references in past  virtual time units.
clear R bit every clock tick, it is set when a page is referenced
on page fault if R is set then update virtual time of last use else set age = current_vt – last_use_vt. If age >  then reclaim page else it is in the working set. continue to scan all entries if no entries found then reclaim oldest. If all had R bit set then randomly select page to reclaim.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

21. WSClock Algorithm
Similar to clock algorithm with circular list and clock hand.
scans list and if R = 1 then it is cleared and the current virtual time is written.
advance hand
if R = 0 then check timestamp > T then replace (if dirty schedule for write else put on free list)
Alternatively, can use two clock hands
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

22. Example Paging System CPU Unitialized data DRAM Stack and heap
Allocated virtual pages
Low Address
(0x )
Text (shared)
Initialized Data
Swap
app1
Address space
Unitialized Data
Disk
Heap (Dynamic)
UFS
High Address
(0x7fffffff)
stack (dynamic)
Environment
Text and
initialized data
app1
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

23. Hardware Requirements
Protection: Prevent process from changing own memory maps
Residency: CPU distinguishes between resident and non-resident pages
Loading: Load pages and restart interrupted program instructions
Dirty: Determine if pages have been modified
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

24. Memory Management Unit
Translates Virtual Addresses
page tables
Translation Lookaside Buffer (TLB)
Page tables
One for kernel addresses
one or more for user space processes
Page Table Entry (PTE) one per virtual page
32 bits - page frame, protection, valid, modified, referenced
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

25. Translation Virtual address: Finds PTE for virtual page
virtual page number + offset
Finds PTE for virtual page
Extract physical page and adds offset
Fail (MMU raises an exception - page fault):
bounds error - outside address range
validation error - non-resident page
protection error - not permitted access
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

26. Some details Limit Page Table size:
segments
page the page table (multi-level page table)
MMU has registers which point to the current page table(s)
kernel and MMU can modify page tables and registers
Problem:
Page tables require perhaps multiple memory access per instruction
Solution:
rely on HW caching (virtual address cache)
cache the translations themselves - TLB
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

27. Translation Lookaside Buffer
Associative cache of address translations
Entries may contain a tag identifying the process as well as the virtual address.
Why is this important?
MMU typically manages the TLB
Kernel may need to invalidate entries,
Would the kernel ever need to invalidate entries?
Contains page table entries that have been most recently used
Functions same way as a memory cache
Given a virtual address, processor examines the TLB
If present (a hit), the frame number is retrieved and the real address is formed
If not found (a miss), page number is used to index the process page table
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

28. Address Translation - General
CPU
virtual
address
cache
MMU
Physical
address
data
Global memory
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

29. Address Translation Overview
MMU
CPU
Virtual
address
TLB
physical
address
cache
context table pointer
context
Page tables
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

30. Cache/Main-Memory Structure
Address
Slot
Number
Tag
Block 1 1 2
Block
(k words) 2 3
C - 1
Block Length
(k words)
(b) Cache
Block
2n - 1
Word Length
(a) Main Memory
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

31. Page Table Entry Resident bit indicates if page is in memory
X bits
Y bits
Virtual address
virtual page number
offset in page
Page Table Entry (PTE) M R
control bits
frame number
Z bits
Resident bit indicates if page is in memory
Modify bit to indicate if page has been altered since loaded into main memory
Other control bits
frame number, this is the physical frame address.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

32. Example 1-level address Translation
Virtual address
20 bits
12 bits
DRAM
Frames
virtual page number
offset in page
Frame X X
offset
add
PTE M R
control bits
frame number
(Process) Page Table
current page table register
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

33. SuperSPARC Reference MMU
Physical address
Context Tbl
Ptr register
Physical page
offset
Context Tbl
24 Bits
12 Bits
PTD
Level 1
Level 2
PTD
Level 2
Context register
PTD
12 bit
PTE
8 bits
6 bits
6 bits
12 bits
Virtual address
index 1
index 2
index 3
offset
4096
virtual page
12 bit index for 4096 entries
8 bit index for 256 entries
6 bit index for 64 entries
Virtual page number has 20 bits for 1M pages
Physical frame number has 24 bits with a 12 bit offset, permitting 16M frames.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

34. Page Table Descriptor/Entry
Page Table Pointer
type
Page Table Entry
Phy Page Number C M R
ACC
type
Type = PTD, PTE, Invalid
C - Cacheable
M - Modify
R - Reference
ACC - Access permissions
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

35. SVR4 VM Architecture File Mapping – Two interpretations
Used as a Fundamental Organizational scheme. Entire Address Space viewed as a collection of mappings to different objects (such as files)
Applications map a file into their address space
Types of Mappings: Shared and Private
Memory Object: represents mapping from region a of memory to backing store (swap, local/remote file, frame buffer)
VM provides common framework, Memory objects provide the specific implementation
operations such as fetching and flushing page to backing store
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

36. VM Address space is a set of mappings to data objects.
An address is only valid if it is mapped to an existing object
File system provides the name space and mechanisms to access data.
Uses the vnode layer to interact with the file system.
Each named memory object is associated with a vnode (but a vnode may map to many objects)
Unnamed objects represented by anonymous objects
Physical memory is treated as a cache for the data objects
Page is the smallest unit of allocation, protection, address translation and mapping.
Address space can be thought of as an array of pages
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

37. File Mapping Versus read/write
process
Process P1
VM Approach
process
Process P2
Traditional Approach
mmap(): Address space
read/write: Copy
Copy
Virtual Memory System
Buffer Cache
P1 pages
Copy
Copy
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

38. Fundamental Abstractions (data structs)
Page (struct page)
Address Space (struct as)
segment (struct seg)
Hardware Address Translation (struct hat)
Anonymous Page (struct anon)
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

39. VM Architecture ... Physical page & offset Proc A virtual address
Translation
physical
address
fork/exec
fault
page tables
AS layer
HAT
anon layer
vnode layer
...
swap layer
page layer
Persistent storage
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

40. Physical Memory Divided into paged and non-paged regions
Paged region described by an array of page structures, each describing one logical page (cluster of hardware pages)
Each physical page (page frame):
described by struct page
mapped to some memory object, with the memory object represented by a vnode
page identity or name = <vnode, offset>
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

41. Page Struct
page struct stores offset and pointer to corresponding vnode
may sit on several linked lists, has 3 sets of pointers
hash table of vnode and offset
vnode contains list of all object pages currently in memory
free page list or list of pages waiting to be written to backing store
Reference count
synchronization flags (lock, wanted, in-transit)
Copies of modified and referenced bits
HAT field used to locate all translations for this page
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

42. AS Layer High-level abstraction describing the virtual address space.
References a linked list of seg (segment) structs that represent non-overlapping page-aligned address regions
Contains the hat structure and a hint to the last segment that had a page fault
Supports two set of operations: those operating on the entire address space and those that affect ranges within the space.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

43. Segment Drivers
Segments represent mappings between backing store and address regions
Segment represents an abstract base class with specific drivers being derived classes.
seg struct contains pointer to
a seg_ops vector, these represent the virtual functions. i.e. the type dependent interface to the class.
Methods = {dup, fault, faulta, setprot, checkprot, unmap, swapout, sync}
type-dependent data structure which hold private data
Each segment defines a create routine
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

44. Segment Drivers Different types: seg_vn, seg_map, seg_dev, seg_kmem
seg_vn: vnode segment, maps to regular files and anonymous object.
Seg_map: One in system. Use by kernel for transient file mappings for implementing read/write.
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

45. Process Address Space proc struct struct segvn_data {} struct seg {
base
size}
struct segvn_data {}
text
struct segvn_data {}
struct seg {
base
size}
struct as {
segment list
hint
struct hat {}}
data
struct seg {
base
size}
struct segvn_data {}
stack
seg_vn ops
struct seg {
base
size}
struct segu_data {}
seg_u ops
u area
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

46. Anonymous Pages
Page with no permanent storage, created when process write to a MAP_PRIVATE object
Pages can be discarded with process terminates or unmaps the page.
Swap device used as backing store
Example: initialized data pages when modified become anonymous pages
Related but distinct concept is an anonymous object.
one anonymous object in system represented bu the NULL vnode pointer (/dev/zero) and is the source of all zero-filled pages.
unitialized data and stack regions are MAP_PRIVATE to it
Shared memory regions are MAP_SHARED to it
anonymous object pages are anonymous pages
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

47. Anonymous vnode layer read and writes pages to swap device
one for each
swap device
Swap device
Swap info
vnode
pointer to
anon and
freelist
anon ref
array as
anon[]
seg
anon_map
free list
segvn_data
page
one entry for
each page
in swap
vnode
page
page
Per page protect
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

48. Hardware Address Translation Layer
Isolates all hardware-dependent code from rest of VM
Responsible for all address translations
setup and maintain mappings used by MMU (page tables, directories etc)
each process has it’s own set of translations
uses struct hat which is part of the as struct
Operations
hat layer: hat_alloc, hat_free, hat_dup, hat_swapin, hat_swapout (build/rebuild tables when swapping)
range of pages: hat_chgprot, hat_unload, hat_memload, hat_devload
all translation of a page: hat_pageunload, hat_pagesync (update modified and referenced bits using values in page struct)
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

49. Misc Topics
Pagedaemon implements page reclamation (replacement) policy. Uses two-handed clock algorithm
Swapping - swapper daemon will swap processes when space gets below a threshold
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

50. Misc Topics Cont Non-page-aligned backing store
Virtual swap space in Solaris (swapfs)
includes physical memory
dynamic reallocation
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

51. Assessment - Advantages
Modular design – OO style interface encapsulating functionality
Portability – HAT layer simplifies porting
Sharing – copy-on-write sharing, MAP_SHARED, shared file mappings
Lightweight file access – mmap
Enabling shared library support
Leveraging existing interfaces (vnode)
Integrating VM and buffer cache
Facilitates break point insertion with MAP_PRIVATE
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

52. Assessment - Disadvantages
Increased table size and maintenance requirements
Increased time to read program text and initialized data from file rather than swap
Performance problems due to longer code paths, complexity and indirection
disk address are computed dynamically
Invariant interfaces to abstractions may lead to inefficient implementation (no control over implementation)
copy-on-write may not be faster than anticipatory copying
Swap space allocated per-page basis preventing optimizations like clustering and prepaging
Fred Kuhns (3/31/2017)
CS523 – Operating Systems

53. Improvements Reduce high fault rate caused by lazy evaluations
Fred Kuhns (3/31/2017)
CS523 – Operating Systems