1. Virtual Memory

2. Virtual memory
Build new hardware that automatically translates each memory reference from a virtual address (that the programmer sees as an array of bytes) to a physical address (that the hardware uses to either index DRAM or identify where the storage resides on disk)

3. Basics of Virtual memory
Any time you see the word virtual in computer science/architecture it means “using a level of indirection”
Virtual memory hardware changes the virtual address the programmer see into the physical ones the memory chips see.
0x800
0x3C00
Disk ID 803C4
Virtual address
Physical address

4. Another View of the Memory Hierarchy
Regs
Upper Level
Instr. Operands
Faster
Thus far {
Cache
Blocks
L2 Cache
Blocks
Memory {
Next:
Virtual
Memory
Pages
Disk
Files
Larger
Tape
Lower Level

5. Virtual Memory
If Principle of Locality allows caches to offer (usually) speed of cache memory with size of DRAM memory, then recursively why not use at next level to give speed of DRAM memory, size of Disk memory?
Called “Virtual Memory”
Also allows OS to share memory, protect programs from each other
Today, more important for protection vs. just another level of memory hierarchy
Historically, it predates caches

6. Basic Issues in Virtual Memory System Design
Size of information blocks that are transferred from secondary to main storage (M)
Block of information brought into M, and M is full, then some region of M must be released to make room for the new block replacement policy
which region of M is to hold the new block --> placement policy
disk
mem
cache
reg
pages
frame
Paging Organization
virtual and physical address space partitioned into blocks of equal size
page frames
pages

7. Virtual Memory View
Virtual memory lets the programmer “see” a memory array larger than the DRAM available on a particular computer system.
Virtual memory enables multiple programs to share the physical memory without:
Knowing other programs exist.
Worrying about one program modifying the data contents of another.

8. Managing virtual memory
Managed by hardware logic and operating system software.
Hardware for speed.
Software for flexibility and because disk storage is controlled by the operating system.

9. Virtual to Physical Address Translation
Program
operates in
its virtual
address
space
Physical
memory
(incl. caches) HW
mapping
virtual
address
(inst. fetch
load, store)
physical
address
(inst. fetch
load, store)
Each program operates in its own virtual address space; ~only program running
Each is protected from the other
OS can decide where each goes in memory
Hardware (HW) provides virtual -> physical mapping

10. Virtual Memory Treat main memory like a cache
Misses go to the disk
How do we minimize disk accesses?
Buy lots of memory.
Exploit temporal locality
Fully associative? Set associative? Direct mapped?
Exploit spatial locality
How big should a block be?
Write-back or write-through?

11. Virtual memory terminology
Blocks are called Pages
A virtual address consists of
A virtual page number
A page offset field (low order bits of the address)
Misses are call Page faults
and they are generally handled as an exception
Virtual page number
Page offset 31 11

12. Address Translation Virtual Physical address address Address
Disk
addresses

13. Need a Page Table (table that tracks where all virtual memory pages are) components
Page table register
Virtual page number
Page offset
valid
Physical page number 1
Physical page number
Physical page number
Page offset

14. Page table components Page table register 0x00004 0x0F3 1 0x0F3
valid
Physical page number 1
0x020C0
0x020C0
0x0F3
Physical address = 0x020C00F3

15. Page table components Page table register 0x00002 0x082 Exception:
valid
Physical page number
Exception:
page fault
Disk address
Stop this process
Pick page to replace
Write back data
Get referenced page
Update page table
Reschedule process

16. Putting it all together
Loading your program in memory
Ask operating system to create a new process
Construct a page table for this process
Mark all page table entries as invalid with a pointer to the disk image of the program
That is, point to the executable file containing the binary.
Run the program and get an immediate page fault on the first instruction.

17. Mapping Virtual Memory to Physical Memory
Divide into equal sized chunks (about 4KB)
Stack
Any chunk of Virtual Memory assigned to any chuck of Physical Memory (“page”)
Page Frame (PF) holds a page in memory
Physical Memory
Heap
64 MB
Static
Code

18. Paging Organization (assume 1 KB pages)
1024
7168
Physical
Address
Memory 1K
page 1
page 7
...
Page is unit of mapping
page 0 1K
1024
31744
Virtual Memory
Virtual
Address
page 1
page 31
2048
page 2
...
Addr
Trans
MAP
Page also unit of transfer from disk to physical memory

19. Virtual Memory Mapping Function
Cannot have simple function to predict arbitrary mapping
Use table lookup of mappings
Page Number Offset
Use table lookup (“Page Table”) for mappings: Page number is index
Virtual Memory Mapping Function
Physical Offset = Virtual Offset
Physical Page Number = PageTable[Virtual Page Number]
(P.P.N. also called “Page Frame”)

20. Page Table
A page table is an operating system structure which contains the mapping of virtual addresses to physical locations
There are several different ways, all up to the operating system, to keep this data around
Each process running in the operating system has its own page table
“State” of process is PC, all registers, plus page table
OS changes page tables by changing contents of Page Table Base Register

21. Size of page table How big is a page table entry?
For MIPS the virtual address is 32 bits
If the machine can support 1GB of physical memory and we use 4KB pages, then the physical page number is or 18 bits. Plus another valid bit + other useful stuff (read only, dirty, etc.)
Let say about 3 bytes.
How many entries in the page table?
MIPS virtual address is 32 bits – 12 bit page offset = 220 or ~1,000,000 entries
Total size of page table: ~ 3 megabytes

22. Address Mapping: Page Table
Virtual Address:
page no.
offset
concatenation
index
into
page
table +
Physical
Memory
Address
Page Table
Val
-id
Access
Rights
Physical
Page
Address . V
A.R.
P. P. A.
...
Page Table
Base Reg
Page Table located in physical memory

23. VM Address Translation
Parameters
P = 2p = page size (bytes). Typically 1KB–16KB
N = 2n = Virtual address limit
M = 2m = Physical address limit
virtual page number
page offset
virtual address
physical page number
physical address
p–1
address translation p
m–1
n–1
Notice that the page offset bits don't change as a result of translation

24. Page Tables Page Table Physical Memory Disk Storage Virtual Page
Number
Page Table
(physical page
or disk address)
Physical Memory
Valid 1 1 1 1 1 1
Disk Storage 1

25. A System with Physical Memory Only
Example:
Most Cray machines, early PCs, nearly all embedded systems, etc.
CPU 0: 1:
N-1:
Memory
Store 0x10
Load 0xf0
CPU’s load or store addresses used directly to access memory.

26. A System with Virtual Memory
Examples:
workstations, servers, modern PCs, etc.
CPU 0: 1:
N-1:
Memory
Load 0xf0
P-1:
Page Table
Store 0x10
Disk
Virtual
Addresses
Physical
Address Translation: the hardware converts virtual addresses into physical addresses via an OS-managed lookup table (page table)

27. Page Faults (Similar to “Cache Misses”)
What if an object is on disk rather than in memory?
Page table entry indicates that the virtual address is not in memory
An OS trap handler is invoked, moving data from disk into memory
current process suspends, others can resume
OS has full control over placement, etc.
CPU 0: 1:
N-1:
Memory
Load 0x05
P-1:
Page Table
Store 0xf8
Disk
Virtual
Addresses
Physical

28. Servicing a Page Fault (1) Initiate Block Read
Processor Signals Controller
Read block of length P starting at disk address X and store starting at memory address Y
Read Occurs
Direct Memory Access
Under control of I/O controller
I / O Controller Signals Completion
Interrupt processor
Can resume suspended process
Processor
Reg
(3) Read Done
Cache
Memory-I/O bus
(2) DMA Transfer
I/O
controller
Memory
disk
Disk
Disk
disk

29. Motivation: Memory Management for Multiple Processes
Multiple processes can reside in physical memory.
How do we resolve address conflicts?
(Virtual) Memory Image for Alpha Process
Reserved
Text (Code)
Static Data
Not yet allocated
Stack
Dynamic Data FF
$gp
$sp
e.g., what if two different processes access their stacks at address 0x11fffff80 at the same time?

30. Solution: Separate Virtual Address Spaces
Virtual and physical address spaces divided into equal-sized blocks
“Pages” (both virtual and physical)
Each process has its own virtual address space
operating system controls how virtual pages as assigned to physical memory
Virtual Addresses
Physical Addresses
Address Translation
VP 1
PP 2
Process 1:
VP 2
N-1
(Read-only library code)
PP 7
Process 2:
VP 1
VP 2
PP 10
N-1
M-1

31. Motivation : Process Protection
Page table entry contains access rights information
Hardware enforces this protection (trap into OS if violation occurs)
Page Tables
Memory
Read?
Write?
Physical Addr 0:
VP 0:
VP 1:
VP 2:
Yes No
PP 9 1:
Process i:
Yes
Yes
PP 4 No No
XXXXXXX • • •
Read?
Write?
Physical Addr
VP 0:
VP 1:
VP 2:
Yes
Yes
PP 6
Process j:
Yes No
PP 9
N-1: No No
XXXXXXX • • •

32. Virtual Memory Problem #1
Not enough physical memory!
Only, say, 64 MB of physical memory
N processes, each 4GB of virtual memory!
Could have 1K virtual pages/physical page!
Spatial Locality to the rescue
Each page is 4 KB, lots of nearby references
No matter how big program is, at any time only accessing a few pages
“Working Set”: recently used pages

33. Virtual Memory Problem #2
Map every address  1 extra memory accesses for every memory access
Observation: since locality in pages of data, must be locality in virtual addresses of those pages
Why not use a cache of virtual to physical address translations to make translation fast? (small is fast)
For historical reasons, cache is called a Translation Lookaside Buffer, or TLB

34. Translation Look-Aside Buffers
TLBs usually small, typically entries
Like any other cache, the TLB can be fully associative, set associative, or direct mapped VA
hit PA
miss
TLB
Lookup
Cache
Main
Memory
Processor
miss
hit
Trans-
lation
data

35. Virtual Memory Summary
Caches: Location, Organization (block size and associativity), Replacement
Virtual memory provides
protection, sharing, illusion of large main memory
Virtual Memory requires twice as many memory accesses, so we cache page table entries in the TLB.
Three things can go wrong on a memory access: cache miss, TLB miss, page fault.