1. Virtual Memory Prof. Eric Rotenberg
ECE 463/563 Fall `18
Virtual Memory
Prof. Eric Rotenberg
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

2. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
Virtual Memory
Every program has its own virtual memory
Large virtual address space
Divided into virtual pages
When a program runs, it needs physical memory
Physical memory is actual storage:
DRAM – main memory
Hard Disk – overflow storage for main memory
Operating System (O/S) manages physical memory as a shared resource among many running programs
When a program first accesses a particular virtual page, O/S is invoked to allocate a physical page in main memory that will now correspond to the program’s virtual page
Upon starting a program: The O/S “loader” allocates initial physical pages for the program’s text (code) and data segments (globals, stack)
During program execution: The O/S “page-fault handler” allocates physical pages for first-time accesses to new virtual pages (e.g., heap, stack). The handler also swaps physical pages between the main memory (in DRAM) and the overflow storage for main memory (in Hard Disk).
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

3. Physical Page Number (PPN)
Virtual Memory (cont.)
Virtual Memory for Program #1
Virtual Memory for Program #2
Physical Memory
Virtual Page Number
(VPN)
Virtual Page Number
(VPN) 1 1 2
DRAM: Main Memory 2 3 3 4 4 5
Physical Page Number (PPN) 5 1 6 6 2 7 7 3 8 8 4 5 … N … …
1 physical page
Hard Disk: Overflow Storage for Main Memory
swap space
file system
1 virtual page
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

4. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
Virtual Memory (cont.)
Virtual Memory for Program #1
Virtual Memory for Program #2
Physical Memory
VPN:
VPN: 1 1 2
DRAM: Main Memory 2 3 3
PPN: 4 4 5 5 1 6 6 2 7 7 3 8 8 4 5 … N … …
Hard Disk: Overflow Storage for Main Memory
swap space
file system
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

5. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
Virtual Memory (cont.)
Virtual Memory for Program #1
Virtual Memory for Program #2
Physical Memory
VPN:
VPN: 1 1 2
DRAM: Main Memory 2 3 3
PPN: 4 4 5 5 1 6 6 2 7 7 3 8 8 4 9 5 10 … 11 12 N … … ??
Hard Disk: Overflow Storage for Main Memory
swap space
file system
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

6. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
O/S Page Tables
O/S maintains a page table (PT) per running program
PT is software data structure used to translate the program’s virtual addresses to physical addresses
PT is searched using the virtual address
There is one PT entry for each virtual page used by the program. Contents of PT entry:
Whether corresponding physical page is in main memory (DRAM) or in swap space (hard disk)
If in main memory: PT entry provides physical page number
If in swap space: PT entry provides location on disk
PT entry typically has other information too (recency of access, protection bits, etc.)
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

7. Virtual-to-Physical Address Translation
A running program
O/S Page Table for running program
Access L1 I/D caches (entry point to memory hierarchy)
virtual address
physical address
translate virtual address to physical address
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

8. Virtual-to-Physical Address Translation
VPN
DRAM
First time VPN referenced?
yes
PPN
first ever access to VPN
Scenario 3
Disk no
In DRAM or on Disk?
DRAM
on Disk
PPN
Scenario 2
Swap-in from disk to DRAM (“page fault”)
Scenario 1
Disk
DRAM
In DRAM
PPN
PPN
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

9. Overhead of Virtual Memory
Program counter is a virtual address
Each instruction fetch requires address translation
Loads and stores generate virtual addresses
Each load and store requires address translation
For every instruction fetch, load, and store, call O/S to search the page table???
This would have unacceptable performance
The O/S address translation function takes 10s to 100s of instructions and several memory accesses (to access page table). Exact overhead depends on the scenario, page table organization, etc.
There has to be a better way…
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

10. Translation Lookaside Buffer (TLB)
The TLB is a small cache of recently used address translations
TLB is defined in the ISA, because software and hardware collaborate w.r.t. the TLB
Hardware role
Hardware provides and accesses the TLB
Provides: TLB is a hardware table
Accesses: Hardware searches the TLB for desired translation. If TLB doesn’t have translation, hardware calls the O/S.
Software role
Software manages the TLB
When there is a TLB miss, O/S is responsible for handling the miss. Once it has a translation, it writes the translation into the TLB, at a location of its choosing.
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

11. Virtual-to-Physical Address Translation, with TLB
translate virtual address to physical address
A running program
TLB
Access L1 I/D caches (entry point to memory hierarchy)
CPU
is executing application
virtual address
physical address
1 cycle
TLB miss exception
TLB-write instruction puts translation into TLB
O/S Page Table for running program
CPU
is executing O/S TLB miss exception handler
10s-100s of cycles
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

12. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
TLB Organization
TLB organization
Can be direct-mapped, set-associative, or fully-associative
Many modern RISC ISAs define it to be fully-associative since software manages it
Unified versus split TLBs
Unified: One TLB for both instruction and data address translation
Split: Separate TLBs for instruction and data address translation
I-TLB: TLB for instruction address translation (program counter). I-TLB sits alongside L1 I-cache.
D-TLB: TLB for data address translation (loads and stores). D-TLB sits alongside L1 D-cache.
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

13. Using the TLB for translation
Example: Fully-associative TLB X
valid bit
virtual page number
physical page number =?
VPN
PPN v =?
VPN
PPN v =?
VPN
PPN v X’ X =?
VPN
PPN v X’
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

14. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
TLB increases hit time
Without Virtual Memory
With Virtual Memory
virtual address
address
TLB
1 cycle
L1 Cache
1 cycle
physical address
L1 Cache
1 cycle
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

15. Using the TLB for translation: A closer look
virtual page number
page offset 31 12 11
TLB
tag
block
offset
index
physical page number
virtual address
Ex: page size = 4KB
# page offset bits = log2(4KB) = 12
physical address
This is how cache interprets
the physical address
Observation: What if index bits were entirely contained in page offset bits?
Then first part of cache access (indexing) would not wait on TLB
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

16. Accessing TLB and Cache in Parallel
data array
word select
tag array =?
virtual page number
page offset 31 12 11
TLB
tag
physical page number
index
block offset
Cache hit time reduces from two cycles to one!
Because cache can now be indexed in parallel with TLB access (only the final tag match uses the output from TLB)
But some constraints...
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

17. Constraint: Size of 1 cache way
Constraint for “physically-indexed cache with parallel cache/TLB access”
Index and block offset bits contained within page offset bits
Therefore: Total amount of storage in 1 way of the cache should not exceed page size
block size
set
Way 1
Way 2
Way N
# sets
N-way set-assoc. cache …
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

18. Page size / associativity tradeoff
From previous slide:
Cache size equation:
Therefore:
Example: MC88110
Page size = 4KB
I$, D$ both: 8KB 2-way set-associative
(8KB/4KB) = 2 ways
Example: VAX series
Page size = 512B
For a 16KB cache, need assoc. = (16KB / 512B) = 32-way set-associative!
Moral: sometimes associativity is thrust upon you
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

19. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
Backup Slides
The following slides are for interest only
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

20. Notation on following slide
X: Virtual page number to be translated into a physical page number
X’: Physical page number corresponding to X
V: Virtual page number of a victim page
struct page_table_entry { bool resident; // ‘true’ if in DRAM, ‘false’ if on disk (swap space) unsigned int ppn; // if resident==true, this is physical page number disk_loc_type disk_loc; // if resident==false, this is location on disk }
// PT is a hash table (unbounded array) of page table entries. // Key (index) of the hash table is virtual page number.
If true: X is in DRAM, at PT[X].ppn. Thus X’ = PT[X].ppn.
PT[X].resident
If false: X is not in DRAM. PT[X].ppn is bogus, PT[X].disk_loc indicates disk location.
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

21. Virtual-to-Physical Address TranslationX
Is X in PT?
Is there a free DRAM page?
SwapOut(V)
PT[V].resident = false PT[V].disk_loc = …
PT[X].resident = true
PT[X].ppn = PT[V].ppn no no
first ever access to X
PT[X].resident = true
PT[X].ppn = freelist.pop()
yes
yes
SwapOut(V)
PT[V].resident = false PT[V].disk_loc = …
PT[X].resident = true
PT[X].ppn = PT[V].ppn
SwapIn(X)
PT[X].resident
Is there a free DRAM page?
false no
Swap-in X from disk to DRAM (“page fault”)
PT[X].resident = true
PT[X].ppn = freelist.pop()
SwapIn(X)
yes
true
X’ = PT[X].ppn X’
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg