1. CS 152 Computer Architecture and Engineering Lecture 11 - Virtual Memory and Caches Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste http://inst.eecs.berkeley.edu/~cs152

2. 3/6/2008CS152-Spring’08 2 Last time in Lectures 10 Modern page-based virtual memory systems provide: –Translation »to avoid memory fragmentation and provide each user with own virtual address space –Protection »to protect users from each other, and to protect operating system from users –Virtual memory »to allow main memory to act as a cache of a larger disk memory, equivalent Translation and protection information stored in page tables, held in main memory Translation and protection information cached in “translation lookaside buffer” (TLB) to provide single cycle translation+protection check in common case

3. 3/6/2008CS152-Spring’08 3 Today is a review Many of you had questions about virtual memory and interaction with caches Going to go back over core of last two lectures with some more interactive explanation

4. 3/6/2008CS152-Spring’08 4 Simple Base and Bound Translation Load X Program Address Space Bound Register  Bounds Violation? Main Memory current segment Base Register + Physical Address Effective Address Base and bounds registers are visible/accessible only when processor is running in the supervisor mode Base Physical Address Segment Length

5. 3/6/2008CS152-Spring’08 5 Separate Areas for Program and Data What is an advantage of this separation? (Scheme used on all Cray vector supercomputers prior to X1, 2002) Load X Program Address Space Main Memory data segment Data Bound Register Effective Addr Register Data Base Register  + Bounds Violation? Program Bound Register Program Counter Program Base Register  + Bounds Violation? program segment

6. 3/6/2008CS152-Spring’08 6 Memory Fragmentation As users come and go, the storage is “fragmented”. Therefore, at some stage programs have to be moved around to compact the storage. OS Space 16K 24K 32K 24K user 1 user 2 user 3 OS Space 16K 24K 16K 32K 24K user 1 user 2 user 3 user 5 user 4 8K Users 4 & 5 arrive Users 2 & 5 leave OS Space 16K 24K 16K 32K 24K user 1 user 4 8K user 3 free

7. 3/6/2008CS152-Spring’08 7 Processor generated address can be interpreted as a pair A page table contains the physical address of the base of each page Paged Memory Systems Page tables make it possible to store the pages of a program non-contiguously. 0 1 2 3 0 1 2 3 Address Space of User-1 Page Table of User-1 1 0 2 3 page number offset

8. 3/6/2008CS152-Spring’08 8 Private Address Space per User Each user has a page table Page table contains an entry for each user page VA1 User 1 Page Table VA1 User 2 Page Table VA1 User 3 Page Table Physical Memory free OS pages

9. 3/6/2008CS152-Spring’08 9 Page Tables in Physical Memory VA1 User 1 PT User 1 PT User 2 VA1 User 2

10. 3/6/2008CS152-Spring’08 10 Linear Page Table VPN Offset Virtual address PT Base Register VPN Data word Data Pages Offset PPN DPN PPN Page Table DPN PPN DPN PPN Page Table Entry (PTE) contains: –A bit to indicate if a page exists –PPN (physical page number) for a memory-resident page –DPN (disk page number) for a page on the disk –Status bits for protection and usage OS sets the Page Table Base Register whenever active user process changes

11. 3/6/2008CS152-Spring’08 11 Size of Linear Page Table With 32-bit addresses, 4-KB pages & 4-byte PTEs:  2 20 PTEs, i.e, 4 MB page table per user  4 GB of swap needed to back up full virtual address space Larger pages? Internal fragmentation (Not all memory in a page is used) Larger page fault penalty (more time to read from disk) What about 64-bit virtual address space??? Even 1MB pages would require 2 44 8-byte PTEs (35 TB!) What is the “ saving grace ” ? sparsity of virtual address usage

12. 3/6/2008CS152-Spring’08 12 Hierarchical Page Table Level 1 Page Table Level 2 Page Tables Data Pages page in primary memory page in secondary memory Root of the Current Page Table p1 offset p2 Virtual Address (Processor Register) PTE of a nonexistent page p1 p2 offset 0 1112212231 10-bit L1 index 10-bit L2 index

13. 3/6/2008CS152-Spring’08 13 Address Translation & Protection Every instruction and data access needs address translation and protection checks A good VM design needs to be fast (~ one cycle) and space efficient Physical Address Virtual Address Address Translation Virtual Page No. (VPN)offset Physical Page No. (PPN)offset Protection Check Exception? Kernel/User Mode Read/Write

14. 3/6/2008CS152-Spring’08 14 Translation Lookaside Buffers Address translation is very expensive! In a two-level page table, each reference becomes several memory accesses Solution: Cache translations in TLB TLB hitSingle Cycle Translation TLB miss Page Table Walk to refill VPN offset V R W D tag PPN physical address PPN offset virtual address hit? (VPN = virtual page number) (PPN = physical page number)

15. 3/6/2008CS152-Spring’08 15 TLB Designs Typically 32-128 entries, usually fully associative –Each entry maps a large page, hence less spatial locality across pages  more likely that two entries conflict –Sometimes larger TLBs (256-512 entries) are 4-8 way set-associative Random or FIFO replacement policy TLB Reach: Size of largest virtual address space that can be simultaneously mapped by TLB Example: 64 TLB entries, 4KB pages, one page per entry TLB Reach = _____________________________________________? 64 entries * 4 KB = 256 KB (if contiguous)

16. 3/6/2008CS152-Spring’08 16 Address Translation in CPU Pipeline Software handlers need restartable exception on TLB fault Handling a TLB miss needs a hardware or software mechanism to refill TLB Need mechanisms to cope with the additional latency of a TLB: – slow down the clock – pipeline the TLB and cache access – virtual address caches – parallel TLB/cache access PC Inst TLB Inst. Cache D Decode EM Data TLB Data Cache W + TLB miss? Page Fault? Protection violation? TLB miss? Page Fault? Protection violation?

17. 3/6/2008CS152-Spring’08 17 Handling a TLB Miss Software (MIPS, Alpha) TLB miss causes an exception and the operating system walks the page tables and reloads TLB. A privileged “untranslated” addressing mode used for walk Hardware (SPARC v8, x86, PowerPC) A memory management unit (MMU) walks the page tables and reloads the TLB If a missing (data or PT) page is encountered during the TLB reloading, MMU gives up and signals an exception for the original instruction

18. 3/6/2008CS152-Spring’08 18 Virtual Memory More than just translation and protection Use disk to extend apparent size of main memory Treat DRAM as cache of disk contents Only need to hold active working set of processes in DRAM, rest of memory image can be swapped to disk Inactive processes can be completely swapped to disk (except usually the root of the page table) Combination of hardware and software used to implement this feature (ATLAS was first implementation of this idea)

19. 3/6/2008CS152-Spring’08 19 Page Fault Handler When the referenced page is not in DRAM: –The missing page is located (or created) –It is brought in from disk, and page table is updated Another job may be run on the CPU while the first job waits for the requested page to be read from disk –If no free pages are left, a page is swapped out Pseudo-LRU replacement policy Since it takes a long time to transfer a page (msecs), page faults are handled completely in software by the OS –Untranslated addressing mode is essential to allow kernel to access page tables

20. 3/6/2008CS152-Spring’08 20 Caching vs. Demand Paging CPU cache primary memory secondary memory Caching Demand paging cache entrypage frame cache block (~32 bytes)page (~4K bytes) cache miss rate (1% to 20%)page miss rate (<0.001%) cache hit (~1 cycle)page hit (~100 cycles) cache miss (~100 cycles)page miss (~5M cycles) a miss is handled in hardware mostly in software primary memory CPU

21. 3/6/2008CS152-Spring’08 21 Address Translation: putting it all together Virtual Address TLB Lookup Page Table Walk Update TLB Page Fault (OS loads page) Protection Check Physical Address (to cache) miss hit the page is  memory  memory denied permitted Protection Fault hardware hardware or software software SEGFAULT Restart instruction

22. 3/6/2008CS152-Spring’08 22 Protection and Translation These have been combined in a modern virtual memory system, but are really separate functions Question, does translation itself provide enough protection?

23. 3/6/2008CS152-Spring’08 23 CS152 Administrivia Tuesday Mar 18, Quiz 3 –Virtual memory hierarchy lectures Lab 8-10

24. 3/6/2008CS152-Spring’08 24 Address Translation in CPU Pipeline Software handlers need restartable exception on TLB fault Handling a TLB miss needs a hardware or software mechanism to refill TLB Need mechanisms to cope with the additional latency of a TLB: – slow down the clock – pipeline the TLB and cache access – virtual address caches – parallel TLB/cache access PC Inst TLB Inst. Cache D Decode EM Data TLB Data Cache W + TLB miss? Page Fault? Protection violation? TLB miss? Page Fault? Protection violation?

25. 3/6/2008CS152-Spring’08 25 Virtual Address Caches one-step process in case of a hit (+) cache needs to be flushed on a context switch unless address space identifiers (ASIDs) included in tags (-) aliasing problems due to the sharing of pages (-) maintaining cache coherence (-) (see later in course) CPU Physical Cache TLB Primary Memory VA PA Alternative: place the cache before the TLB CPU VA (StrongARM) Virtual Cache PA TLB Primary Memory

26. 3/6/2008CS152-Spring’08 26 Aliasing in Virtual-Address Caches VA 1 VA 2 Page Table Data Pages PA VA 1 VA 2 1st Copy of Data at PA 2nd Copy of Data at PA TagData Two virtual pages share one physical page Virtual cache can have two copies of same physical data. Writes to one copy not visible to reads of other! General Solution: Disallow aliases to coexist in cache Software (i.e., OS) solution for direct-mapped cache VAs of shared pages must agree in cache index bits; this ensures all VAs accessing same PA will conflict in direct- mapped cache (early SPARCs)

27. 3/6/2008CS152-Spring’08 27 Concurrent Access to TLB & Cache Index L is available without consulting the TLB cache and TLB accesses can begin simultaneously Tag comparison is made after both accesses are completed Cases: L + b = kL + b k VPN L b TLB Direct-map Cache 2 L blocks 2 b -byte block PPN Page Offset = hit? DataPhysical Tag Tag VA PA Virtual Index k

28. 3/6/2008CS152-Spring’08 28 Virtual-Index Physical-Tag Caches: Associative Organization Is this scheme realistic? VPN a L = k-b b TLB Direct-map 2 L blocks PPN Page Offset = hit? Data Phy. Tag VA PA Virtual Index k Direct-map 2 L blocks 2a2a = 2a2a After the PPN is known, 2 a physical tags are compared

29. 3/6/2008CS152-Spring’08 29 Concurrent Access to TLB & Large L1 The problem with L1 > Page size Can VA 1 and VA 2 both map to PA ? VPN a Page Offset b TLB PPN Page Offset b Tag VA PA Virtual Index L1 PA cache Direct-map = hit? PPN a Data VA 1 VA 2

30. 3/6/2008CS152-Spring’08 30 A solution via Second Level Cache Usually a common L2 cache backs up both Instruction and Data L1 caches L2 is “inclusive” of both Instruction and Data caches CPU L1 Data Cache L1 Instruction Cache Unified L2 Cache RF Memory

31. 3/6/2008CS152-Spring’08 31 Anti-Aliasing Using L2: MIPS R10000 VPN a Page Offset b TLB PPN Page Offset b Tag VA PA Virtual Index L1 PA cache Direct-map = hit? PPN a Data VA 1 VA 2 Direct-Mapped L2 PA a 1 Data PPN into L2 tag Suppose VA1 and VA2 both map to PA and VA1 is already in L1, L2 (VA1  VA2) After VA2 is resolved to PA, a collision will be detected in L2. VA1 will be purged from L1 and L2, and VA2 will be loaded  no aliasing !

32. 3/6/2008CS152-Spring’08 32 Virtually-Addressed L1: Anti-Aliasing using L2 VPN Page Offset b TLB PPN Page Offset b Tag VA PA Virtual Index & Tag Physical Index & Tag L1 VA Cache L2 PA Cache L2 “contains” L1 PA VA 1 Data VA 1 Data VA 2 Data “Virtual Tag” Physically-addressed L2 can also be used to avoid aliases in virtually- addressed L1

33. 3/6/2008CS152-Spring’08 33 Variable-Sized Page Support Level 1 Page Table Level 2 Page Tables Data Pages page in primary memory large page in primary memory page in secondary memory PTE of a nonexistent page Root of the Current Page Table p1 offset p2 Virtual Address (Processor Register) p1 p2 offset 0 1112212231 10-bit L1 index 10-bit L2 index

34. 3/6/2008CS152-Spring’08 34 Variable-Size Page TLB Some systems support multiple page sizes. VPN offset physical address PPN offset virtual address hit? VRWD Tag PPN L

35. 3/6/2008CS152-Spring’08 35 Atlas Revisited One PAR for each physical page PAR’s contain the VPN’s of the pages resident in primary memory Advantage: The size is proportional to the size of the primary memory What is the disadvantage ? VPN PAR’s PPN

36. 3/6/2008CS152-Spring’08 36 Hashed Page Table: Approximating Associative Addressing hash Offset Base of Table + PA of PTE Primary Memory VPN PID PPN Page Table VPNd Virtual Address VPN PID DPN VPN PID PID Hashed Page Table is typically 2 to 3 times larger than the number of PPN’s to reduce collision probability It can also contain DPN’s for some non-resident pages (not common) If a translation cannot be resolved in this table then the software consults a data structure that has an entry for every existing page

37. 3/6/2008CS152-Spring’08 37 Acknowledgements These slides contain material developed and copyright by: –Arvind (MIT) –Krste Asanovic (MIT/UCB) –Joel Emer (Intel/MIT) –James Hoe (CMU) –John Kubiatowicz (UCB) –David Patterson (UCB) MIT material derived from course 6.823 UCB material derived from course CS252