Virtual Memory Prof. Eric Rotenberg ECE 463/563 Fall `18 Virtual Memory Prof. Eric Rotenberg Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg Virtual Memory Every program has its own virtual memory Large virtual address space Divided into virtual pages When a program runs, it needs physical memory Physical memory is actual storage: DRAM – main memory Hard Disk – overflow storage for main memory Operating System (O/S) manages physical memory as a shared resource among many running programs When a program first accesses a particular virtual page, O/S is invoked to allocate a physical page in main memory that will now correspond to the program’s virtual page Upon starting a program: The O/S “loader” allocates initial physical pages for the program’s text (code) and data segments (globals, stack) During program execution: The O/S “page-fault handler” allocates physical pages for first-time accesses to new virtual pages (e.g., heap, stack). The handler also swaps physical pages between the main memory (in DRAM) and the overflow storage for main memory (in Hard Disk). Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Physical Page Number (PPN) Virtual Memory (cont.) Virtual Memory for Program #1 Virtual Memory for Program #2 Physical Memory Virtual Page Number (VPN) Virtual Page Number (VPN) 1 1 2 DRAM: Main Memory 2 3 3 4 4 5 Physical Page Number (PPN) 5 1 6 6 2 7 7 3 8 8 4 5 … N … … 1 physical page Hard Disk: Overflow Storage for Main Memory swap space file system 1 virtual page Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg Virtual Memory (cont.) Virtual Memory for Program #1 Virtual Memory for Program #2 Physical Memory VPN: VPN: 1 1 2 DRAM: Main Memory 2 3 3 PPN: 4 4 5 5 1 6 6 2 7 7 3 8 8 4 5 … N … … Hard Disk: Overflow Storage for Main Memory swap space file system Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg Virtual Memory (cont.) Virtual Memory for Program #1 Virtual Memory for Program #2 Physical Memory VPN: VPN: 1 1 2 DRAM: Main Memory 2 3 3 PPN: 4 4 5 5 1 6 6 2 7 7 3 8 8 4 9 5 10 … 11 12 N … … ?? Hard Disk: Overflow Storage for Main Memory swap space file system Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg O/S Page Tables O/S maintains a page table (PT) per running program PT is software data structure used to translate the program’s virtual addresses to physical addresses PT is searched using the virtual address There is one PT entry for each virtual page used by the program. Contents of PT entry: Whether corresponding physical page is in main memory (DRAM) or in swap space (hard disk) If in main memory: PT entry provides physical page number If in swap space: PT entry provides location on disk PT entry typically has other information too (recency of access, protection bits, etc.) Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Virtual-to-Physical Address Translation A running program O/S Page Table for running program Access L1 I/D caches (entry point to memory hierarchy) virtual address physical address translate virtual address to physical address Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Virtual-to-Physical Address Translation VPN DRAM First time VPN referenced? yes PPN first ever access to VPN Scenario 3 Disk no In DRAM or on Disk? DRAM on Disk PPN Scenario 2 Swap-in from disk to DRAM (“page fault”) Scenario 1 Disk DRAM In DRAM PPN PPN Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Overhead of Virtual Memory Program counter is a virtual address Each instruction fetch requires address translation Loads and stores generate virtual addresses Each load and store requires address translation For every instruction fetch, load, and store, call O/S to search the page table??? This would have unacceptable performance The O/S address translation function takes 10s to 100s of instructions and several memory accesses (to access page table). Exact overhead depends on the scenario, page table organization, etc. There has to be a better way… Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Translation Lookaside Buffer (TLB) The TLB is a small cache of recently used address translations TLB is defined in the ISA, because software and hardware collaborate w.r.t. the TLB Hardware role Hardware provides and accesses the TLB Provides: TLB is a hardware table Accesses: Hardware searches the TLB for desired translation. If TLB doesn’t have translation, hardware calls the O/S. Software role Software manages the TLB When there is a TLB miss, O/S is responsible for handling the miss. Once it has a translation, it writes the translation into the TLB, at a location of its choosing. Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Virtual-to-Physical Address Translation, with TLB translate virtual address to physical address A running program TLB Access L1 I/D caches (entry point to memory hierarchy) CPU is executing application virtual address physical address 1 cycle TLB miss exception TLB-write instruction puts translation into TLB O/S Page Table for running program CPU is executing O/S TLB miss exception handler 10s-100s of cycles Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg TLB Organization TLB organization Can be direct-mapped, set-associative, or fully-associative Many modern RISC ISAs define it to be fully-associative since software manages it Unified versus split TLBs Unified: One TLB for both instruction and data address translation Split: Separate TLBs for instruction and data address translation I-TLB: TLB for instruction address translation (program counter). I-TLB sits alongside L1 I-cache. D-TLB: TLB for data address translation (loads and stores). D-TLB sits alongside L1 D-cache. Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Using the TLB for translation Example: Fully-associative TLB X valid bit virtual page number physical page number =? VPN PPN v =? VPN PPN v =? VPN PPN v X’ X =? VPN PPN v X’ Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg TLB increases hit time Without Virtual Memory With Virtual Memory virtual address address TLB 1 cycle L1 Cache 1 cycle physical address L1 Cache 1 cycle Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Using the TLB for translation: A closer look virtual page number page offset 31 12 11 TLB tag block offset index physical page number virtual address Ex: page size = 4KB # page offset bits = log2(4KB) = 12 physical address This is how cache interprets the physical address Observation: What if index bits were entirely contained in page offset bits? Then first part of cache access (indexing) would not wait on TLB Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Accessing TLB and Cache in Parallel data array word select tag array =? virtual page number page offset 31 12 11 TLB tag physical page number index block offset Cache hit time reduces from two cycles to one! Because cache can now be indexed in parallel with TLB access (only the final tag match uses the output from TLB) But some constraints... Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Constraint: Size of 1 cache way Constraint for “physically-indexed cache with parallel cache/TLB access” Index and block offset bits contained within page offset bits Therefore: Total amount of storage in 1 way of the cache should not exceed page size block size set Way 1 Way 2 Way N # sets N-way set-assoc. cache … Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Page size / associativity tradeoff From previous slide: Cache size equation: Therefore: Example: MC88110 Page size = 4KB I$, D$ both: 8KB 2-way set-associative (8KB/4KB) = 2 ways Example: VAX series Page size = 512B For a 16KB cache, need assoc. = (16KB / 512B) = 32-way set-associative! Moral: sometimes associativity is thrust upon you Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg Backup Slides The following slides are for interest only Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Notation on following slide X: Virtual page number to be translated into a physical page number X’: Physical page number corresponding to X V: Virtual page number of a victim page struct page_table_entry { bool resident; // ‘true’ if in DRAM, ‘false’ if on disk (swap space) unsigned int ppn; // if resident==true, this is physical page number disk_loc_type disk_loc; // if resident==false, this is location on disk } // PT is a hash table (unbounded array) of page table entries. // Key (index) of the hash table is virtual page number. If true: X is in DRAM, at PT[X].ppn. Thus X’ = PT[X].ppn. PT[X].resident If false: X is not in DRAM. PT[X].ppn is bogus, PT[X].disk_loc indicates disk location. Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

Virtual-to-Physical Address Translation X Is X in PT? Is there a free DRAM page? SwapOut(V) PT[V].resident = false PT[V].disk_loc = … PT[X].resident = true PT[X].ppn = PT[V].ppn no no first ever access to X PT[X].resident = true PT[X].ppn = freelist.pop() yes yes SwapOut(V) PT[V].resident = false PT[V].disk_loc = … PT[X].resident = true PT[X].ppn = PT[V].ppn SwapIn(X) PT[X].resident Is there a free DRAM page? false no Swap-in X from disk to DRAM (“page fault”) PT[X].resident = true PT[X].ppn = freelist.pop() SwapIn(X) yes true X’ = PT[X].ppn X’ Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg