CS162 Operating Systems and Systems Programming Lecture 12 Address Translation October 10th, 2016 Prof. Anthony D. Joseph http://cs162.eecs.Berkeley.edu

Review: Simple Example – Base and Bounds (CRAY-1) DRAM <? + Base Bound (Limit) CPU Virtual Address Physical No: Error! Could use base/bounds for dynamic address translation – translation happens at execution: Alter address of every load/store by adding “base” Generate error if address bigger than limit This gives program the illusion that it is running on its own dedicated machine, with memory starting at 0 Program gets continuous region of memory Addresses within program do not have to be relocated when program placed in different region of DRAM

Review: Issues with Simple B&B Method process 6 process 6 process 5 OS process 6 process 5 OS process 9 process 6 process 9 OS process 10 process 11 process 5 process 2 OS Fragmentation problem over time Not every process is same size → memory becomes fragmented Missing support for sparse address space Would like to have multiple chunks/program (Code, Data, Stack) Hard to do inter-process sharing Want to share code segments when possible Want to share memory between processes Helped by providing multiple segments per process

More Flexible Segmentation 1 3 2 4 user view of memory space physical Logical View: multiple separate segments Typical: Code, Data, Stack Others: memory sharing, etc Each segment is given region of contiguous memory Has a base and limit Can reside anywhere in physical memory

Implementation of Multi-Segment Model offset Offset Seg # Virtual Address > Error + Physical Address Base0 Limit0 V Base1 Limit1 Base2 Limit2 Base3 Limit3 N Base4 Limit4 Base5 Limit5 Base6 Limit6 Base7 Limit7 Base2 Limit2 V Check Valid Access Error Segment map resides in processor Segment number mapped into base/limit pair Base added to offset to generate physical address Error check catches offset out of range As many chunks of physical memory as entries Segment addressed by portion of virtual address However, could be included in instruction instead: x86 Example: mov [es:bx],ax. What is “V/N” (valid / not valid)? Can mark segments as invalid; requires check as well

Intel x86 Special Registers Typical Segment Register Current Priority is RPL Of Code Segment (CS)

Example: Four Segments (16 bit addresses) Seg ID # Base Limit 0 (code) 0x4000 0x0800 1 (data) 0x4800 0x1400 2 (shared) 0xF000 0x1000 3 (stack) 0x0000 0x3000 Offset Seg 14 13 15 Virtual Address Format SegID = 0 0x0000 0x4800 0x5C00 0x4000 0xF000 Physical Address Space 0x4000 0x0000 0x8000 0xC000 Virtual Address Space Might be shared SegID = 1 Space for Other Apps Shared with Other Apps

Example of Segment Translation 0x240 main: la $a0, varx 0x244 jal strlen … … 0x360 strlen: li $v0, 0 ;count 0x364 loop: lb $t0, ($a0) 0x368 beq $r0,$t1, done 0x4050 varx dw 0x314159 Seg ID # Base Limit 0 (code) 0x4000 0x0800 1 (data) 0x4800 0x1400 2 (shared) 0xF000 0x1000 3 (stack) 0x0000 0x3000 Let’s simulate a bit of this code to see what happens (PC=0x240): Fetch 0x240. Virtual segment #? 0; Offset? 0x240 Physical address? Base=0x4000, so physical addr=0x4240 Fetch instruction at 0x4240. Get “la $a0, varx” Move 0x4050  $a0, Move PC+4PC Fetch 0x244. Translated to Physical=0x4244. Get “jal strlen” Move 0x0248  $ra (return address!), Move 0x0360  PC Fetch 0x360. Translated to Physical=0x4360. Get “li $v0,0” Move 0x0000  $v0, Move PC+4PC Fetch 0x364. Translated to Physical=0x4364. Get “lb $t0,($a0)” Since $a0 is 0x4050, try to load byte from 0x4050 Translate 0x4050. Virtual segment #? 1; Offset? 0x50 Physical address? Base=0x4800, Physical addr = 0x4850, Load Byte from 0x4850$t0, Move PC+4PC

Observations about Segmentation Virtual address space has holes Segmentation efficient for sparse address spaces A correct program should never address gaps (except as mentioned in moment) If it does, trap to kernel and dump core When it is OK to address outside valid range: This is how the stack and heap are allowed to grow For instance, stack takes fault, system automatically increases size of stack Need protection mode in segment table For example, code segment would be read-only Data and stack would be read-write (stores allowed) Shared segment could be read-only or read-write What must be saved/restored on context switch? Segment table stored in CPU, not in memory (small) Might store all of processes memory onto disk when switched (called “swapping”)

Problems with Segmentation Must fit variable-sized chunks into physical memory May move processes multiple times to fit everything Limited options for swapping to disk Fragmentation: wasted space External: free gaps between allocated chunks Internal: don’t need all memory within allocated chunks

Recall: General Address Translation Data 2 Stack 1 Heap 1 OS heap & Stacks Code 1 Stack 2 Data 1 Heap 2 Code 2 OS code OS data Code Data Heap Stack Code Data Heap Stack Prog 1 Virtual Address Space 1 Prog 2 Virtual Address Space 2 Translation Map 1 Translation Map 2 Physical Address Space

Paging: Physical Memory in Fixed Size Chunks Solution to fragmentation from segments? Allocate physical memory in fixed size chunks (“pages”) Every chunk of physical memory is equivalent Can use simple vector of bits to handle allocation: 00110001110001101 … 110010 Each bit represents page of physical memory 1  allocated, 0  free Should pages be as big as our previous segments? No: Can lead to lots of internal fragmentation Typically have small pages (1K-16K) Consequently: need multiple pages/segment

How to Implement Paging? Offset Virtual Page # Virtual Address: Physical Address Offset PageTablePtr page #0 page #2 page #3 page #4 page #5 V,R page #1 V,R,W N Physical Page # page #1 V,R Access Error > PageTableSize Check Perm Access Error Page Table (One per process) Resides in physical memory Contains physical page and permission for each virtual page Permissions include: Valid bits, Read, Write, etc Virtual address mapping Offset from Virtual address copied to Physical Address Example: 10 bit offset  1024-byte pages Virtual page # is all remaining bits Example for 32-bits: 32-10 = 22 bits, i.e. 4 million entries Physical page # copied from table into physical address Check Page Table bounds and permissions

Simple Page Table Example Example (4 byte pages) 0000 0000 0x00 a b c d e f g h i j k l 0x00 0x04 0x08 Virtual Memory 0001 0000 i j k l 0x04 0x08 4 3 1 Page Table 2 0000 0100 0000 0100 0000 1100 0x05! 0x06? 0000 1000 e f g h 0x0C 0x09? What is virtual address 0x6? 1|10 = 3|2 = 0xE What is virtual address 0x9? 10|01 = 1|1 = 0x5 0x0E! a b c d 0x10 0000 0110 0000 1110 0000 1001 0000 0101 Physical Memory

What about Sharing? Shared Page This physical page appears in address Offset Virtual Page # Virtual Address (Process A): PageTablePtrA page #0 page #1 page #3 page #4 page #5 V,R page #2 V,R,W N Shared Page page #2 V,R,W PageTablePtrB page #0 page #1 page #2 page #3 page #5 V,R N V,R,W page #4 This physical page appears in address space of both processes page #4 V,R Offset Virtual Page # Virtual Address (Process B):

Administrivia Midterm #1 regrades open until today Mon 10/10 11:59PM Upcoming deadlines (nothing due this week!): Project 2 design doc due Wed 10/19 HW 3 due 11/7

break

Memory Layout for Linux 32-bit http://static.duartes.org/img/blogPosts/linuxFlexibleAddressSpaceLayout.png

Summary: Paging stack stack heap heap data data code code Page Table Virtual memory view Physical memory view 11111 11101 11110 11100 11101 null 11100 null 11011 null 11010 null 11001 null 11000 null 10111 null 10110 null 10101 null 10100 null 10011 null 10010 10000 10001 01111 10000 01110 01111 null 01110 null 01101 null 01100 null 01011 01101 01010 01100 01001 01011 01000 01010 00111 null 00110 null 00101 null 00100 null 00011 00101 00010 00100 00001 00011 00000 00010 1111 1111 1110 1111 stack 1111 0000 stack 1110 0000 1100 0000 heap 1000 0000 heap 0111 000 data data 0101 000 0100 0000 code code 0001 0000 0000 0000 0000 0000 page # offset

Summary: Paging stack stack What happens if stack grows to 1110 0000? Page Table Virtual memory view Physical memory view 11111 11101 11110 11100 11101 null 11100 null 11011 null 11010 null 11001 null 11000 null 10111 null 10110 null 10101 null 10100 null 10011 null 10010 10000 10001 01111 10000 01110 01111 null 01110 null 01101 null 01100 null 01011 01101 01010 01100 01001 01011 01000 01010 00111 null 00110 null 00101 null 00100 null 00011 00101 00010 00100 00001 00011 00000 00010 1111 1111 stack stack 1110 0000 1110 0000 What happens if stack grows to 1110 0000? 1100 0000 heap 1000 0000 heap 0111 000 data data 0101 000 0100 0000 code code 0001 0000 0000 0000 0000 0000 page # offset

Challenge: Table size equal to # of pages in virtual memory! Summary: Paging Page Table Virtual memory view Physical memory view 11111 11101 11110 11100 11101 10111 11100 10110 11011 null 11010 null 11001 null 11000 null 10111 null 10110 null 10101 null 10100 null 10011 null 10010 10000 10001 01111 10000 01110 01111 null 01110 null 01101 null 01100 null 01011 01101 01010 01100 01001 01011 01000 01010 00111 null 00110 null 00101 null 00100 null 00011 00101 00010 00100 00001 00011 00000 00010 1111 1111 stack stack 1110 0000 1110 0000 1100 0000 stack Allocate new pages where room! heap 1000 0000 heap 0111 000 data Challenge: Table size equal to # of pages in virtual memory! data 0101 000 0100 0000 code code 0001 0000 0000 0000 0000 0000 page # offset

Page Table Discussion What needs to be switched on a context switch? Page table pointer and limit Analysis Pros Simple memory allocation Easy to Share Con: What if address space is sparse? E.g., on UNIX, code starts at 0, stack starts at (231-1) With 1K pages, need 2 million page table entries! Con: What if table really big? Not all pages used all the time  would be nice to have working set of page table in memory How about combining paging and segmentation? What if page size is very small? VAX had a 512-byte page size = lots of space for page table entries What if page size is really big? Wastes space inside of page (internal fragmentation)

Fix for sparse address space: The two-level page table Physical Address: Offset Page # 4KB 10 bits 12 bits Virtual Address: Offset P2 index P1 index 4 bytes 4 bytes PageTablePtr Tree of Page Tables Tables fixed size (1024 entries) On context-switch: save single PageTablePtr register Valid bits on Page Table Entries Don’t need every 2nd-level table Even when exist, 2nd-level tables can reside on disk if not in use

Summary: Two-Level Paging Virtual memory view Page Tables (level 2) Physical memory view 1111 1111 stack 11 11101 10 11100 01 10111 00 10110 stack 1110 0000 1111 0000 Page Table (level 1) 1100 0000 stack 111 110 null 101 null 100 011 null 010 001 null 000 11 null 10 10000 01 01111 00 01110 heap 1000 0000 heap 0111 000 11 01101 10 01100 01 01011 00 01010 data data 0101 000 0100 0000 11 00101 10 00100 01 00011 00 00010 code page2 # code 0001 0000 0000 0000 0000 0000 page1 # offset

Summary: Two-Level Paging Virtual memory view Page Tables (level 2) Physical memory view stack 11 11101 10 11100 01 10111 00 10110 stack 1110 0000 Page Table (level 1) stack 111 110 null 101 null 100 011 null 010 001 null 000 11 null 10 10000 01 01111 00 01110 1001 0000 (0x90) heap 1000 0000 (0x80) heap In best case, total size of page tables ≈ number of pages used by program virtual memory. Requires two additional memory access! 11 01101 10 01100 01 01011 00 01010 data data 11 00101 10 00100 01 00011 00 00010 code code 0001 0000 0000 0000

Multi-level Translation: Segments + Pages What about a tree of tables? Lowest level page table  memory still allocated with bitmap Higher levels often segmented Could have any number of levels. Example (top segment): What must be saved/restored on context switch? Contents of top-level segment registers (for this example) Pointer to top-level table (page table) Virtual Address: Offset Page # Seg # Offset Physical Address page #0 page #1 page #3 page #4 page #5 V,R page #2 V,R,W N Physical Page # Base0 Limit0 V Base1 Limit1 Base2 Limit2 Base3 Limit3 N Base4 Limit4 Base5 Limit5 Base6 Limit6 Base7 Limit7 Access Error > page #2 V,R,W Check Permissions Access Error Base2 Limit2 V

What about Sharing (Complete Segment)? Process A Offset Virtual Page # Seg # page #0 page #1 page #2 page #3 page #4 page #5 V,R V,R,W N Shared Segment Base0 Limit0 V Base1 Limit1 Base2 Limit2 Base3 Limit3 N Base4 Limit4 Base5 Limit5 Base6 Limit6 Base7 Limit7 Base0 Limit0 V Base1 Limit1 Base2 Limit2 Base3 Limit3 N Base4 Limit4 Base5 Limit5 Base6 Limit6 Base7 Limit7 Process B Offset Virtual Page # Seg #

Multi-level Translation Analysis Pros: Only need to allocate as many page table entries as we need for application In other wards, sparse address spaces are easy Easy memory allocation Easy Sharing Share at segment or page level (need additional reference counting) Cons: One pointer per page (typically 4K – 16K pages today) Page tables need to be contiguous However, previous example keeps tables to exactly one page in size Two (or more, if >2 levels) lookups per reference Seems very expensive!

What is in a Page Table Entry? What is in a Page Table Entry (or PTE)? Pointer to next-level page table or to actual page Permission bits: valid, read-only, read-write, write-only Example: Intel x86 architecture PTE: Address same format previous slide (10, 10, 12-bit offset) Intermediate page tables called “Directories” P: Present (same as “valid” bit in other architectures) W: Writeable U: User accessible PWT: Page write transparent: external cache write-through PCD: Page cache disabled (page cannot be cached) A: Accessed: page has been accessed recently D: Dirty (PTE only): page has been modified recently L: L=14MB page (directory only). Bottom 22 bits of virtual address serve as offset Page Frame Number (Physical Page Number) Free (OS) L D A PCD PWT U W P 1 2 3 4 5 6 7 8 11-9 31-12

Examples of how to use a PTE How do we use the PTE? Invalid PTE can imply different things: Region of address space is actually invalid or Page/directory is just somewhere else than memory Validity checked first OS can use other (say) 31 bits for location info Usage Example: Demand Paging Keep only active pages in memory Place others on disk and mark their PTEs invalid Usage Example: Copy on Write UNIX fork gives copy of parent address space to child Address spaces disconnected after child created How to do this cheaply? Make copy of parent’s page tables (point at same memory) Mark entries in both sets of page tables as read-only Page fault on write creates two copies Usage Example: Zero Fill On Demand New data pages must carry no information (say be zeroed) Mark PTEs as invalid; page fault on use gets zeroed page Often, OS creates zeroed pages in background

break

X86_64: Four-level page table! 9 bits 9 bits 9 bits 9 bits 12 bits 48-bit Virtual Address: Virtual P1 index Virtual P2 index Virtual P3 index Virtual P4 index Offset PageTablePtr 8 bytes 4096-byte pages (12 bit offset) Page tables also 4k bytes (pageable) Physical Address: (40-50 bits) 12bit Offset Physical Page #

No! IA64: 64-bit addresses: Six-level page table?!? Too slow 64-bit Virtual Address: 7 bits 9 bits 9 bits 9 bits 9 bits 9 bits 12 bits Virtual P1 index Virtual P2 index Virtual P3 index Virtual P4 index Virtual P5 index Virtual P6 index Offset No! Too slow Too many almost-empty tables Only really use last two levels (1 GB)

Inverted Page Table With all previous examples (“Forward Page Tables”) Size of page table is at least as large as amount of virtual memory allocated to processes Physical memory may be much less Much of process space may be out on disk or not in use Answer: use a hash table Called an “Inverted Page Table” Size is independent of virtual address space Directly related to amount of physical memory Very attractive option for 64-bit address spaces Cons: Complexity of managing hash changes Often in hardware! Offset Virtual Page # Hash Table Physical

IA64: Inverse Page Table (IPT) Idea: index page table by physical pages instead of VM VMpage0, pid 0 VMpage2, pid 0 VMpage1, pid 0 VMpage3, pid 0 0x0000 0x1000 0x2000 0x3000 0x4000 0x5000 0x6000 0x7000 VMpage0 VMpage1 VMpage2 VMpage3 pid 0 VMpage0 pid 1 … VMpage2 VMpage1 xx free pid 2 VMpage3 0x0 0x1 Process id 0 Virtual memory 0x2 0x3 0x4 0x5 0x6 0x7 Physical memory in 4kB pages Page numbers in red Inverse Page Table

IPT address translation Need an associative map from VM page to IPT address: Use a hash map Process 0 virtual address Physical address VMpage2 (52b) Offset (12b) 0x3 Offset (12b) VMpage0, pid 0 VMpage2, pid 0 VMpage1, pid 0 VMpage3, pid 0 0x0000 0x1000 0x2000 0x3000 0x4000 0x5000 0x6000 0x7000 pid 0 VMpage0 pid 1 VMpage1 VMpage2 xx free pid 2 VMpage3 0x0 0x1 0x2 0x3 Hash VM page # 0x4 0x5 0x6 0x7 Inverse Page Table

Summary: Inverted Table Virtual memory view stack Physical memory view data code heap 0000 0000 0001 0000 0101 0000 0111 0000 1110 0000 11101 11100 10111 10110 10000 01111 01110 01101 01100 01011 01010 00101 00100 00011 00010 1011 0000 Inverted Table Hash(procID & virt. page #) = phys. page # 1111 1111 stack 1110 0000 1100 0000 h(11111) = h(11110) = h(11101) = h(11100) = h(10010)= h(10001)= h(10000)= h(01011)= h(01010)= h(01001)= h(01000)= h(00011)= h(00010)= h(00001)= h(00000)= heap 1000 0000 Total size of page table ≈ number of pages used by program in physical memory. Hash more complex data 0100 0000 code 0000 0000 page # offset

Address Translation Comparison Advantages Disadvantages Simple Segmentation Fast context switching: Segment mapping maintained by CPU External fragmentation Paging (single-level page) No external fragmentation, fast easy allocation Large table size ~ virtual memory Internal fragmentation Paged segmentation Table size ~ # of pages in virtual memory, fast easy allocation Multiple memory references per page access Two-level pages Inverted Table Table size ~ # of pages in physical memory Hash function more complex

Summary Segment Mapping Page Tables Multi-Level Tables Segment registers within processor Segment ID associated with each access Often comes from portion of virtual address Can come from bits in instruction instead (x86) Each segment contains base and limit information Offset (rest of address) adjusted by adding base Page Tables Memory divided into fixed-sized chunks of memory Virtual page number from virtual address mapped through page table to physical page number Offset of virtual address same as physical address Large page tables can be placed into virtual memory Multi-Level Tables Virtual address mapped to series of tables Permit sparse population of address space Inverted page table Size of page table related to physical memory size