1. 國立台灣大學 資訊工程學系 薛智文 cwhsueh@csie.ntu.edu.tw http://www.csie.ntu.edu.tw/~cwhsueh/ 101 Spring, March 15, Fri 678, DTH 104 前瞻資訊科技 (II) - 虛擬化 (1) -Virtualization(V12N)

2. 資工系網媒所 NEWS 實驗室 Steve Jobs (Apple, 1955-2011) Stay hunger, stay foolish. Dennis Ritchie (C language, 1941-2011) Skype eBay (4.1B USD, 2005)  Microsoft (8.5B USD, 2011) Linux (Linus Torvalds, 1991) Android (Danger, 2003  Google, 2005, 34 eng.) Meego (Intel Samsung, Feb 2010 ) Tizen (Intel Samsung [Nokia], Sep 2011) Windows 8 (Microsoft, nVidia 2011) IOS 5 (Apple, 2011) Firefox OS (Mozilla, 2012) MSN (Microsoft, 2013) Ubuntu Touch (Canonical, 2013) Android (85 eng., 2013), Andy Rubin (Today), Asus NB(200+ eng.) iPhone, iPad, HTC One, SamSung Galaxy S4 (Today) Preface /411

3. 資工系網媒所 NEWS 實驗室 Introduction What is virtualization? Cloud? Why is virtualization difficult? How to virtualize? Case Study Mobile Virtualization Inline Emulation Domain 1 Q&A Outline /412

4. 資工系網媒所 NEWS 實驗室 Virtual address Virtual assistant Virtual class Virtual circuit Virtual community Virtual Data Center Virtual device Virtual disk Virtual host Virtual keyboard Virtual machine Virtual market Virtual memory Virtual money Virtual Private Network Virtual reality … What is Virtualization ? Etc. Virtualization Running Applications (x-platform) Running Applications (x-platform) Security Sharing Hardware Resource Sharing Hardware Resource Fully Utilizing Hardware Fully Utilizing Hardware The creation of a virtual version of something. /413

5. 資工系網媒所 NEWS 實驗室 Virtual Assistant/Secretary S/he screens my email. She checks my main email accounts, handles what she can, and “redirects” the messages that require my personal attention to my private account. She has reduced my email load by 90 percent. She books my travel. She handles all the details, including airline reservations, hotels, car rental, etc. She sets up a trip in TripIt, so I have everything I need in one place. She makes calls on my behalf. She makes appointments (both personal and professional), confirms my appointments, checks my voice mail, and follows up as needed. She manages my calendar. Almost nothing gets on my calendar unless it passes through her first. We have agreed together that I will only accept appointments on two afternoons a week, and she works to stay within those boundaries. She handles other projects as needed. I continue to turn over more and more to her. For example, she recently screened all the people who had applied to be a community leader on my site. She and my manager, Joy, ended up picking the final ten I appointed. File my files. /414

6. 資工系網媒所 NEWS 實驗室 Types of Virtualization Hardware/platform virtualization Desktop virtualization Software virtualization OS-level, Workspace, Application Storage virtualization Data virtualization Database virtualization Network virtualization /415

7. 資工系網媒所 NEWS 實驗室 How fast can virtualization achieve? What kinds of applications can there be? What problems it might incur? Technical Security Business Politics … Homework: Turn in a 3-5 page report answering any of the above or related questions, what problems you solve? How? 1-3 members per group, will be posted on course wiki. Q&A in the last hour of class. Big Questions for Virtualization /416

8. 資工系網媒所 NEWS 實驗室 假若真時真亦假 /417 System type Platform VirtualReal Data Virtual simulationevaluation Real emulationimplementation

9. 資工系網媒所 NEWS 實驗室 /418

10. 資工系網媒所 NEWS 實驗室 Why Virtualization is Difficult? OS is moved to ringr1/ring3 On x86 Some instructions Sensitive Instructions Cannot be trapped 0/1/3 Ring, e.g. x86_32 0/3/3 Ring, e.g. x86_64, ARM OS Critical Instructions Instructions Sensitive Register Instructions SGDT, SIDT, SLDT SMSW PUSHF(D), POPF(D) Protected System Instructions LAR, LSL, VERR, VERW PUSH, POP CALL, JMP, INT, RET STR MOV /419

11. 資工系網媒所 NEWS 實驗室 Hardware Hypervisor, e.g. Xen VM 0 VM 1 VM N … Virtual Machine Monitor (VMM) Hypervisor Hardware Hosted VMM, e.g. VMware VM 0 VM 1 VM N … Host Operating System Type I - HypervisorType II – Hosted VMM VM : Virtual Machine, Guest OS + Virtual Devices /4110

12. 資工系網媒所 NEWS 實驗室 Software Execution Modes in Virtualization Environment ModePhysical modeVirtual mode Description HypervisorPrivilegedN/A For executing the hypervisor only. KernelUserPrivileged For executing the kernel of a virtual machine. User For executing user processes of a guest OS. /4111

13. 資工系網媒所 NEWS 實驗室 According to Popek and Goldberg † in 1974, Virtual machines can be constructed for a platform if Sensitive Instructions might change the state of system resources Privileged Instructions must be executed with sufficient privilege The First Challenge of Virtualization Virtualizable † G. J. Popek and R. P. Goldberg, “Formal requirements for virtualizable third generation architectures,” Commun. ACM, vol. 17, no. 7, pp. 412–421, Jul. 1974. /4112

14. 資工系網媒所 NEWS 實驗室  Binary translation  Hypercall How to Virtualize ? Full Virtualizatio n Para Virtualizatio n Hardware Assisted Virtualization Intel VT-x & AMD SVM Trap and emulate /4113

15. 資工系網媒所 NEWS 實驗室 Case Study 1.Mobile Virtualization 2.Inline Emulation † 3.Domain 1 with Insyde Inc. /4114 † Yuan-Cheng Lee, Chih-Wen Hsueh, and Rong-Guey Chang, "Inline Emulation: An Optimization Technique for Virtualization on Embedded Systems," Proc. of the 17th International Conference on Real-Time and Embedded Computing Systems and Applications (RTCSA'11), Toyama, Japan, August 2011.

16. 資工系網媒所 NEWS 實驗室 /4115 Mobile Virtualization 90+% performance on PC  embedded multicore systems To run multiple OSes on a mobile phone… iPhone+Android is possible! Break the limitation of OSes!

17. 資工系網媒所 NEWS 實驗室 Main Memory Hardware Assisted Paging Primary MMU Extended MMU Page Table Extended Page Table VAGPAMPA VA: virtual address GPA: guest physical address MPA: machine physical address MMU: memory management unit /4116

18. 資工系網媒所 NEWS 實驗室 Main Memory 0-miss Page Translation Page Table VAGPAMPA bTLB Primary MMU VA: virtual address GPA: guest physical address MPA: machine physical address MMU: memory management unit bTLB: buddy TLB † Yuan-Cheng Lee, Chih-Wen Hsueh, "An Optimized Page Translation for Mobile Virtualization," to appear in Proc. of 50 th Design Automation Conference (DAC), Austin, TX, USA, June 2013. (Top conference) /4117 50+% speedup

19. 資工系網媒所 NEWS 實驗室 The First Challenge of Virtualization Idea of Inline Emulation Design of Inline Emulation Evaluation and Analysis Conclusions Inline Emulation /4118

20. 資工系網媒所 NEWS 實驗室 Related Work Secure Xen on ARM (Samsung) It proved virtualization is possible for ARM platform. The PENAR project (University of Applied Sciences, Western Switzerland) It integrated the source trees of Xen, RTLinux, and Linux for ARM. OKL4 (Open Kernel Labs) A hypervisor which adopts microkernel architecture for embedded systems /4119

21. 資工系網媒所 NEWS 實驗室 Issues on Virtualization for ARM The most critical issue is: Example MOVS PC, LR // move the value in link register to PC It will cause unpredictable behavior when executed in user mode. SPSR: Saved Program Status Register CPSR: Current Program Status Register /4120

22. 資工系網媒所 NEWS 實驗室 The Problematic Instructions (1/3) Type I Instructions which executed in user mode will cause undefined instruction (UDI) exception We call them Canonical Privileged Instructions. Example MCRp15, 0, r0, c2, c0, 0 Move r0 to c2 and c0 in coprocessor specified by p15 for operation according to option 0 and 0 Operand-dependent operation /4121

23. 資工系網媒所 NEWS 實驗室 The Problematic Instructions (2/3) Type II Instructions which executed in user mode will have no effect Example MSRcpsr_c, #0xD3 Switch to privileged mode and disable interrupt NZCVQ--J GE[3:0]--EAIFTM[4:0] 31 0 Execution Flags Exception Mask Execution Mode Program Status Register (PSR) /4122

24. 資工系網媒所 NEWS 實驗室 The Problematic Instructions (3/3) Type III Instructions which executed in user mode will cause unpredictable behaviors Example MOVS PC, LR /4123

25. 資工系網媒所 NEWS 實驗室 Solutions Complexity Binary translation Hypercall Inline emulation DesignHighLow ImplementationMediumHighLow RuntimeHighMediumLow Counterpart (in programming languages) Virtual functionNormal functionInline function /4124

26. 資工系網媒所 NEWS 實驗室 For the ARM architecture, the instruction (TYPE III) MOVS PC, LR Changes the program counter and switches to user mode. However, it causes unpredictable behavior when executed in user mode. Therefore, it is a sensitive instruction but not a privileged instruction. The First Challenge of Virtualization Example /4125

27. 資工系網媒所 NEWS 實驗室 Dynamic Binary Translation The First Challenge of Virtualization Solutions (1/2) BL TLB_FLUSH_DENTRY … TLB_FLUSH_DENTRY: MCR p15, 0, R0, C8, C6, 1 MOV PC, LR … BL TLB_FLUSH_DENTRY_NEW … TLB_FLUSH_DENTRY: MCR p15, 0, R0, C8, C6, 1 MOV PC, LR … TLB_FLUSH_DENTRY_NEW: MOV R1, R0 MOV R0, #CMD_FLUSH_DENTRY SWI #HYPER_CALL_TLB Translation Basic Block /4126

28. 資工系網媒所 NEWS 實驗室 Virtualization APIs – hypercalls The First Challenge of Virtualization Solutions (2/2) BL TLB_FLUSH_DENTRY … TLB_FLUSH_DENTRY: MOV R1, R0 MOV R0, #CMD_FLUSH_DENTRY SWI #HYPER_CALL_TLB … Restore User Context & PC SWI Handler Hypercall Handler …… LDRR1, [SP, #4] MCRp15, 0, R1, C8, C6, 1 /* In Hypervisor */ /* In Guest OS */ /4127

29. 資工系網媒所 NEWS 實驗室 Hypercall Guest OS Hypervisor SWI Handler Hypercalls Software Interrupt Hyper Call Handler reschedule? No Yes context switch /4128

30. 資工系網媒所 NEWS 實驗室 Idea of Inline Emulation The Original Instruction Hypercall MOVR0, VIRT_ADDR MCRp15, 0, R0, C8, C6, 1 MOVR0, #CMD_FLUSH_DENTRY MOVR1, VIRT_ADDR SWI#HYPER_CALL_TLB LDRR1, [SP, #4] MCRp15, 0, R1, C8, C6, 1 Restore User Context & PC Hypercall Handler …… Guest OS Inline Emulation Restore PC Inline Emulation Handler …… Guest OS MOVR0, VIRT_ADDR MCRp15, 0, R0, C8, C6, 1 /* restore user context */ LDMIASP, [R0 – R14] MCRp15, 0, R0, C8, C6, 1 /4129

31. 資工系網媒所 NEWS 實驗室 Inline Emulation Guest OS Hypervisor SWI Handler Inline Emulation Canonical Privileged Instructions (TYPE I) Canonical Privileged Instructions (TYPE I) UDI Exception return to guest Hypercalls Software Interrupt Hyper Call Handler reschedule? No Yes context switch UDI Handler /4130

32. 資工系網媒所 NEWS 實驗室 Design of Inline Emulation The Main Handler A handler for the instruction is found No handler for the instruction was found /4131

33. 資工系網媒所 NEWS 實驗室 The Issue of Finding an Inline Emulation Handler It is hard to find a simple hash function. Because the encoding of ARM instructions is complicated. Instead, we can construct an efficient search table. Because there are a few frequently used instructions. InstructionRatio (%) mcr p15, 0, Rd, c3, c0, 058.44 mcr p15, 0, Rd, c7, c14, 139.73 mcr p15, 0, Rd, c8, c5, 10.49 mcr p15, 0, Rd, c8, c6, 10.49 mcr p15, 0, Rd, c7, c10, 40.24 mcr p15, 0, Rd, c2, c0, 00.23 mcr p15, 0, Rd, c7, c5, 00.11 mcr p15, 0, Rd, c8, c5, 00.08 mcr p15, 0, Rd, c8, c6, 00.08 mrc p15, 0, Rd, c7, c14, 30.11 Others<0.01 /4132

34. 資工系網媒所 NEWS 實驗室 Example of Mto1 Search Table Encoding of MCR instruction Syntax: MCR{cond} cp, op1, Rd, CRn, CRm, op2 maskvaluehandlerSet 0x0F1F0F100x0E130F10handler_CR3MCR 15, op1, Rd, c3, CRm, op2 0x0F1C0F100x0E100F10handler_CR02MCR 15, op1, Rd, {c0 - c2}, CRm, op2 0x0F100F100x0E100F10handler_CRXMCR 15, op1, Rd, {c4 - c15}, CRm, op2 …… 0x00000000 End of Table cond1110op10CRnRdcpop21CRm 310 /4133

35. 資工系網媒所 NEWS 實驗室 Design of Inline Emulation Dynamic Inline Emulation (DIE) Handler Self-modifying inlining the instruction flushing caches /4134

36. 資工系網媒所 NEWS 實驗室 Design of Inline Emulation Static Inline Emulation (SIE) Handler /* data synchronization barrier */ executing the hard-coded instructions restoring user context & PC /4135

37. 資工系網媒所 NEWS 實驗室 EmulatorAndroid emulator (ARMv5) Memory12MB for the hypervisor 32MB for the guest OS HypervisorXen 4.0.1 for ARMv5 Guest OSLinux 2.6.29-Goldfish CompilationUsing GCC with debug (-g) flag Evaluation and Analysis The Experiment Environment /4136

38. 資工系網媒所 NEWS 實驗室 Evaluation and Analysis The Distribution of Emulated Instructions InstructionCRn, CRm, op2Ratio(%) MCRc3, c0, 058.44 c7, c14, 139.73 c8, c5, 10.49 c8, c6, 10.49 c7, c10, 40.24 c2, c0, 00.23 c7, c5, 00.11 c8, c5, 00.08 c8, c6, 00.08 MRCc7, c14, 30.11 Others<0.01 More than 98% /4137

39. 資工系網媒所 NEWS 實驗室 Evaluation and Analysis The Micro-Level Analysis (1/2) Operation - Invalidating TLB Mode (instructions)Improvement PV/IE (%) USERUNDSWITotal A single entry ( DIE handler ) PV 13.000.00305.97318.97 613.39 IE 3.0049.000.0052.00 The entire TLB ( SIE handler ) PV 11.000.00305.80316.80 704.01 IE 3.0042.000.0045.00 /4138

40. 資工系網媒所 NEWS 實驗室 Evaluation and Analysis The Micro-Level Analysis (2/2) Instruction Mode (instructions)Improvement PV/IE (%) USERUNDSWITotal MCR p15, 0, Rd, c3, c0, 0 (DIE handler) PV 9.000.00203.29212.29 424.57 IE 3.0047.000.0050.00 MCR p15, 0, Rd, c7, c14, 1 (DIE handler) PV 13.000.00304.50317.50 566.94 IE 3.0053.000.0056.00 Inline emulation can achieve at least 4.24X performance of hypercalls in most cases (about 98%). /4139

41. 資工系網媒所 NEWS 實驗室 Evaluation and Analysis The Macro-Level Analysis Data Processing Data Transfer BranchSoftware Interrupt Coprocessor and Other Total Paravirtualization (instructions) 89.22M91.28M27.08M485604.79M212.42M Inline Emulation (instructions) 89.04M90.66M26.93M336584.93M211.59M (PV – IE) / PV (%) 0.200.680.5330.69-2.720.39 /4140

42. 資工系網媒所 NEWS 實驗室 Inline emulation : Reduces the efforts to port guest operating systems Increases the handling of sensitive instructions (4-7x) Increases the overall system performance ( 0.39% ) Future work Optimization for memory virtualization Much higher the overall speedup is possible. Conclusions /4141