1. © 2010 VMware Inc. All rights reserved Mobile I/O virtualization Harvey Tuch, Staff Engineer, Mobile Virtualization Platform January 25 th 2012 Sponsored by MIT and VMware Academic Programs VMware: www.vmware.com VMware Labs: labs.vmware.com

2. 2 Agenda Smartphone hardware I/O virtualization techniques Examples from MVP hypervisor Storage virtualization GPS virtualization Telephony virtualization

3. 3 Smartphone hardware

4. 4 Computer System Organization NIC LA N CPU MMU Memory Controller Local Bus Interface High-Speed I/O Bus ControllerBridge Frame Buffer Low-Speed I/O Bus USBCD-ROM

5. 5 Evolution of mobile phone to Mobile Personal Computer

6. 6 Recent smartphone specs sample (source: pdadb.net ) 6 Copyright ® VMware, Inc. All Rights Reserved.

7. 7 Hardware trends Example smartphone circa 2012 > 1GHz dual or quad core ARM Cortex-A9/A15 or Snapdragon 1GB SDRAM 512MB internal NAND Flash 32GB microSD Integrated GPS, > 10MP camera, 3G/4G (LTE), 802.11, Bluetooth 480x800 OLED Ice Cream Sandwich Android OS Tablets, smartbooks, etc.

8. 8 Smartphone physical hardware Processor core – ARM Power efficiency, battery life E.g. 8 DMIPS/mW (Cortex-A9) ARMv7 ISA Devices – highly integrated SoC organization Plethora of devices (IP blocks) Limited standardization Storage – solid state NAND

9. 9 Agenda Smartphone hardware I/O virtualization techniques Examples from MVP hypervisor Storage virtualization GPS virtualization Telephony virtualization

10. 10 I/O virtualization techniques

11. 11 Device virtualization goals Present to guest a virtual platform (set of devices) Guest applications, middleware, OS expect standard set of mobile devices E.g. touchscreen, display, storage, GPS, Bluetooth, WiFi, GSM, GPU, cameras, accelerometers, audio, keyboard, etc. Multiplex physical hardware between VMs Securely sharing physical hardware across domains Efficiency Fidelity Portability Interposition (checkpointing, live migration) Minimizing engineering effort (cost)

12. 12 Device virtualization techniques Device emulation VMM emulates registers and memory map of physical HW devices Paravirtualization VMM provides optimized high-level API for guest devices Pass-thru VMM provides guest direct access to physical HW devices

13. 13 Device virtualization techniques – device emulation VMM emulates registers and memory map of physical HW devices Leverage existing guest OS driver support Interposable Emulation overhead VMM implementation cost Hypervisor Guest Operating System ACME UART driver ACME UART device model Physical FOOBAR UART Write FIFO a Read status register CSR Write FIFO b ….. Write FIFO a Write FIFO b Read status register FSR …..

14. 14 Device virtualization techniques - paravirtualization VMM provides optimized high-level API for guest devices Higher performance possible Simplified VMM Interposable Requires custom drivers/libraries per guest Hypervisor Guest Operating System PV UART driver PV UART device model Physical FOOBAR UART Write shared buffer abcdefghijklmno… ….. Write FIFO a Write FIFO b Read status register FSR …..

15. 15 Device virtualization techniques – pass-thru VMM provides guest direct access to physical HW devices Highest performance, even simpler VMM I/O MMU required for safety (DMA danger) Multiplexing HW devices requires extra HW support Interposition hard (hybrid models separate control/data paths) Hypervisor Guest Operating System ACME UART driver Physical ACME UART Write FIFO a Read status register Write FIFO b …..

16. 16 Bare-metal (Type-1) hypervisor architecture Execute directly on physical HW Preferred architecture for high-end servers E.g. VMware ESX, Xen, Microsoft Hyper-V

17. 17 Hosted (Type-2) hypervisor architecture Executes alongside existing host OS, e.g. Mac OS X, Linux, Windows Leverage host device drivers and resources management E.g. VMware Workstation/Fusion, KVM, Parallels Desktop

18. 18 VMware MVP system architecture Personal applications run natively Corporate applications run in a virtual machine 18 Copyright ® VMware, Inc. All Rights Reserved.

19. 19 MVP as a hosted (Type-2) hypervisor Hosted hypervisor benefits: Tackle SoC diversity Reduced TTM Zero virtualization overhead for personal phone. Almost all components can be downloaded OTA (app store). 19 Copyright ® VMware, Inc. All Rights Reserved.

20. 20 Device virtualization stack Frontend (RHS) Guest OS – VMM interface for virtual devices Backend (LHS) Hypervisor implementation of virtual device semantics on host

21. 21 Frontend virtualization – where to slice? Virtualization Layer R: 137 G: 203 B: 223 Operating System Application Libraries, middleware R: 192 G: 192 B: 192 R: 217 G: 84 B: 30 R: 248 G: 152 B: 29 Device emulation Device level PV Library level PV Application level PV

22. 22 Backend implementation Host user space (application level) Translate guest device I/Os to standard kernel/library calls (e.g. POSIX) E.g. write a to /dev/ttyS0 for UART transmit Portable, simple, language independent System call overhead, latency Host kernel Translate guest device I/Os to kernel level function calls Lowest overhead, direct access to device subsystem Kernel dependent, modularity issues, increased TCB Hardware (pass-thru)

23. 23 Agenda Smartphone hardware I/O virtualization techniques Examples from MVP hypervisor Storage virtualization GPS virtualization Telephony virtualization

24. 24 Storage virtualization

25. 25 Smartphone storage devices Internal storage NAND flash devices Software Flash Translation Layer (FTL) Limited size (256MB – several GB) Kernel, application code, libraries, middleware External storage microSD cards Hardware FTL Up to 32GB today (2TB future) Economics of semiconductor scaling Optimized for cost, media workloads Application data (and some code)

26. 26 Storage virtualization stack

27. 27 VM image storage on SD cards Why? Storage footprint of guest may be several GB (including checkpoint images) Challenges Performance Reliability Security

28. 28 VM image storage on SD cards - performance FTL optimized for cost, media workloads Poor non-sequential I/O performance VM workloads exhibit far more non-sequential I/O than MP3 streaming, photos, videos, etc.

29. 29 VM image storage on SD cards - performance.

30. 30 VM image storage on SD cards - performance.

31. 31 VM image storage on SD cards - performance.

32. 32 VM image storage on SD cards – other challenges Reliability Dropped phone, whoops Battery dies Host software crashes (in particular at kernel level) Security SD cards are FAT formatted for compatibility No access controls Malicious host apps can read/write VM images

33. 33 Log structured filesystems (LSF) An excursion back in time to… 1991… Slow seek latency on mechnical disks, CPUs getting faster Non-sequential file writes expensive Solution: every file update gets appended to a sequential log Maintain meta-data to later locate the most recent block for a file in the log Reads mostly hit in the page cache, dont worry about non-sequentiality Mendel Rosenblum and John K. Ousterhout. The design and implementation of a log-structured file system. In ACM Symposium on Operating System Principles, 1991. Benefits beyond performance… Meta-data only updated after new data is on disk Old data still exists earlier in log Crash recovery discovers latest valid block on disk

34. 34 LSF example

35. 35 LSF example

36. 36 LSF example

37. 37 LSF example

38. 38 Storage backend - Logging block store (LBS) Backend represents VM disk & checkpoint images with files on host Similar format to a LFS (performance, reliability) Split data and meta-data between internal & external storage Encryption & integrity checks on data file Meta-data and encryption keys located on protected internal storage (security)

39. 39 LBS performance.

40. 40 Agenda Smartphone hardware I/O virtualization techniques Examples from MVP hypervisor Storage virtualization GPS virtualization Telephony virtualization

41. 41 GPS virtualization

42. 42 GPS virtualization

43. 43 Agenda Smartphone hardware I/O virtualization techniques Examples from MVP hypervisor Storage virtualization GPS virtualization Telephony virtualization

44. 44 Telephony virtualization

45. 45 Telephony virtualization GSM Dual SIM smartphones (geography dependent) Multiple International Mobile Subscriber Identities (IMSIs) per SIM (geo dep) GSM Alternate Line Service (ALS) (network dependent) Network Call forwarding, ala Google Voice, Line2 Voice-over-IP (VoIP) Realistic with 4G/LTE Integration with enterprise Unified Commmunications (UC)