Quantifying and Improving I/O Predictability in Virtualized Systems Cheng Li, Inigo Goiri, Abhishek Bhattacharjee, Ricardo Bianchini, Thu D. Nguyen 1

Problem IaaS cloud providers (e.g., Amazon EC2) – Virtualization to consolidate virtual machines Performance may vary with consolidation – Interference and variable resource allocation – Inconsistent and unpredictable performance 2 VM Physical Machine VM Physical Machine VM Physical Machine

Our solution Virtualized systems with predictable performance – Consolidation should not affect throughput, response time Predictability is different than isolation – Assignment of resources to VMs must be fixed at all times New class of predictable-performance service This paper: storage I/O predictability in VirtualFence 3

Why? Many users desire predictable performance – Streaming and gaming apps – Performance tuning, debugging, diagnosis – Proper app design (e.g., workflows, pipelines) Predictability benefits providers – Can charge for exactly the resources used – Direct relationship between resources and performance Predictability benefits users – Can implement apps that need predictability – Predictable cloud costs 4

Outline Motivation Quantifying unpredictability VirtualFence Evaluation Conclusions 5

How to measure unpredictability? Performance deviation – Relative change in performance – Stand-alone (P I ) vs. co-located (P D ) – Average throughput or average response time 6

Studied deviation across VMMs, storage devices, etc – I/O performance deviation is endemic Some main sources of deviation: – Resource allocation policy (e.g., work-conserving) – Device-specific characteristics (e.g., SSD erasure) More findings in Rutgers DCS-TR-697 Quantifying performance deviation 7

Outline Motivation Quantifying unpredictability VirtualFence Evaluation Conclusions 8

VirtualFence Predictable-performance storage system for Xen 9 Dom0 Disk VirtualFence Scheduler Virtual Device Driver VM Kernel VMM SSD cache

VirtualFence techniques 1.Non-work-conserving time partitioning – Each VM is assigned a fixed amount of I/O time – Avoids interleaving requests from multiple VMs Stand-alone scenario Co-located scenario 10 VM 1 T1 ``` VM 1 ``` T2T3T4T5T6T7T8 … VM 1 T1 VM 2 VM 3 VM 4 VM 1 VM 2 VM 3 VM 4 T2T3T4T5T6T7T8 …

2.Small SSD cache in front of the HDD – Targets the HDD seek at the beginning of each time slot 3.Non-work-conserving space partitioning – Fixed size SSD cache per VM – Guarantees each VM’s cache space share Users can purchase multiple time and space slots! 11 VirtualFence techniques VM 1 VM 2VM 3VM 4 SSD cache co-located: SSD cache stand-alone:

Outline Motivation Quantifying unpredictability VirtualFence Evaluation Conclusions 12

Experimental environment Aggressive consolidation (80% utilization) Filebench workloads – Webserver: read-only – Mailserver: mixed reads/sync writes Physical machine: 4-core Xeon, 1 SSD, 1 HDD (22ms) VM1( 8%) VM2 (24%)VM4 (24%) VM3 (24%) 13

VirtualFence evaluation VirtualFence benefits Contribution of each technique Impact of the workload Absolute performance and VirtualFence overheads Performance vs. deviation tradeoff 14

VirtualFence combines all three techniques Approaches the deviation of SSD+TP at lower cost 15 VirtualFence results VirtualFence SSD+TP

Impact of number of time slots More slots decrease deviation, degrade performance Ideal: fewest slots that allow enough consolidation 16

Impact of time slot length Longer slots decrease deviation, degrade performance Ideal: shortest slot that produces enough predictability 17

Conclusions Consolidation leads to unpredictability VirtualFence – Software/hardware solution – Improves I/O predictability significantly – Provider selects best predictability vs. performance tradeoff – User rents as many slots as needed for good performance 18

Quantifying and Improving I/O Predictability in Virtualized Systems Cheng Li, Inigo Goiri, Abhishek Bhattacharjee, Ricardo Bianchini, Thu D. Nguyen Q&A