1. Scalable Rule Management for Data Centers Masoud Moshref, Minlan Yu, Abhishek Sharma, Ramesh Govindan 4/3/2013

2. Motivation Design Evaluation Introduction Introduction: Definitions 2 Datacenters use rules to implement management policies Access control Rate limiting Traffic measurement Traffic engineering Datacenters use rules to implement management policies An action on a set of ranges on flow fields Examples: Deny Accept Enqueue Flow fields examples: Src IP / Dst IP Protocol Src Port / Dst Port

3. Motivation Design Evaluation Introduction Introduction: Definitions 3 Datacenters use rules to implement management policies R2 R1 Src IP R1: Accept  SrcIP: 12.0.0.0/7  DstIP: 10.0.0.0/8 An action on a set of ranges on flow fields Dst IP R2: Deny  SrcIP: 12.0.0.0/8  DstIP: 8.0.0.0/6

4. Motivation Design Evaluation Introduction Current practice 4 Rules are saved on predefined fixed machines On hypervisorsOn switches

5. Motivation Design Evaluation Introduction Machines have limited resources 5 Top-of-Rack switch Network Interface Card Software switches on servers

6. Motivation Design Evaluation Introduction Future datacenters will have many fine-grained rules 6 Regulating VM pair communication Access control (CloudPolice) Bandwidth allocation (Seawall) Per flow decision Flow measurement for traffic engineering (MicroTE, Hedera) VLAN per server Traffic management (NetLord, Spain) 1B – 20B rules 10M – 100M rules 1M rules

7. Introduction Architecture Motivation Design Evaluation Rule location trade-off (resource vs. bandwidth usage) 7 Storing rules at hypervisor incurs CPU overhead R0

8. Introduction Architecture Motivation Design Evaluation Rule location trade-off (resource vs. bandwidth usage) 8 Storing rules at hypervisor incurs CPU overheadMove the rule to ToR switch and forward traffic R0

9. Introduction Architecture Motivation Design Evaluation Rule location trade-off: Offload to servers 9 R1

10. Introduction Architecture Motivation Design Evaluation Challenges: Concrete example 10 Src IP Dst IP R0 R1 R5 R6 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 VM0 VM1 VM2 VM3 VM4 VM5 VM6 VM7

11. Introduction Architecture Motivation Design Evaluation Challenges: Overlapping rules 11 R0 R1 R5 R6 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 VM2 VM6 R0 R1 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 Source Placement: Saving rules on the source machine means minimum overhead Src IP Dst IP

12. Introduction Architecture Motivation Design Evaluation Challenges: Overlapping rules 12 R0 R1 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 VM2 VM6 R4 R0 R1 R2 R3 01234567 1 2 3 4 5 6 7 0 If Source Placement is not feasible Src IP Dst IP

13. Introduction Architecture Motivation Design Evaluation Heterogeneous devices Challenges 13 Respect resource constraints Minimize traffic overhead Traffic changes Rule changes VM Migration Preserve the semantics of overlapping rules Handle Dynamics

14. Introduction Design Motivation Evaluation Contribution: vCRIB, a Virtual Cloud Rule Information Base 14 Proactive rule placement abstraction layer Optimize traffic given resource constraints & changes Rules R1R2 R3R4 R3

15. Introduction Design Motivation Evaluation vCRIB design 15 Source Partitioning with Replication Topology & Routing Rules Partitions Overlapping Rules Minimum Traffic Feasible Placement

16. Introduction Design Motivation Evaluation R0 R 1 R5 R7 R8 R2 R3 R6 R4 Partitioning with replication 16 P1P3 P2 P1P3 01234567 1 2 3 4 5 6 7 0 R0 R 1 R8 R3 R4 Smaller partitions have more flexibility Cutting causes rule inflation 01234567 1 2 3 4 5 6 7 0 R0 R3 R1 R7 R6 01234567 1 2 3 4 5 6 7 0 R0 R1 R5 R2 R3 01234567 1 2 3 4 5 6 7 0 P2 R0 R 1 R8 R3 R4 R0 R5 R 1 R7 R2 R3 R6

17. Introduction Design Motivation Evaluation Partitioning with replication 17 R0 R3 R1 A7 R6 R0 R 1 R8 R3 R4 R0 R1 R5 R2 R3 R0 R1 R5 R2 R3 R7 R6 P1 (5 rules) P3 (5 rules) 01234567 1 2 3 4 5 6 7 0 01234567 1 2 3 4 5 6 7 0 01234567 1 2 3 4 5 6 7 0

18. Introduction Design Motivation Evaluation Per-source partitions 18 R0 R1 R5 R6 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 Limited resource for forwarding No need for replication to approximate source-placement Closer partitions are more similar Src IP Dst IP

19. Introduction Design Motivation Evaluation vCRIB design: Placement 19 Source Partitioning with Replication Topology & Routing Rules Partitions Placement T 11 T 21 T 22 T 23 T 32 T 33 Traffic Overhead Resource Constraints Minimum Traffic Feasible Placement

20. Introduction Design Motivation Evaluation vCRIB design: Placement 20 Source Partitioning with Replication Topology & Routing Rules Partitions Minimum Traffic Feasible Placement Feasible Placement Traffic Overhead Resource Constraints Traffic Overhead Resource-Aware Placement Traffic-Aware Refinement

21. Introduction Design Motivation Evaluation FFDS (First Fit Decreasing Similarity) 21 1.Put a random partition on an empty device 2.Add the most similar partitions to the initial partition until the device is full Find the lower bound for optimal solution for rules Prove the algorithm is a 2-approximation of the lower bound

22. Introduction Design Motivation Evaluation vCRIB design: Heterogeneous resources 22 Source Partitioning with Replication Topology & Routing Rules Partitions Minimum Traffic Feasible Placement Resource-Aware Placement Traffic-Aware Refinement Feasible Placement Resource Heterogeneity Resource Usage Function

23. Introduction Design Motivation Evaluation vCRIB design: Traffic-Aware Refinement 23 Source Partitioning with Replication Resource-Aware Placement Traffic-Aware Refinement Partitions Feasible Placement Minimum Traffic Feasible Placement Resource Usage Function Traffic Overhead Topology & Routing Rules

24. Introduction Design Motivation Evaluation Traffic-aware refinement Overhead greedy approach 1. Pick maximum overhead partition 2. Put it where minimizes the overhead and maintains feasibility 24 VM 2 VM 4 P2 P4

25. Introduction Design Motivation Evaluation Traffic-aware refinement Overhead greedy approach 1. Pick maximum overhead partition 2. Put it where minimizes the overhead and maintains feasibility  Problem: Local minima Our approach: Benefit greedy 25 VM 2 VM 4 P2 P4

26. Introduction Design Motivation Evaluation vCRIB design: Dynamics 26 Source Partitioning with Replication Resource-Aware Placement Traffic-Aware Refinement Rule/VM Dynamics Partitions Feasible Placement Minimum Traffic Feasible Placement Resource Usage Function Major Traffic Changes Dynamics Topology & Routing Rules

27. Introduction Design Motivation Evaluation vCRIB design 27 Source Partitioning with Replication Resource-Aware Placement Traffic-Aware Refinement Rule/VM Dynamics Partitions Feasible Placement Minimum Traffic Feasible Placement Resource Usage Function Traffic Overhead Major Traffic Changes Dynamics Topology & Routing Rules Resource Constraints Resource Heterogeneity Overlapping Rules

28. Evaluation Introduction Motivation Design Evaluation Comparing vCRIB vs. Source-Placement Parameter sensitivity analysis Rules in partitions Traffic locality VMs per server Different memory sizes Where is the traffic overhead added? Traffic-aware refinement for online scenarios Heterogeneous resource constraints Switch-only scenarios 28

29. Evaluation Introduction Motivation Design Simulation setup 1k servers with 20 VMs per server in a Fat-tree network 200k rules generated by ClassBench and random action IPs are assigned in two ways: Random Range Flows Size follows long-tail distribution Local traffic matrix (0.5 same rack, 0.3 same pod, 0.2 interpod) 29 01234567

30. Evaluation Introduction Motivation Design Comparing vCRIB vs. Source-Placement 30 Maximum Load is 5K Capacity is 4K Range is better as similar partitions are from the same source Random: Average load is 4.2K Adding more resources helps vCRIB reduce traffic overhead vCRIB finds low traffic feasible solution

31. Evaluation Introduction Motivation Design Parameter sensitivity analysis: Rules in partitions 31 Total space vCRIB Defined by maximum load on a server Feasible Source placement

32. Evaluation Introduction Motivation Design Total space vCRIB Parameter sensitivity analysis: Rules in partitions 32 Lower traffic overhead for smaller partitions and more similar ones <10% Traffic vCRIB <10% Traffic vCRIB <10% Traffic Source placement A A

33. Evaluation Introduction Motivation Design Future work Conclusion Conclusion and future work 33 vCRIB allows operators and users to specify rules, and manages their placement in a way that respects resource constraints and minimizes traffic overhead. Support reactive placement by adding the controller in the loop Break a partition for large number of rules per VM Test for other rulesets

34. Scalable Rule Management for Data Centers Masoud Moshref, Minlan Yu, Abhishek Sharma, Ramesh Govindan 4/3/2013

35. Motivation Design Evaluation Introduction Related work 35 DIFANE A few authority switches 1 partition per authority switch Cache micro rules at edge Assign partitions randomly Arbitrary partitions with cutting vCRIB in HotCloud’12 Arbitrary partitions with cutting Offline placement DFS branch and bound Only TCAM resource model vCRIB Save on all switches and servers Multiple partitions per device Forward traffic to host of rules Per-source partitions Online solution: two-step 1.Find feasible solution 2.Refine to minimize traffic Support heterogeneous resource

36. Motivation Design Evaluation Introduction Where to place rules? Hypervisors vs. Switches 36 HypervisorSwitch Limited CPU budget Software, Slow Complex rules # TCAM entries Hardware, Fast OpenFlow rules Performance Close to VMs External traffic, Aggregate traffic Flexibility Entry point Resources

37. Introduction Architecture Motivation Design Evaluation Challenges: Dynamics Traffic changes Rule changes VM migration 37 R0 R1 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 VM6 VM2

38. Introduction Design Motivation Evaluation Partitioning with split 38 A0 A 1 A5 A7 A8 A2 A3 A6 A4 R0 R 1 R8 R3 R4 R0 R5 R 1 R7 R2 R3 R6 R0 R 1 R8 R3 R4 R0 R3 A 1 R7 R6 A0 R1 R5 R2 R3 Finding the balanced minimum inflation partitions is NP-Hard

39. Introduction Design Motivation Evaluation Partitioning with split 39 R0 R 1 R8 R3 R4 R0 R3 R1 R7 R6 A0 R1 R5 R2 R3 S1S2

40. Introduction Design Motivation Evaluation Redirection rules in Partitioning with replication 40 R0 R3 R1 R7 R6 R0 R 1 R8 R3 R4 R0 R1 R5 R2 R3 VM6 VM2 R2 F3

41. Introduction Design Motivation Evaluation Per-source partitions 41 R0 R1 R5 R6 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 Rules are always contiguous blocks crossing only the neighboring partitions If i<j<k, any common rule between P_i and P_k is also in P_j i jk

42. Introduction Design Motivation Evaluation Divide the problem into two phases 42 Generalized Assignment Problem T 11 T 21 T 22 T 23 T 32 T 33 Different partition sizes Different device capacities Different traffic overhead for each partition location Find a feasible solution Iterate in the feasible solutions space to minimize traffic overhead

43. Introduction Design Motivation Evaluation Bin-packing for similar partitions 43 ToR1S1S2 ToR1S1S2 The bin-packing algorithm must be similarity-aware

44. Introduction Design Motivation Evaluation FFDS algorithm and per-source partitions 44 R0 R1 R5 R6 R2 R3 R4 01234567 1 2 3 4 5 6 7 0 Devices have 5 spaces

45. Motivation Design Evaluation Introduction Approximation bound proof Use Sindelar algorithm[1] on binary tree Partitions are leaves Create tree by joining the most similar nodes Count the number of devices the fractional placement needs Run a greedy algorithm on tree Fractional placement is lower bound for placement if tree captures 100% similarity of rules For per-source partitions and prefix-wildcard rules, no similarity will be lost FFDS is always better than Sindelar greedy algorithm 45 [1] M. Sindelar, R. K. Sitaram, and P. Shenoy. Sharing-Aware Algorithms for Virtual Machine Colocation. In SPAA, 2011

46. Introduction Design Motivation Evaluation Extend to CPU model 46

47. Introduction Design Motivation Evaluation Traffic-aware refinement 47 VM 2 VM 4 P2 P4

48. Evaluation Introduction Motivation Design Traffic monitoring No need for VM migration and rule changes Sources may not know the rules so we can only get flow statistics Monitoring implementation choices Open vSwitch already has active flows cache MicroTE adds a kernel module Only report large changes The controller sets a threshold in response of each report Set threshold for Flows for each rule per source VM and destination machine Flows per source VM and destination machine 48

49. Evaluation Introduction Motivation Design Traffic overhead 49 Traffic on core links increased by 2.5% 70% of traffic overhead is inside the racks

50. Evaluation Introduction Motivation Design CPU model and heterogeneous case evaluation 50 Maximum is 48% (68%) Range is not so good as before as similarity function changed

51. Evaluation Introduction Motivation Design FFDS algorithm performance 51

52. Motivation Design Evaluation Introduction Movement Definition 52 Avg Partition Size Avg Partition Similarity Add micro rules Always make sure maximum load on machines does not change So there is a bound in each direction Top: Extending a rule reduce load on max machine Right: All machines have the same maximum load Left: No micro rule on any non- max load machine Down: Halving any rule increases the load more than max machine Remove micro rules Half rules Extend rules

53. Evaluation Introduction Motivation Design Effect of similarity on traffic overhead Change similarity by selecting IP addresses closer to each other 53

54. Evaluation Introduction Motivation Design Other results 54 No change for skewed number of VMs per server Lower traffic overhead for less local traffic patterns 70% of traffic overhead is inside the racks vCRIB can handle Heterogeneous resources

55. Evaluation Introduction Motivation Design Online scenarios 55 Benefit Greedy can evade local minima of Overhead Greedy The benefit is not only for a special scenario

56. Motivation Design Evaluation Introduction FAQ 1. What kind of policies vCRIB does not support? 2. How do you get traffic changes? 3. What is the bandwidth overhead of migrating partitions? 4. How do you ensure consistency in migration? 5. What is the proof of approximation bound? 6. How do you change the ruleset to explore the space? 7. How partitioning with replication works with overlapping rules? 56