1. vCRIB: Virtual Cloud Rule Information Base Masoud Moshref, Minlan Yu, Abhishek Sharma, Ramesh Govindan HotCloud 2012

2. Introduction Motivation Architecture Evaluation Conclusion Introduction 2 Datacenters use rules to implement management policies Access control Rate limiting Traffic measurement Traffic engineering Datacenters use rules to implement management policies R1 Src IP Dst IP R1=Accept  SrcIP: 12.0.0.0/8  DstIP: 10.0.0.0/16 An action on a hypercube of flow space Examples: Deny Normal Enqueue Flow fields examples: Src IP / Dst IP Protocol Src Port / Dst Port

3. Introduction Motivation Architecture Evaluation Conclusion Current Practice 3 Rules are saved on predefined fixed machines Multiple policies may compete for resources Machines have limited resources Datacenters have different resource constraints 1 2 3

4. Introduction Motivation Architecture Evaluation Conclusion vCRIB Goal: Flexible Rule Placement 4 Find the best feasible rule placement based on resource constraints A B C

5. Introduction Architecture Evaluation Conclusion Motivation Future Datacenters will have many fine-grained rules 5 Regulating VM pair communication Access control (CloudPolice) Bandwidth allocation (Seawall) Per flow decision Flow measurement for traffic engineering (MicroTE, Hedera) VLAN per server Traffic management (NetLord, Spain) 100K – 1M 10 – 100M 1M

6. Introduction Architecture Evaluation Conclusion Motivation Where to place rules? Hypervisors vs. Switches 6 HypervisorSwitch Limited CPU budget Software, Slow Complex rules # TCAM entries Hardware, Fast OpenFlow rules Performance Close to VMs External traffic, Aggregate traffic Flexibility Entry point Resources

7. Introduction Architecture Evaluation Conclusion Motivation Rule Location Trade-off (Resource vs. Bandwidth Usage) 7 Storing rules at hypervisor incurs CPU processing overhead

8. Introduction Architecture Evaluation Conclusion Motivation Rule Location Trade-off (Resource vs. Bandwidth Usage) 8 Move the rule to ToR switch and forward traffic Saving the rules at hypervisor uses all CPU budget

9. Introduction Architecture Evaluation Conclusion Motivation Can we reduce Open vSwitch CPU usage? 9 CPU usage # wildcard patterns # rules The set of ignore bits in the mask R1=Accept, DstIP: 10.0.0.0/16, SrcIP: 12.0.0.0/8 1111111111111111****************, 11111111************************ Handle same number of new flows with lower CPU budget

10. Introduction Architecture Evaluation Conclusion Motivation Rule Location Trade-off (Resource vs. Bandwidth Usage) 10 If rule memory is limited in one switch

11. Introduction Architecture Evaluation Conclusion Motivation Rule Location Trade-off (Resource vs. Bandwidth Usage) 11 just move the rule to another switch Can tradeoff bandwidth within the switch fabric, in addition to trading-off bandwidth between hypervisors and switches

12. Introduction Motivation Evaluation Conclusion Architecture Our Approach: vCRIB, a Virtual Cloud Rule Information Base 12 Allow operators to define fine-grained rules without worrying about placement Proactive rule placement abstraction layer Optimize performance given resource constraints Flexible rule placement at hypervisors and switches Rules

13. Introduction Motivation Evaluation Conclusion Architecture Challenges: Overlapping Rules 13 Rules

14. Introduction Motivation Evaluation Conclusion Architecture Challenges: Overlapping Rules 14 R0 R4 R3 R2 R1 Src IP Dst IP R1

15. Introduction Motivation Evaluation Conclusion Architecture R0 R4 R3 R2 R1 R5 Challenges: Overlapping Rules 15 R0 R2 R6 R1R3 R7 R4 R8 R6 R4 R8 R5 R3 R7 R0 R2 R1 Partitions rules to reduce overlapping rules dependency Splitting rules covering multiple partitions causes inflation

16. Introduction Motivation Evaluation Conclusion Architecture vCRIB: Partitioning 16 Recursively cut partitions to create a BSP tree R6 R4 R8 R5 R3 R7 R0 R2 R1 R0 R4 R3 R2 R1 R5 R4 R3 R7 Select a cut that balances two partitions creates fewest number of new rules Smaller partitions are more flexible to place match fewer communicating VMs Stop whenever a resource at a node is exhausted

17. Introduction Motivation Evaluation Conclusion Architecture Challenges: Placement Complexity 17 R6 R4 R8 R5 R3 R7 R0 R2 R1 Constraints Functionality Machine resources Goal Minimize traffic overhead Minimize delay Minimize cost of bandwidth usage vs. saved CPU Generalized Assignment Problem T 11 T 21 T 22 T 23 T 32 T 33 Different partition sizes Different machine capacities Different traffic overhead for each partition location

18. Introduction Motivation Evaluation Conclusion Architecture vCRIB: Placement ( Branch and Bound) 18 Select the largest unassigned partition Capable of handling its rules  Functionality  Resources Make minimum traffic overhead Place it on a switch/hypervisor

19. Introduction Motivation Evaluation Conclusion Architecture vCRIB Architecture 19 R6 R4 R8 R5 R3 R7 R0 R2 R1 R0 R4 R3 R2 R1 R5 R4 R3 R7 Partitioning Placement R6 R4 R8 R5 R3 R7 R0 R2 R1 Partitions Traffic and Topology information Rule Placement vCRIB Manager Rules

20. Introduction Motivation Architecture Conclusion Evaluation Evaluation: Goal 20 Can partitioning algorithm achieve small partitions? Can placement algorithm leverage resource availability to decrease traffic overhead? Configuration 100 VMs per machine 10K flows (10KB) per machine ClassBench rules 1K rule capacity per switches

21. Introduction Motivation Architecture Conclusion Evaluation Evaluation: Partitioning 21 Change rule capacity to show the effect of different CPU budgets Maximum size of partitions goes down as resources increase

22. Introduction Motivation Architecture Conclusion Evaluation Evaluation: Placement 22 For each machine select VM addresses from a contiguous IP range Traffic decreases as resources increase Aggregated addresses make lower traffic overhead No traffic decrease R5 R3 R7 Replication

23. Introduction Motivation Architecture Evaluation Conclusion Conclusion 23 vCRIB provides an abstraction layer for placement of rules in datacenters Places the rules on both hypervisors and switches to achieve the best performance given the resource constraints

24. Introduction Motivation Architecture Evaluation Conclusion Future Work 24 Exploit performance model of hypervisors & switches Online Algorithm adjusting to traffic changes Replication in the partitioning and placement algorithm

25. vCRIB: Virtual Cloud Rule Information Base Masoud Moshref, Minlan Yu, Abhishek Sharma, Ramesh Govindan HotCloud 2012

26. Introduction Motivation Architecture Evaluation Conclusion OVS paper diagrams 26

27. Introduction Motivation Architecture Evaluation Conclusion Distributed Firewall 27

28. Introduction Motivation Architecture Conclusion Evaluation Evaluation: Placement 28

29. Introduction Motivation Evaluation Conclusion Architecture vCRIB: Partitioning 29 Recursively cut partitions to create a BSP tree R6 R4 R8 R5 R3 R7 R0 R2 R1 R0 R4 R3 R2 R1 R5 R4 R3 R7 Select a cut that minimizes F=α×max p S(p) + (1−α)×N Smaller partitions are more flexible to place match fewer communicating VMs Stop whenever resources finish