1. 1 Logically Centralized? State Distribution Trade-offs in Software Defined Networks Written By Dan Levin, Andreas Wundsam, Brandon Heller, Nikhil Handigol, and Anja Feldmann Presented By Michael Over

2. 2 Abstract Software Defined Networks (SDNs) give network designers freedom to refactor the network control plane Software Defined Networks (SDNs) give network designers freedom to refactor the network control plane Enables the network control logic to be designed and operated on a global network view Enables the network control logic to be designed and operated on a global network view Control plane state and logic must inevitably be physically distributed to achieve responsiveness, reliability, and scalability goals Control plane state and logic must inevitably be physically distributed to achieve responsiveness, reliability, and scalability goals Motivating Question: How does distributed SDN state impact the performance of a logically centralized control application? Motivating Question: How does distributed SDN state impact the performance of a logically centralized control application?

3. 3 Abstract Characterize the state exchange points in a distributed SDN control plane Characterize the state exchange points in a distributed SDN control plane Two key state distribution trade-offs simulated in the context of an existing SDN load balancer application Two key state distribution trade-offs simulated in the context of an existing SDN load balancer application The impact of inconsistent global network view on load balancer performance The impact of inconsistent global network view on load balancer performance Compare different state management approaches Compare different state management approaches Results show that SDN control state inconsisntency significantly degrades performance of logically centralized control applications Results show that SDN control state inconsisntency significantly degrades performance of logically centralized control applications

4. 4 Outline Introduction Introduction Related Work Related Work Distributed State Management and Trade-Offs Distributed State Management and Trade-Offs Example Application Example Application Experiments Experiments Summary Summary

5. 5 Introduction SDNs have sparked significant interest in rethinking classical approaches to network architecture and design SDNs have sparked significant interest in rethinking classical approaches to network architecture and design SDN enables the network control plane logic to be decoupled from the network forwarding hardware SDN enables the network control plane logic to be decoupled from the network forwarding hardware Enables the ability to design and reason about the network control plane as a centrally controlled application operating on a global network view (GNV) as its input Enables the ability to design and reason about the network control plane as a centrally controlled application operating on a global network view (GNV) as its input

6. 6 Introduction Freedom to refactor the network control plane – logically centralized Freedom to refactor the network control plane – logically centralized New choices: How centralized or distributed should the network control plane be? New choices: How centralized or distributed should the network control plane be? Fully physically centralized control is inadequate – limits responsiveness, reliability, and scalability Fully physically centralized control is inadequate – limits responsiveness, reliability, and scalability Resort to a physically distributed control plane, on which a logically centralized control plane operators Resort to a physically distributed control plane, on which a logically centralized control plane operators Trade-offs between different consistency models and associated liveness properties Trade-offs between different consistency models and associated liveness properties

7. 7 Introduction Strongly consistent control designs always operate on a consistent world view Strongly consistent control designs always operate on a consistent world view Limits responsiveness which can lead to suboptimal decisions Limits responsiveness which can lead to suboptimal decisions Eventually consistent designs integrade information as it becomes available Eventually consistent designs integrade information as it becomes available React faster and can cope with higher update rates React faster and can cope with higher update rates Potentially present a temporarily inconsistent world view and thus may cause incorrect behavior Potentially present a temporarily inconsistent world view and thus may cause incorrect behavior

8. 8 Introduction Need to understand how physically distributed control plane state will impact the performance and correctness of a control application logic designed to operate as though it were centralized Need to understand how physically distributed control plane state will impact the performance and correctness of a control application logic designed to operate as though it were centralized When the underlying distributed control plane state leads to inconsistency or staleness in the global network view, how much does the network performance suffer? When the underlying distributed control plane state leads to inconsistency or staleness in the global network view, how much does the network performance suffer?

9. 9 Introduction Characterize the state exchange points in a distributed SDN control plane Characterize the state exchange points in a distributed SDN control plane Two key state distribution trade-offs arise: Two key state distribution trade-offs arise: Control application performance (optimality) vs. state distribution overhead Control application performance (optimality) vs. state distribution overhead Application logic complexity vs. robustness to inconsistency Application logic complexity vs. robustness to inconsistency These trade-offs are simulated in the context of an existing SDN application for flow-based load balancing These trade-offs are simulated in the context of an existing SDN application for flow-based load balancing

10. 10 Introduction Two different control application approaches: Two different control application approaches: A simple approach that is ignorant to potential inconsistency A simple approach that is ignorant to potential inconsistency A more complex approach that considers the potential inconsistency when making a load balancing decision A more complex approach that considers the potential inconsistency when making a load balancing decision Initial results demonstrate that global network view (GNV) inconsistency significantly degrades the performance of the simple approach, while the more complex approach is more robust Initial results demonstrate that global network view (GNV) inconsistency significantly degrades the performance of the simple approach, while the more complex approach is more robust

11. 11 Outline Introduction Introduction Related Work Related Work Distributed State Management and Trade-Offs Distributed State Management and Trade-Offs Example Application Example Application Experiments Experiments Summary Summary

12. 12 Related Work Three main categories: Three main categories: SDN applications that motivate this study SDN applications that motivate this study Control frameworks that provide a logically centralized view to SDN applications Control frameworks that provide a logically centralized view to SDN applications Previous studies on routing state distribution trade- offs Previous studies on routing state distribution trade- offs Onix is a control plane platform designed to enable scalable control applications – abstract away the task of network state distribution Onix is a control plane platform designed to enable scalable control applications – abstract away the task of network state distribution

13. 13 Related Work Hyperflow is a distributed event-based control plane for OpenFlow that allows control applications to make decisions locally by passively synchronizing network- wide views of the individual controller instances Hyperflow is a distributed event-based control plane for OpenFlow that allows control applications to make decisions locally by passively synchronizing network- wide views of the individual controller instances Consistent Updates focuses on state management between the physical network and the network information base to enforce consistent forwarding state at different levels Consistent Updates focuses on state management between the physical network and the network information base to enforce consistent forwarding state at different levels Consensus Routing presents a consistency-first approach to adopting forwarding upates in the context of inter-domain routing Consensus Routing presents a consistency-first approach to adopting forwarding upates in the context of inter-domain routing Probabilistically Bounded Staleness is a set of models that predicts the expected consistency of an eventually- consistent data store Probabilistically Bounded Staleness is a set of models that predicts the expected consistency of an eventually- consistent data store

14. 14 Outline Introduction Introduction Related Work Related Work Distributed State Management and Trade-Offs Distributed State Management and Trade-Offs Example Application Example Application Experiments Experiments Summary Summary

15. 15 Distributed State Management and Trade Offs

16. 16 Distributed State Management and Trade Offs Each controller maintains a Network Information Base (NIB), a view of the global network state presented to an application. Each controller maintains a Network Information Base (NIB), a view of the global network state presented to an application. The NIB at each controller is periodically and independently updated with state collected from the physical network. The NIB at each controller is periodically and independently updated with state collected from the physical network. Controllers synchronize their NIB state among themselves Controllers synchronize their NIB state among themselves

17. 17 Distributed State Management and Trade Offs Different manners of distributed, replicated storage models may be used to realize the NOS state distribution and management Different manners of distributed, replicated storage models may be used to realize the NOS state distribution and management Key Property of any NOS state distribution approach – The degree of state consistency achieved: Key Property of any NOS state distribution approach – The degree of state consistency achieved: Strong consistency Strong consistency Eventual consistency Eventual consistency

18. 18 Distributed State Management and Trade Offs A strongly consistent NOS will never present inconsistent NIB state to an application A strongly consistent NOS will never present inconsistent NIB state to an application However, limits the rate at which NOS state can be updated However, limits the rate at which NOS state can be updated While an update is being processed, applications continue to operate on a stale (but consistent) world view While an update is being processed, applications continue to operate on a stale (but consistent) world view Eventually consistent approaches react faster but temporarily introduce inconsistency Eventually consistent approaches react faster but temporarily introduce inconsistency

19. 19 Distributed State Management and Trade Offs Trade-off #1: Consistency model of NOS state distribution vs. application objective optimality Trade-off #1: Consistency model of NOS state distribution vs. application objective optimality The performance of the network in relation to the control application’s objective can suffer with inconsistent or stale global network view The performance of the network in relation to the control application’s objective can suffer with inconsistent or stale global network view The cost to achieving consistent state in the global network view entails higher rates of control synchronization and communication overhead -> decreases responsiveness The cost to achieving consistent state in the global network view entails higher rates of control synchronization and communication overhead -> decreases responsiveness

20. 20 Distributed State Management and Trade Offs Trade-off #2: Application logic complexity vs. robustness to stale NOS state Trade-off #2: Application logic complexity vs. robustness to stale NOS state A “logically centralized” application that is unaware of the potential staleness of its input is simpler to design A “logically centralized” application that is unaware of the potential staleness of its input is simpler to design An application which is aware of the underlying distributed NOS state can take measures to separate and compare the inter-domain global network view with its own local domain view An application which is aware of the underlying distributed NOS state can take measures to separate and compare the inter-domain global network view with its own local domain view

21. 21 Outline Introduction Introduction Related Work Related Work Distributed State Management and Trade-Offs Distributed State Management and Trade-Offs Example Application Example Application Experiments Experiments Summary Summary

22. 22 Example Application An arrival-based network load balancer control application An arrival-based network load balancer control application Objective: Minimize the maximum link utilization in the network Objective: Minimize the maximum link utilization in the network Two implementations featuring different state (and staleness) awareness and management approaches: Two implementations featuring different state (and staleness) awareness and management approaches: Link Balancer Controller (LBC) Link Balancer Controller (LBC) Separate State Link Balancer Controller (SSLBC) Separate State Link Balancer Controller (SSLBC)

23. 23 Example Application Link Balancer Controller (LBC) – the simpler of the two approaches Link Balancer Controller (LBC) – the simpler of the two approaches Upon a dataplane triggered event, a global network view is presented by the NOS to the domain application instance Upon a dataplane triggered event, a global network view is presented by the NOS to the domain application instance Combines both the physical network state from within the domain as well as any inter-domain link utilization updates from other controllers Combines both the physical network state from within the domain as well as any inter-domain link utilization updates from other controllers A list of paths with utilizations is generated and used to choose the path with the lowest max link utilization to assign the next arriving flow A list of paths with utilizations is generated and used to choose the path with the lowest max link utilization to assign the next arriving flow

24. 24 Example Application Separate State Link Balancer Controller (SSLBC) Separate State Link Balancer Controller (SSLBC) Keeps fresh intra-domain physical network state separate from updates learned through inter-domain controller synchronization events Keeps fresh intra-domain physical network state separate from updates learned through inter-domain controller synchronization events The arrival-based path selection incorporates logic to ensure convergence properties on load distribution The arrival-based path selection incorporates logic to ensure convergence properties on load distribution Calculates new link metrics and chooses the path with the minimum max link utilization Calculates new link metrics and chooses the path with the minimum max link utilization Effectively, the global network view guides each application instance to redistribute a scaled fraction of its local l ink imbalance on a flow-by-blow arrival basis Effectively, the global network view guides each application instance to redistribute a scaled fraction of its local l ink imbalance on a flow-by-blow arrival basis

25. 25 Outline Introduction Introduction Related Work Related Work Distributed State Management and Trade-Offs Distributed State Management and Trade-Offs Example Application Example Application Experiments Experiments Summary Summary

26. 26 Experiments Developed a custom simulator to implement the key state-exchange interfaces of an SDN Developed a custom simulator to implement the key state-exchange interfaces of an SDN Designed to capture interactions between the three SDN layers from Figure 1 – Physical Network, State Management, and Application Designed to capture interactions between the three SDN layers from Figure 1 – Physical Network, State Management, and Application Very simple topology chosen for the experiment – two cooperating controller domains, with each domain having a single switch and server Very simple topology chosen for the experiment – two cooperating controller domains, with each domain having a single switch and server Consider only downstream traffic Consider only downstream traffic Load balancer objective is to minimize the different between all link utilizations; they use RMSE of the maximum link utilization along each path Load balancer objective is to minimize the different between all link utilizations; they use RMSE of the maximum link utilization along each path

27. 27 Experiments

28. 28 Experiments Two different workloads to explore the impact of inconsistent NOS state on network load balance Two different workloads to explore the impact of inconsistent NOS state on network load balance First, a deterministic, controlled workload to impart a link utilization imbalance First, a deterministic, controlled workload to impart a link utilization imbalance Second, a more realistic workload Second, a more realistic workload First, the load balancer based upon the LBC state management approach with different synchronization intervals: 0, 1, 2, 4, 8, and 16 timesteps First, the load balancer based upon the LBC state management approach with different synchronization intervals: 0, 1, 2, 4, 8, and 16 timesteps

29. 29 Experiments

30. 30 Experiments

31. 31 Experiments Controlled Workkload – SSLBC Controlled Workkload – SSLBC Next the second trade-off is examined, namely, how the SSLBC state management approach is able to handle the workload Next the second trade-off is examined, namely, how the SSLBC state management approach is able to handle the workload

32. 32 Experiments

33. 33 Experiments

34. 34 Experiments Using a more realistic workload, the authors also demonstrated similar results Using a more realistic workload, the authors also demonstrated similar results

35. 35 Outline Introduction Introduction Related Work Related Work Distributed State Management and Trade-Offs Distributed State Management and Trade-Offs Example Application Example Application Experiments Experiments Summary Summary

36. 36 Summary Logically centralized world views enable simplified programming models Logically centralized world views enable simplified programming models However, as the logically centralized world view is mapped to a physically distributed system, fundamental trade-offs emerge that affect application performance, liveness, robustness, and correctness However, as the logically centralized world view is mapped to a physically distributed system, fundamental trade-offs emerge that affect application performance, liveness, robustness, and correctness These trade-offs were revisisted in the context of design choices exposed by SDNs These trade-offs were revisisted in the context of design choices exposed by SDNs Staleness vs. optimality Staleness vs. optimality Application logic complexity vs. robustness to inconsistency Application logic complexity vs. robustness to inconsistency

37. 37 Summary Similar trade-offs arise in other SDN control applications Similar trade-offs arise in other SDN control applications Future Work: Future Work: Extend the simulator to more realistically model traffic characteristics and delays, and the overhead of synchronization Extend the simulator to more realistically model traffic characteristics and delays, and the overhead of synchronization Apply tools to larger and more complex toplogies Apply tools to larger and more complex toplogies Compare the results of different applications Compare the results of different applications

38. 38 Questions?