1. Scalable and Crash-Tolerant Load Balancing based on Switch Migration
for Multiple OpenFlow Controllers 　
Chu LIANG＊ Ryota KAWASHIMA＊ Hiroshi MATSUO＊
＊Nagoya Institute of Technology, Japan

2. Distributed control plane has been proposed
Research Background
The spread of Software-Defined Networking
Easier management and faster innovation
A centralized controller and programmable network devices
The complication of controller is increasing
Load balancing, QoS controlling, Security…
The size of networks continues to increase
The growing number of OpenFlow-enabled devices
A centralized controller has a potential bottleneck
Distributed control plane has been proposed

3. Distributed OpenFlow Controllers
OpenFlow : one of the mostly representative protocol for SDN
Packet-in : not match any forwarding rule forward to controller
OpenFlow Controller
OFC 1
OpenFlow controller cluster
OFC 2
Packet-in
OpenFlow switches
Group B
Group A
physically distributed controllers achieve better scalability

4. Problems in Distributed Controllers
Load imbalance results in suboptimal performance
Topology change
Variety user traffic
Elastic resources, VMs migration, Variety utilizations
OFC 1
OpenFlow controller cluster
OFC 2
High loaded
Low loaded
Static mapping configuration
OpenFlow switches
Dynamic load balance among the controllers is required
Group B
Group A

5. Problems in OpenFlow Controllers
Multiple Controllers in OpenFlow
Roles : Master/ Slave/ Equal
Master
Slave ？
Switch
OFC 1
OFC 2 S1 S2 S3
OFC1
OFC2
Master
Slave
Master
Slave
Master
Slave
Master-Request
Role-Reply
Role-Reply
Role-Reply
Master-Request
Master-Request
Each controller only has one role S1 S2 S3
The coordination of “role changing” is not provided

6. Related Work
Scalable OpenFlow Controller Redundancy Tackling Local and Global Recoveries　
Keisuke Kuroki, Nobutaka Matsumoto, Michiaki Hayashi, The Fifth International Conference on Advances in Future Internet
Towards an Elastic Distributed SDN Controller　
Advait Dixit, Fang Hao, Sarit Mukherjee, T.V. Lakshman, Ramana Kompella, ACM SIGCOMM HotSDN, 2013
Proposed a crash-tolerant method based on with multiple controllers of OpenFlow1.3
Do not support load balancing among the controllers
Role-Management server can be single point of failure
Proposed a switch migration protocol according to controller load
Be complex and do not support the crash-tolerant for master controller

7. Proposed Method
Dynamically shift the load across the multiple controllers
different controllers can be set master for individual switch
switch migration
Support the crash-tolerant for controllers
distributed architecture
automatic failover in the event of a failure
JGroups based communication
Simplification of the switch management by grouping
each controller only manage switches in the same group
switch migration is performed in group

8. Proposed Architecture
Global DB
Local controller cluster
Local DB
Global controller cluster
OpenFlow
switch
Group A
Group B
Group C

9. Global controller cluster
Global DB
Global controller cluster
Based on the global JGroups channel
Share global data
tenant information, user data etc.
Provide global view of network to upper controller
can be considered as a logically centralized controller plane

10. Dynamically shift the load across the multiple controllers
Local controller cluster
Local controller cluster
reduce network delay
reduce communicate traffic
Synchronize network status
switch-controller mapping, link, port
Perform controller load scheduling
Coordinate switch migration
set master/slave role for switches
Local DB
Dynamically shift the load across the multiple controllers

11. Implementation Controller structure (OpenDaylight based)
Distributed Key-Value Store (Infinispan)
Application
Application
Event Notification (JGroups)
OpenDaylight　APIs
A : Load Monitoring Module
A : Load Monitoring Module
Collect and calculate controllers load
B : Load Scheduling Module
B : Load Scheduling Module
Selected master controller
C : Switch Migration Module
Perform switch migration
Link
Discovery
Switch
Manger
Host
Manger
OpenDaylight Core
OpenFlow Driver

12. A. Load Calculation
Coordinator : collecting and computing load information
Controller Load : switch metric and server metric
the number of active switches, packets requests rate (switch metric )
usage of cpu, memory, network bandwidth (server metric)
coordinator
OFC1
OFC2
Local Controller Cluster
OFC3
OFC4

13. Perform Switch Migration
B. Load Scheduling
When and Which controller should be elected as master
The lightest-load controller
Which switches should be selected to migrate
Dynamic round-trip time feedback based switch selection
Perform Switch Migration
OFC1
OFC2
OFC3
Controller failover
Add new switches ？

14. C. Switch Migration Initial heaviest-load Controller A OpenFlow
Switch T
Initial lightest-load
Controller B
Switch T migration Request
Role_request to Master
Master for T
Slave for T
Role_reply for Master
Switch T migration Reply
Slave for T
Master for T

15. C. Switch Migration Initial heaviest-load Controller A OpenFlow
Switch T
Initial lightest-load
Controller B
Switch T migration Request
Role_request to Master
Master for T
Slave for T
Role_reply for Master
Switch T migration Reply
Slave for T
Master for T
Role_request to Master
Failover time
Role_reply to Master
Master for T

16. Preliminary evaluation (1/2)
The switch migration process
The migration process takes about 2ms
Initial heaviest-load
Controller A
OpenFlow
Switch T
Initial lightest-load
Controller B
Switch T migration Request
Role_request to Master
Master for T
2ms
Slave for T
Role_reply for Master
Switch T migration Reply
Slave for T
Master for T

17. Preliminary evaluation (2/2)
The controller failover process
The failover process takes about an average of 20ms
mostly affected by the failure detection provided by JGroups.
Initial heaviest-load
Controller A
OpenFlow
Switch T
Initial lightest-load
Controller B
Slave for T
Master for T
Failure detection
Role_request to Master
Failover time
20ms
Role_reply to Master
17/18

18. Evaluation environment
Host 3
Iperf client
VM1
Host 1
Host 2
(OFC A)
(OFC B)
VM2
Iperf server
SW 1
SW 2
SW 3
SW 4
SW 5
SW 6
SW 7
SW 8
Host
Host
Host
Host
Host
Host
Host
Host
(Mininet Network)
Host 4
(Traffic Generator)

19. Evaluation Three kind of workloads Machine specifications Switch
Workload A
1000 pps
2000 pps
4000 pps
Workload B
6000 pps
Workload C
8000 pps
Load
Machine specifications
Controller Node
Traffic Generator
Evaluation Node OS
Ubuntu-server 12.04
Centos bit
CPU
Core i5(4 core)
Core i7(1 core)
Memory
16GB
8GB
Network
100Mbps Ethernet
OpenFlow Switch -
Open vSwtich

20. Results : Throughput
static : existing ( static switch-controller mapping)
proposal : dynamically switch migration
OFC A:6 switches
OFC B:2 switches
OFC A:5 switches
OFC B:3 switches
Throughput (Mbit/sec)
Load Difference
OFC A:7 switches
OFC B:1 switches
OFC A:6 switches
OFC B:2 switches
Workload A
Workload B
Workload C
Run Time (in sec)

21. Results : Response Time
Response Time: VM——OFC B (Ping)
Packets loss
cumulative distribution function (CDF)
workload A (static)
workload A (proposal)
workload B (static)
workload B (proposal)
workload C (static)
workload C (proposal)
Response Time (in msec)

22. Conclusion & Future Work
Proposed a scalable and crash-tolerant load balancing based on switch migration for multiple OpenFlow controllers
Enable the controllers coordinate actions to dynamically shift the load across the multiple controllers
Improve the throughput and response time of control plane
Future Work
Optimize of the load scheduling modules
Implement a topology aware switch migration algorithms to improve the scalability in the real large scale network
Evaluate the performance in vary applications and topologies with more practical traffics