1. M. S. Thesis Defense, 09/24/20011 Migratory TCP (MTCP) Transport Layer Support for Highly- Available Network Services Kiran Srinivasan DisCoLab Division of Computer and Information Sciences Rutgers University Advisor: Liviu Iftode

2. M. S. Thesis Defense, 09/24/20012 Services Rather Than Servers Client oblivious to a given server’s location Client is interested only in the content provided and QoS A Service -> set of equivalent servers Internet services Most services are based on TCP Clients demand both high-availability and performance

3. M. S. Thesis Defense, 09/24/20013 TCP-based Internet Services Adverse conditions Core network congestion, overloaded servers, failed servers or under DoS attack TCP response Network delays => packet loss => retransmission TCP limitations Creates an implicit binding of service to a server ! Clients cannot change to another server for sustained service

4. M. S. Thesis Defense, 09/24/20014 Migratory TCP Transport layer protocol that supports dynamic connection migration Extension to TCP A client connection can transparently migrate to different servers during its lifetime Modified server applications cooperate to support the handoff Client applications do not change Servers can be geographically distributed Does not depend on application-specific info

5. M. S. Thesis Defense, 09/24/20015 Contributions Designed a transport layer protocol that enables dynamic connection migration Implemented a prototype on FreeBSD Demonstrated the utilization of our protocol in building highly-available services Conducted preliminary performance evaluation F. Sultan, K. Srinivasan, L. Iftode - “Transport Layer Support for Highly-Available Network Services ”, HotOS 2001

6. M. S. Thesis Defense, 09/24/20016 Outline Motivation Protocol Design Prototype Implementation Preliminary Performance Evaluation Protocol Utilization Related Work Conclusions and Future Work

7. M. S. Thesis Defense, 09/24/20017 The Migration Model Client Server 1 - overloaded Server 2 congestion

8. M. S. Thesis Defense, 09/24/20018 Triggers and Initiators Client Server 1 Server 2 MIGRATE_TRIGGER MIGRATE_INITIATE

9. M. S. Thesis Defense, 09/24/20019 Design Issues How to resume the service at the new server? Without the involvement of the client application Without full migration of the server application With low overhead What is the state that needs to be transferred? Application state Protocol state

10. M. S. Thesis Defense, 09/24/200110 Contract Between Server Application and MTCP Application Define fine-grained per-connection application state Export per-connection application state snapshot periodically to the kernel After migration, import per-connection state to resume service MTCP Transfer the per-connection state Synchronize the application state with the protocol state

11. M. S. Thesis Defense, 09/24/200111 Server Application Interface export_state Export the state snapshot at the origin server Many times periodically import_state Import the state snapshot at the destination server Enables the destination server to resume from the last snapshot Only once at the destination server Light-weight connection migration

12. M. S. Thesis Defense, 09/24/200112 Server Application Example while ((s = accept(ssock)) != -1) { if (import_state(s, &state)) num = state else num = 0 recv(s, &buf) state = ++ num export_state(s, &state) recv(s, &buf) state = ++ num export_state(s, &state)... }

13. M. S. Thesis Defense, 09/24/200113 Contract Between Server Application and MTCP Application Define fine-grained per-connection application state Export per-connection application state snapshot periodically After migration, import per-connection state to resume service MTCP Transfer the per-connection state Synchronize the application state with the protocol state

14. M. S. Thesis Defense, 09/24/200114 MTCP Protocol Server 1Server 2 Application Protocol Connections

15. M. S. Thesis Defense, 09/24/200115 State Synchronization Example while ((s = accept(ssock)) != -1) { if (import_state(s, &state)) num = state else num = 0 recv(s, &buf) state = ++ num export_state(s, &state) recv(s, &buf) state = ++ num export_state(s, &state)... }

16. M. S. Thesis Defense, 09/24/200116 State Synchronization Example while ((s=accept(ssock)) != -1) { if (import_state(s, &state)) num = state else num = 0 recv(s, &buf) state = ++num export_state(s, &state) recv(s, &buf) state = ++num export_state(s, &state)... }

17. M. S. Thesis Defense, 09/24/200117 State Synchronization Example

18. M. S. Thesis Defense, 09/24/200118 Log-Based State Synchronization Logs are maintained in the protocol At export time, protocol discards logs (sync-ed) Logs form part of the state to be transferred during migration Resumption of service starts from the last state snapshot

19. M. S. Thesis Defense, 09/24/200119 Implementation Modified the TCP/IP stack in FreeBSD kernel Lazy connection migration mechanism Evaluation on synthetic benchmarks Target applications HTTP server PostgreSQL front-end

20. M. S. Thesis Defense, 09/24/200120 Lazy Connection Migration C (0) C’C’ (2) (3) Client Server 1 Server 2 (1) (4)

21. M. S. Thesis Defense, 09/24/200121 Implementation: Per-Connection State snap not ack’ed Write Buffer Application Write Application Protocol

22. M. S. Thesis Defense, 09/24/200122 Implementation: Per-Connection State X From network Read pointer Read since snap Read before snap ack’ed Application Protocol Read Buffer snap

23. M. S. Thesis Defense, 09/24/200123 Preliminary Performance Evaluation Experimental setup Two servers and a client: P III 233MHz, 256 MB RAM Servers are connected by a dedicated network link Benchmarks Microbenchmark Simple web server like application

24. M. S. Thesis Defense, 09/24/200124 Microbenchmark One migration averaged over 200 runs for each state size All times are in microseconds The state size is in bytes

25. M. S. Thesis Defense, 09/24/200125 Microbenchmark The time for migration varies linearly with state size

26. M. S. Thesis Defense, 09/24/200126 Stream Server Experiment Server application sends a stream of 256K bytes in chunks of 1K each Server performance degrades after a threshold Degradation is emulated by means of delays in the server Migration policy module inserted into kernel at the client Smoothed rate estimator Migration occurs when the estimated rate drops to 70% of max. observed rate

27. M. S. Thesis Defense, 09/24/200127 Stream Server Experiment Results Effective rate is almost equal to the best rate from the server

28. M. S. Thesis Defense, 09/24/200128 Implementation Summary Addition of ~5000 lines of kernel code Changes to TCP finite state machine Addition of two logical TCP states: Migrating and Blackhole Time spent on the design and implementation was 9 months

29. M. S. Thesis Defense, 09/24/200129 Protocol Utilization For high availability For performance through load balancing schemes Servers trigger migrations based on a load balancing policy Fault tolerance Eager transfer of connection state information

30. M. S. Thesis Defense, 09/24/200130 Related Work Classes of related work TCP Fault tolerance Process migration

31. M. S. Thesis Defense, 09/24/200131 TCP Related Works HTTP server fail-over by connection migration [Snoeren ‘00] soft TCP and HTTP state maintained at back-up server Fault-tolerant TCP [Alvisi ‘00] failure masking persistent connections across server crashes Stream Control Transmission Protocol (SCTP) - RFC 2960 multi-homing (many IP addresses / endpoint) ensures connectivity in the face of network failure

32. M. S. Thesis Defense, 09/24/200132 TCP Related Works (cont’d) MSOCKS [Bhagwat `98] Proxy based “TCP splicing” for mobile clients with multiple interfaces Indirect TCP [Bakre ‘95] Connection handoffs between MSRs allow mobile hosts to maintain open connections with a fixed host

33. M. S. Thesis Defense, 09/24/200133 Fault Tolerance Related Works Log-based rollback recovery The NonStop kernel [Bartlett ’81]

34. M. S. Thesis Defense, 09/24/200134 Process Migration Related Works Condor [Litzkow ‘88] Locus [Popek ‘81] MOSIX [Barak ‘85] Sprite [Ousterhout ‘88]

35. M. S. Thesis Defense, 09/24/200135 Conclusions We provide a transport layer protocol that enables building of highly-available services The transport layer protocol allows dynamic server endpoint connection migration We implemented a prototype in a FreeBSD kernel We designed an API for the server applications Preliminary performance evaluation on a web server- like application achieved constant rate in the presence of server degradation http://discolab.rutgers.edu/mtcp

36. M. S. Thesis Defense, 09/24/200136 Future Work Investigate migration policies Alternate state transfer implementations Recursive connection migration mechanism Future applications

37. M. S. Thesis Defense, 09/24/200137 Acknowledgment Florin – for contributing to the design and implementation of the project Other members of DisCo Lab

38. M. S. Thesis Defense, 09/24/200138 Thank You Demo in CoRE 336 !