Presentation is loading. Please wait.

Presentation is loading. Please wait.

The TIME-WAIT state in TCP and its Effect on Busy Servers Theodore Faber University of Southern California Xindian Long.

Similar presentations

Presentation on theme: "The TIME-WAIT state in TCP and its Effect on Busy Servers Theodore Faber University of Southern California Xindian Long."— Presentation transcript:

1 The TIME-WAIT state in TCP and its Effect on Busy Servers Theodore Faber University of Southern California Xindian Long

2 Outline TIME-WAIT state and its function
Performance problems on busy web server Solution: move the TIME-WAIT to clients End points negotiation TCP, HTTP modification Experiments Conclusion

3 Delayed Packet Problem

4 TIME-WAIT State Blocking connections between the same address/port pair Holding a Time-Wait State at one endpoint 2MSL The end doing active close holds the TIME-WAIT State


6 TIME-WAIT state in the server
The server actively closes the connection, and maintains the TIME-WAIT state Server sending data TCP connection closure as end-of-transaction marker Simple protocol, fast response Otherwise: Knowing the content length Extra end-of-transaction marker requires masking & restoring the marker HTTP, FTP servers use TCP connection closure as an unambiguous end-of-transaction marker

7 Performance Problems on Busy Server
Too many connections in TIME-WAIT State TCB Consumes memory Slows active connections A shorter MSL weaken the protection Solutions like persistent connection are not enough

8 Solution: TIME-WAIT in Clients
Blocking at one end is enough Move the TIME-WAIT to the Client End points negotiating TCP Modification HTTP Modification Scales better than persistent connection

9 TIME-WAIT Negotiation
Non-simultaneous connection establishment SYN (TW-Negotiate) Client Server SYN, Ack (TW-Negotiate set to its choice) Ack (TW-Negotiate set to the same value) If Acceptable If not Acceptable RST If the server does not respond with TW-negotiate option, the current TCP semantic is applied.

10 Either Contains No Option
TIME-WAIT Negotiation Simultaneous connection establishment SYN (TW-Negotiate) SYN (TW-Negotiate) SYN-RCVD SYN-RCVD endpoint1 endpoint2 Value Known Value To Use Either Contains No Option No Option Same IP Address That IP Address Different IP Address Larger IP Address

11 TIME-WAIT Negotiation
Advantage Makes post-connection memory requirement explict Transparent to applications No hidden performance penalty Disadvantage Significant change to TCP state machine Information from connection establishment affects the closure Negotiating at closure reduces endpoints’ control over their resource Allow either endpoint to decline the connection if the overhead is unacceptable. Only move to the TIME-WAIT those clients willing to accept the overhead Significant programming and testing effort Correctness proof based on TCP state machine would be invalidated

12 TCP Level Solution Modify the clients’ TCP implementation
Sends a RST packet to the server Puts itself into a TIME-WAIT state Final Ack TW RST TW closed Client Server

13 Modified TCP State Machine

14 TCP Level Solution Only works for system that accepts <RST> in TIME-WAIT state Performance limited by the way <RST> is processed Changes the meaning of <RST> packet

15 Application Level Solution
Decouple the end-of-connection with end-of-transaction indication HTTP 1.1 Modification Content Length as the end of transaction New Extension Request: CLIENT_CLOSE The client opens a connection Sending series of requests Collecting the response Sending a Client-Close, closing connection The server closes its end without responding

16 CLIENT_CLOSE Extention
[Conection: close] line Client Server data CLIENT_CLOSE Detect the end of data close FIN . Last request: Connection: close data close FIN . Client Server

17 Application Level Solution
CLIENT-WAIT notify the server about the client close Requires change only on the client side Conforms to HTTP 1.1 Requires no change to other protocols Creates no new security vulnerability ONLY reduces the load by HTTP

18 Experiments Environment Three Tests Implemented under SUNOS 4.1.3
Evaluation using custom benchmark program and WebSTONE benchmark 640 Mb/s Myrinet LAN Three Tests Demonstration of Worst-case Server Loading HTTP Experiment Time wait avoidance and Persistent Connections

19 Demonstration of Worst-Case Server Loading
Test Configuration A Server Two clients doing simultaneous bulk transfer Server loaded with TIME-WAIT state by a fourth machine Constructing a worst case scenario Client connections are put at the end of the list of TCBs Two clients are used to neutralize the simple caching behavior Two distinct clients allow bursts from different clients to interleave The first experiment was designed to determine if TCB load reduces server throughput and if our modifications alleviate that effect;

20 Demonstration of Worst-Case Server Loading

21 HTTP Load Experiments Two workstations each running 20 clients
File Size: 9KB-5MB System Throughput Mb/s Conn/s TCP Mem (Kb) Unmodified 20.97 49.09 722.7 TCP Mods 26.40 62.02 23.1 HTTP Mods 31.73 74.70 23.4 8 clients on 4 workstations, small files System Throughput Mb/s Conn/s TCP Mem (Kb) Unmodified Fails TCP Mods 1.14 223.8 16.1 HTTP Mods 222.4

22 TIME-WAIT Avoidance and Persistent Connection
2 Client machine, 1 Server 5 HTTP request bursts 10 request / Connection TIME-WAIT avoidance methods increase the per-connection throughput as client load increases Throughput vs. Clients

23 TIME-WAIT Avoidance and Persistent Connection
Connection Rate vs. Clients

24 TIME-WAIT Avoidance and Persistent Connection
Memory vs. Clients

25 Conclusion TCP With TCP With CLIENT_CLOSE TIME-WAIT Client HTTP
Negotiation <RST> Extension Reduces TIME-WAIT Loading Yes Yes Yes Compatible With Current Protocols Yes Yes Yes Changes Are Effective If Only No Yes Yes The Client Is Modified Allows System To Prevent Yes No Yes TIME-WAIT Assassination No Changes To Transport Protocol No No Yes No Changes To Application Protocols Yes Yes No Adds No Packet Exchanges To Yes No No Modified Protocol TIME-WAIT Allocation Is A Requirement Yes No No of Connection Establishment

Download ppt "The TIME-WAIT state in TCP and its Effect on Busy Servers Theodore Faber University of Southern California Xindian Long."

Similar presentations

Ads by Google