1. Web Server Benchmarking Using the Internet Protocol Traffic and Network Emulator Carey Williamson, Rob Simmonds, Martin Arlitt et al. University of Calgary

2. TeleSim : IP-TNE 2 Network Emulation A network emulator is a network simulator with an interface that allows client applications to interact with it in real-time Provides a reliable, repeatable test environment for distributed applications Internet games Multi-media/video conferencing Distributed file systems …

3. Simulation Real World

4. TeleSim : IP-TNE 4 IP-TNE Internet Protocol Traffic and Network Emulator The Internet Protocol Traffic and Network Emulator (IP-TNE) is a network emulator based on IP-TN Enables interaction between IP based clients via an IP-TN simulated network Distributed applications can interact with IP-TNE without modification

6. TeleSim : IP-TNE 6 IP-TNE Overview TasKit with real-time extensions provides an environment for “fast” network emulation IP-TNE provides routing methods suitable for shared environments and dedicated test environments Now has HTTP and TCP client models that can be used for Web server benchmarking

7. TeleSim : IP-TNE 7 Web Server Benchmarking: Objectives of Study Evaluate new approach to WAN emulation, and demonstrate feasibility Validate prior results by Nahum et al on effects of WAN conditions on Web server performance How fast can Apache Web server go? How fast can IP-TNE go?

8. TeleSim : IP-TNE 8 Experimental Setup IP-TNE on Compaq ES-40 (4 CPU) Apache (1.3.23) on another ES-40 Gigabit Ethernet (1 Gbps) in between Simple regular network topology model Experimental factors: N subnetworks, H client hosts per subnet, propagation delay D, link speed C, HTTP version…

9. TeleSim : IP-TNE 9 Web Workload Model Fixed-size Web objects Small (1 KB) Large (64 KB) Variable-size Web objects Median 3 KB Mean 9 KB Pareto heavy tail (alpha = 1.2) Zipf-like document popularity profile Note: static content only

10. TeleSim : IP-TNE 10 Performance Metrics Two primary metrics HTTP transaction rate (trans/sec) Network throughput (Mbps) Several secondary metrics Response time Failure rate Packet loss…

11. TeleSim : IP-TNE 11 Results Overview For 1 KB transfers with HTTP/1.0: Single client: 170 transactions/sec Transaction rate scales up with number of clients up to about H = 32 Transaction rate flattens, then drops sharply as num clients is increased more (closed loop) Peak rate achieved: 3800 trans/sec Peak throughput approximately 40 Mbps Transaction rate is (strongly) inversely related to the client round trip time (RTT)

12. TeleSim : IP-TNE 12 Results Overview (Cont’d) For 64 KB transfers with HTTP/1.0: Single client: 18 transactions/sec Transaction rate scales up with number of clients up to about H = 32 Transaction rate flattens, then drops slightly as num clients is increased more Peak rate achieved: 220 trans/sec Peak throughput approximately 115 Mbps Transaction rate is (weakly) inversely related to the client round trip time (RTT)

13. TeleSim : IP-TNE 13 Results Overview (Cont’d) For variable-size transfers with HTTP/1.0: Single client: 60 transactions/sec Transaction rate scales up with number of clients up to about H = 32 Transaction rate flattens, then drops as num clients is increased more Peak rate achieved: 1300 trans/sec Peak throughput approximately 90 Mbps Transaction rate is inversely related to the client round trip time (RTT) Behaviour is in between 1 KB and 64 KB results

14. TeleSim : IP-TNE 14 Results Overview (Cont’d) Concurrent connections with HTTP/1.0: Single client: 600 transactions/sec Qualitatively similar results to before, except that fewer clients are needed to drive the server to full load Conceptually concurrent connections are no different than adding more clients

15. TeleSim : IP-TNE 15 Results Overview (Cont’d) Persistent connections with HTTP/1.1: Single client: 300 transactions/sec Qualitatively similar results to before, except that transaction rate is about 70% higher than for HTTP/1.0 (since multiple HTTP req’s per TCP conn) Peak transaction rate 6500 trans/sec Much less dependency on RTT effects

16. TeleSim : IP-TNE 16 Results Overview (Cont’d) Pipelined persistent connections with HTTP/1.1: Single client: 800 transactions/sec Qualitatively similar results to before, except that transaction rate is about 100% higher than for HTTP/1.0 Peak transaction rate 7600 trans/sec Much less dependency on RTT effects

17. TeleSim : IP-TNE 17 Results Overview (Cont’d) Effect of WAN RTT delays: Increasing the per-link propagation delay increases the client RTT delay, which in turn reduces the transaction rate and throughput (as expected) As RTT increases, more and more clients are needed in order to drive the Web server to full load Similar to [Nahum et al. 2001]

18. TeleSim : IP-TNE 18 Results Overview (Cont’d) Effect of WAN packet losses: Decreasing the router queue size at the bottleneck link increases the packet loss ratio (as expected) As the level of packet loss increases, the HTTP transaction rate and the network throughput decrease Similar results to [Nahum et al. 2001]

19. TeleSim : IP-TNE 19 Results Overview (Cont’d) Effect of MTU size: The smaller the MTU size, the lower the transaction rate and network throughput Reasons: Smaller MTU size increases the relative header overhead per packet Smaller MTU size increases the number of packets that need to be sent Smaller MTU size (and MSS) increases the number of RTT’s required by TCP

20. TeleSim : IP-TNE 20 Results Overview (Cont’d) Effect of bandwidth asymmetry: For asymmetric access technologies such as ADSL (Asymmetric Digital Subscriber Line), the upstream link from the client to the server can sometimes be the bottleneck for TCP, even though it is primarily carrying ACKs only Depends on normalized bandwidth ratio Greater asymmetry, worse performance

21. TeleSim : IP-TNE 21 Summary The IP-TNE is a useful tool for Web server benchmarking Demonstrates feasibility of WAN emulation using a single computer Validates prior results by Nahum et al discussing the effects of WAN conditions on Web server performance Demonstrates performance advantages of HTTP/1.1

22. TeleSim : IP-TNE 22 Future Ideas with IP-TNE Benchmarking Web caching appliances Evaluating SRPT scheduling in WAN setting Connection/packet-level scheduling algorithms Evaluating CATNIP approach to TCP/IP Evaluating portable (wireless) Web servers Dynamic content (CGI, etc) Workload sensitivities (Zipf, Pareto, corr, mods) Asymmetric networks, Ensemble-TCP Parallel TCP connections: friend or foe? Evaluating effect of TCP SACK in WAN Validating IP-TNE (and IP-TN) Benchmarking IP-TNE vs IP-TNE