2. Hyun-Jin Choi, CORE Lab. E.E. 1 httperf – A Tool for Measuring Web Server Performance Dec. 12. 2000 Choi, Hyun-Jin David Mosberger and Tai Jin HP Research Labs, 1998

3. Hyun-Jin Choi, CORE Lab. E.E. 2 Introduction httperfhttperf –cf : SPECweb or WebStone Simulate a fixed number of clients –Generates and sustains overload –Various kinds of web-server related measurements SPECweb/WebSton-like measurements Session-based measurements –Independent of shortcomings of current Oses

4. Hyun-Jin Choi, CORE Lab. E.E. 3 An Example of Using httperf Once a test finishes, several statistics are printedOnce a test finishes, several statistics are printed –Overall results –TCP connections –Send requests –Results for the replies –CPU & network utilization figures –errors Httperf --server hostname \ --port 80 --uri /test.html \ --rate 150 --num-conn 27000 \ --num-call 1 --timeout 5 Invoke httperf on each client machinesInvoke httperf on each client machines

5. Hyun-Jin Choi, CORE Lab. E.E. 4 Design Sustaining OverloadSustaining Overload –In case of server’s overload The client would run out of resources Example –Each httperf process : has at most 2,000 TCP connection –Offered rate - Sustained rate = 100 reqs/s –A test could last at most 20 secs Solution –Time out –Example »100 reqs/s, time out : 5 secs »At most 500 connections

6. Hyun-Jin Choi, CORE Lab. E.E. 5 Design (2) Limits to Client-Sustainable LoadLimits to Client-Sustainable Load –Size of TCP port space TCP port numbers : 16bits wide  64K available port numbers 1,024 are typically reserved for privileged processes –Ex : ftp – 21, telnet – 23 –At most 64,512 port numbers A given port number cannot be reused until the TCP TIME_WAIT state expires –1 minute timeout : BSD-derived OSes »Maximum sustainable rate per client : 1,075 reqs/s –4 minute timeout : RFC-793 recommended value »Maximum sustainable rate per client : 268 reqs/s

7. Hyun-Jin Choi, CORE Lab. E.E. 6 Design (3) –Number of open file descriptors Limitation of per-process # of file descriptors –256 ~ 2,048 –TCP TIME_WAIT state plays no role here –httperf timeout value »A value of 5 sec »A limit of 2,000 open file descriptors per process »400 reqs/s –Solution »Tuning the OS to allow a larger # of open file descriptors »Decreasing the httperf timeout value »Multiple processes : undesirable… later TP

8. Hyun-Jin Choi, CORE Lab. E.E. 7 Design (4) –Socket buffer memory Each TCP connection contains a socket receive and send buffer httperf : send buffers – 4KB, receive buffers – 16KB How much memory is available for socket buffers Example –40MB available for socket buffers –A client could sustain at most 2,048 concurrent TCP connections

9. Hyun-Jin Choi, CORE Lab. E.E. 8 Design (5) –Prediction of the exact rate at which a client will start to become the performance bottleneck ??? Empirical verification –Observed performance »A reflection of the server’s capacity ??? »Not that of the client ??? –Independence of the # of clients Measuring ThroughputMeasuring Throughput –httperf samples the reply throughput once every 5 secs

10. Hyun-Jin Choi, CORE Lab. E.E. 9 Implementation Three subsystems connected by event facilityThree subsystems connected by event facility –HTTP Core Engine Handles all communication with the server ~500 req/sec/machine Supports HTTP/1.1 –Workload Generators Request Generation –Generates new connections –Creates sessions URL Generation –Performance Monitors

11. Hyun-Jin Choi, CORE Lab. E.E. 10 Implementation (2) Implementation issuesImplementation issues –Scheduling Granularity Most OSes use a timer tick of around 10 ms In case of 150 reqs/s – below has problems –During even-numbered tick : Sending one request –During odd-numbered tick : Sending two requests –Unforced burstness httperf executes in a tight loop select() –Checks for network I/O activity via select() gettimeofday() –Keeps track of real time via gettimeofday() –One httperf process can be run per client machine

12. Hyun-Jin Choi, CORE Lab. E.E. 11 Implementation (3) –Limited Number of Ephemeral (short lived) Ports Typically in the range from 1,024 to 5,000 Example –TCP TIME_WAIT state timeout : 1 minutes –Maximum sustainable rate : 66 regs/s Solution –Optionally maintain its own bitmap of ports that it believes to be available »UNIX TCP implementation : allow pre-empting the TIME_WAIT state if a new SYN segment arrives »NT disallows such pre-emption  TCP RESET

13. Hyun-Jin Choi, CORE Lab. E.E. 12 Future Directions & Critique Its development has by no means come to a haltIts development has by no means come to a halt –Mimic the real-world traffic patterns –Allow log file based URL generation –Make it easier to run tests with multiple clients A daemon-based approach –Advantages Generation and sustenance of server’s overload Modularity : flexible, extensible –Disadvantages Not so good for testing huge server –Hasn’t good test equipment –One process – multiple threads