1. CS526 - Advanced Internet & Web Systems May 11, 2009 Nathan Archer

2. Background  Websites compete based on usability & responsiveness  Offline load tests may not present realistic traffic.  End-to-end response times may not provide sufficient troubleshooting information.

3. Scope  Confine research to LAMP server (Linux, Apache, MySQL, PHP)  Monitoring methods should be minimally- intrusive- something that could realistically be used on a production system.

4. Basic Web Performance Tests  A typical test:  Perform repeated “HTTP-GET” requests on a set of URLs & report measurements such as:  File transfer time & throughput rate  Connection time  Number of concurrent connections  Port-open time  Service-connection time  Limitations:  Real traffic is less homogenous (fewer cache hits in real life)  Little troubleshooting information (what caused the delay?)

5. Why use logs?  Already well-integrated with system (low overhead, low development cost)  Can provide more detail for troubleshooting than end-to-end response times.  Another purpose: logs can be analyzed to produce real-world profiles of user requests  Determine how users are really using the website-what content is most important?  Use data to create realistic traffic profiles for off-line load testing  Example tools: Analog, Webalyzer, etc. etc.

6. Drawbacks of Using Logging  Logging adds overhead, worsening the performance we are measuring  High-detail information may require some effort to customize scripts & search through log files

7. Potential Bottlenecks (1/2)  Network  Web server  CPU & RAM limitations  Caching scheme  Multi-threading configuration  Network interface  File system  Hard disk  SAN or Network File System

8. Potential Bottlenecks (2/2)  Application server  Scripting/programming technologies  Database  Security  SSL  Firewall

9. Apache Logging Features  Use mod_log_config module to log time for each request.  Add “%D” to the LogFormat directive to show time to serve the request (microsec.)  Example: LogFormat "%v %h %t \"%r\" %>s %b bytes in %d usec.“ common_format

10. Adding More Detail  Use CustomLog rule to filter on request:  To filter on only PHP requests:  SetEnvIf Request_URI "(\.php)$" php_request  CustomLog logs/imageAccess.log common env=php_request  To filter on everything but HTML files:  SetEnvIf Request_URI "(\.html|\.shtml)$" html_req  CustomLog logs/nonHTMLAccess.log common env=!html_req  http://www.serverwatch.com/tutorials/article.php/10825_3376671_2#condlogging

11. PHP Performance Logging  No PHP logging solution available without running a debugger.  However, we can customize PHP scripts to report run-time using microtime command.  Example timer class found at: http://www.phpclasses.org/browse/file/19094.html  Call start() & stop() to run timer.  Add custom logging code in your own script. http://www.phpguru.org/article/129

12. MySQL Performance Logging  MySQL’s binary log keeps track of how long queries took.  To enable, invoke MySQL service with –log-bin=mylogfile option.  We can edit the config file to log the slowest queries- for example, those over 2 seconds:  [mysqld] log_slow_queries=/var/log/mysql_slow_query.log long_query_time=2  http://homepage.mac.com/kelleherk/iblog/C711669388/E1631032272/index.html

13. Putting it all Together: A Proposed Monitoring Scheme  Web server components each publish logs on a shared network folder.  Every night, run a shell script to find the slowest requests for each server component from the logs.  If any Apache responses are > 2 sec., send an email alerting the webmaster.

14. Questions?