Analysis and Performance Information Systems 337 Prof. Harry Plantinga

Analysis Your boss says "Who's visiting our site? Where are they coming from? What pages are popular? What paths are users taking through the site? What are the problem pages where we lose visitors?" Your boss says "Who's visiting our site? Where are they coming from? What pages are popular? What paths are users taking through the site? What are the problem pages where we lose visitors?" Analysis options: Analysis options: Read the raw logfile Read the raw logfile Set up a logfile analysis program Set up a logfile analysis program Sign up for Google Analytics Sign up for Google Analytics

Google Analytics An amazing tool… An amazing tool… Where do my customers live? Where do my customers live? What path do they take through the site? What path do they take through the site? What links are popular? What links are popular? Would a certain change improve the site? (A/B testing) Would a certain change improve the site? (A/B testing) How to add Google Analytics to a web site? How to add Google Analytics to a web site? Sign up for GA, install the google_analytics module Sign up for GA, install the google_analytics module

Suppose… You set up a new web site using Drupal You set up a new web site using Drupal Performance is mediocre Performance is mediocre What was a site in the top 5% (for speed) drops near the bottom… What was a site in the top 5% (for speed) drops near the bottom… 5 seconds for a page load 5 seconds for a page load 0.75sec minimum for every file served 0.75sec minimum for every file served high server load averages high server load averages occasionally, load average shoots up and server crashes occasionally, load average shoots up and server crashes What to do? What to do?

Minimize download time Reduce the number of hits (stylesheets, images) per pageview Reduce the number of hits (stylesheets, images) per pageview Use Drupal's CSS and JS optimization (although they didn't exist in Drupal 4…) Use Drupal's CSS and JS optimization (although they didn't exist in Drupal 4…) [ Reduced from >30 to about 20 files per pageview (stylesheets, artwork, etc) this way] Can also minimize javascript Can also minimize javascript Limit 2 files at once per server Limit 2 files at once per server Any way to improve this? Any way to improve this?

Second domain name Add a second server on a different domain name Add a second server on a different domain name Serve static files (artwork, stylesheets, etc) from a separate server Serve static files (artwork, stylesheets, etc) from a separate server Allow 4 files to be downloaded at once between the two domain names Allow 4 files to be downloaded at once between the two domain names Can turn on maximum caching for static files Can turn on maximum caching for static files

Caching Turn on Drupal's page cache Turn on Drupal's page cache Make sure users' browser cache works as well as possible Make sure users' browser cache works as well as possible Static content should never expire Static content should never expire Change filename for revised files Change filename for revised files Static server: add an Expires or Cache-control header: make sure content doesn't expire Static server: add an Expires or Cache-control header: make sure content doesn't expire Still, 40-60% of visitors come with an empty cache… Still, 40-60% of visitors come with an empty cache…

Optimize artwork Make sure artwork files are small Make sure artwork files are small optimize in Photoshop or another program optimize in Photoshop or another program optimize Many small files == slow. How to reduce? Many small files == slow. How to reduce? combine artwork into one background image combine artwork into one background imageone background imageone background image use jQuery to round corners use jQuery to round cornersround cornersround corners use CSS Sprites use CSS SpritesCSS SpritesCSS Sprites use inline images in stylesheets with data: urls use inline images in stylesheets with data: urlsdata: urlsdata: urls

Reduce download times 80-90% of wait time is downloading content 80-90% of wait time is downloading content More ideas More ideas Compress components Compress components Accept-Encoding: gzip, deflate? Then deflate! Accept-Encoding: gzip, deflate? Then deflate! Apache: use mod_deflate Apache: use mod_deflate Use a content delivery network Use a content delivery network E.g. Akami E.g. Akami Can improve av. response time 20% or more Can improve av. response time 20% or more

Make pages appear faster Stylesheets at the top, so pages can be rendered right away Stylesheets at the top, so pages can be rendered right away Scripts at the bottom Scripts at the bottom Specify image dimensions Specify image dimensions Minimize expensive page components Minimize expensive page components

Page Speed Google Page Speed tools help identify ways to improve performance Google Page Speed tools help identify ways to improve performancePage Speed Page Speed Firebug add-on for Firefox Firebug add-on for Firefox Apache module mod_pagespeed Apache module mod_pagespeed

Problems remain? At this point we still had very slow page display times and occasional load average spike/crash At this point we still had very slow page display times and occasional load average spike/crash How to debug? How to debug? What are the possible server performance bottlenecks? What are the possible server performance bottlenecks?

Server Bottlenecks Possible server bottlenecks Possible server bottlenecks Is the CPU maxed out? Is the CPU maxed out? Running out of RAM Running out of RAM Disk speed, transactions per second Disk speed, transactions per second Network bandwidth Network bandwidth Database capacity Database capacity How to test? How to test?

Disk maxed out? Check disk activity (reads vs. writes) Check disk activity (reads vs. writes) vmstat on linux vmstat on linux performance monitor on windows performance monitor on windows or write your own… or write your own… If pegged, find cause If pegged, find cause excessively verbose logging? excessively verbose logging? background process, e.g. backups? background process, e.g. backups? poorly configured database? poorly configured database? too many database writes? too many database writes? some other bad algorithm? some other bad algorithm?

Running out of bandwidth? Get a bigger pipe Get a bigger pipe Or, send less data Or, send less data smaller artwork smaller artwork better caching better caching gzip compression gzip compression

Running out of RAM? Check with linux top or free, performance monitor, etc. Check with linux top or free, performance monitor, etc. Each server process takes MB or more Each server process takes MB or more Each Apache or Drupal module takes more Each Apache or Drupal module takes more Our crashes were due to running out of RAM, then swap space Our crashes were due to running out of RAM, then swap space Solutions? Solutions? Set MaxClients appropriately Set MaxClients appropriately Turn down Keepalive time! (Default 15, to 2?) Turn down Keepalive time! (Default 15, to 2?) MaxRequestsPerChild to lower number, e.g. 300 MaxRequestsPerChild to lower number, e.g. 300 Get more RAM Get more RAM

Running out of CPU? Check with top (linux/mac) or performance monitor (windows) Check with top (linux/mac) or performance monitor (windows) PHP is interpreted PHP is interpreted Each request: load, compile all the code… Each request: load, compile all the code… This was taking us something like 0.5 secs of CPU time per page This was taking us something like 0.5 secs of CPU time per page Use a PHP Accelerator (bytecode cache) Use a PHP Accelerator (bytecode cache)PHP Accelerator PHP Accelerator e.g. eAccelerator, Zend, Alternative PHP Cache e.g. eAccelerator, Zend, Alternative PHP Cache Application profiling Application profiling Get a faster CPU Get a faster CPU

Database swamped? Use mysqladmin to figure out how many transactions per second you are using Use mysqladmin to figure out how many transactions per second you are using How many transactions per second can a database process? How many transactions per second can a database process? fully cached: lots fully cached: lots requiring a disk access: 100? 200? requiring a disk access: 100? 200? Solutions: Solutions: optimize expensive queries optimize expensive queries add indexes on tables add indexes on tables enlarge server's query cache / tune server enlarge server's query cache / tune server reduce database writes reduce database writes

Drupal-specific optimizations Some improvements you can make in Drupal: Some improvements you can make in Drupal: Turn on page caching Turn on page caching Stylesheet optimization Stylesheet optimization Prune the sessions table/make sessions shorter Prune the sessions table/make sessions shorter Reduce time before garbage collection (cron) Reduce time before garbage collection (cron) Prune error reporting logs (watchdog table) Prune error reporting logs (watchdog table) Automatic throttling Automatic throttling

What next? I've done all that. What else can I do? I've done all that. What else can I do? Separate database server Separate database server Load balancer and additional servers Load balancer and additional servers Database replication Database replication