Choosing A Proxy Server OSCON 2014 Bryan Call Yahoo

About Me Apache – Committer and PMC for Apache Traffic Server (ATS) – Apache Member IETF – HTTP/2 – TLS – TCP Enhancement Yahoo – WebRing, GeoCities, People Search, Personals, Tiger Team, Platform Architect, Edge Group, Network Research, Apache/IETF

Overview Types of Proxies Features Architecture Cache Architecture Performance Pros and Cons

How are you going to use a proxy server?

Reverse Proxy

Proxy in front of your own web servers Caching? Geographic location? Connection handling? SSL termination? SPDY support? Adding business logic?

Forward Proxy

Intercepting Proxy

Forward / Intercepting Proxy Proxy in front of the Internet Configure clients to use proxy? Caching? SSL - CONNECT? SSL - termination?

Choices

Plenty of Proxy Servers PerlBal

Plenty of Proxy Servers

Features And Options

Features ATSNGiNXSquidVarnishApache httpd mod_proxy Reverse ProxyYYYYY Forward ProxyYNYNY Transp. ProxyYNYNY Plugin APIsYYpartialYY CacheYYYYY ESIYNYpartialN ICPYNYNN SSLYYYNY SPDYYYNNpartial

SSL Features Source: - Ilya Grigorik

What type of proxy do you need? Of our candidates, only three fully supports all proxy modes

HTTP/1.1 Compliance

Accept-Encoding - gzip Vary Age If-None-Match

How things can go wrong: Vary $ curl -D - -o /dev/null -s --compress HTTP/ OK Server: nginx/1.3.9 Date: Wed, 12 Dec :00:48 GMT Content-Type: text/html; charset=utf-8 Content-Length: 8051 Connection: keep-alive Cache-Control: public, max-age=900 Last-Modified: Wed, 12 Dec :52: Expires: Sun, 19 Nov :00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip

How things can go wrong: Vary $ curl -D - -o /dev/null -s HTTP/ OK Server: nginx/1.3.9 Date: Wed, 12 Dec :00:57 GMT Content-Type: text/html; charset=utf-8 Content-Length: 8051 Connection: keep-alive Cache-Control: public, max-age=900 Last-Modified: Wed, 12 Dec :52: Expires: Sun, 19 Nov :00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip EPIC FAIL! Note: no gzip request

CoAdvisor HTTP protocol quality tests for reverse proxies 49% 81% 51% 68%

25% 6% 27% 15%

Architecture

Architecture And Process Models Multithreading Events Process Fibers – Co-operative multitasking, getcontext/setcontext

Threads

Pros – Easy to share memory – Lightweight context switching Cons – Easy to (accidently) share memory Overwriting another threads memory – Locking Deadlocks, race conditions, starvation

Event Processing

Problems with Event Processing Doesn’t work well with blocking APIs – open(), locking It doesn’t scale on SMP by itself

Process Model And Architecture ATSNGiNXSquidVarnishApache httpd mod_proxy ThreadsXXX EventsXXXpartialX ProcessesXXX

Caching Architecture

Cache Mainly two types – File system – Database like In memory index – Bytes per object Minimize disk seeks and system calls

Cache ATSNGiNXSquidVarnishApache httpd mod_cache File systemXXX mmapX Raw disk/direct IOXX Ram cacheXX Memory indexXXX* Persistent cacheXXXX

Performance Testing

ATS Configuration etc/trafficserver/remap.config: map / etc/trafficserver/records.config: CONFIG proxy.config.http.server_ports STRING 80 CONFIG proxy.config.accept_threads INT 3

NGiNX Configuration worker_processes 24; access_log logs/access.log main; proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m; proxy_temp_path /mnt/nginx_temp; server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; } location / { proxy_pass proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; } location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }

Squid Configuration http_access allow all http_port 80 accel workers 24 cache_mem 4096 MB memory_cache_shared on cache_dir rock /usr/local/squid/cache 1000 max-size=32768 cache_peer origin.example.com parent 80 0 no-query originserver

Varnish Configuration backend default {.host = ”origin.example.com”;.port = "80"; }

Varnish Configuration (Cont) sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10

Apache httpd Configuration LoadModule cache_module modules/mod_cache.so LoadModule cache_disk_module modules/mod_cache_disk.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so Include conf/extra/httpd-mpm.conf ProxyPass / CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3 MaxKeepAliveRequests 10000

Benchmark 1 1,000 clients 8KB response 100% cache hit Keep-alive on 100K rps rate limited

Squid used the most CPU and the worst median latency 95 th percentile latency with NiGNX, Squid and httpd

Benchmark 2 1,000 clients 8KB response 100% cache hit Keep-alive off

Squid used the most CPU again NGiNX had latency issues ATS most throughput

ATS Pros – Scales well automatically, little config needed – Best cache implementation Cons – Too many config files – Too many options in the default config files (5.x fixed this)

NGiNX Pros – Lots of plugins – FastCGI support Cons – HTTP/1.1 compliance – Latency issues around accepting new connections – Rebuild server for new plugins

Squid Pros – Best HTTP/1.1 compliance Cons – Memory index for cache using 10x that of ATS – Least efficient with CPU – Worst median latency for keep-alive benchmarks

Varnish Pros – VCL (Varnish Configuration Language) Can do a lot without writing plugins Cons – Thread per connection – mmap for cache Persistence is experimental – No SSL or SPDY support

Apache httpd Pros – Lots of plugins – Most used http server – Best 95 th percentile latency for non-keep-alive Cons – SPDY Support

Why ATS? Scales well – CPU Usage, auto config Cache scales well – Efficient memory index, minimizes seeks Apache Community Plugin support – Dynamic loading of plugins and easy to port existing plugins over

References ATS - NGiNX - Squid - Varnish - Apache httpd -