1. Choosing A Proxy Server OSCON 2014 Bryan Call Yahoo

2. About Me Apache – Committer and PMC for Apache Traffic Server (ATS) – Apache Member IETF – HTTP/2 – TLS – TCP Enhancement Yahoo – WebRing, GeoCities, People Search, Personals, Tiger Team, Platform Architect, Edge Group, Network Research, Apache/IETF

3. Overview Types of Proxies Features Architecture Cache Architecture Performance Pros and Cons

4. How are you going to use a proxy server?

5. Reverse Proxy

6. Proxy in front of your own web servers Caching? Geographic location? Connection handling? SSL termination? SPDY support? Adding business logic?

7. Forward Proxy

8. Intercepting Proxy

9. Forward / Intercepting Proxy Proxy in front of the Internet Configure clients to use proxy? Caching? SSL - CONNECT? SSL - termination?

10. Choices

11. Plenty of Proxy Servers PerlBal

12. Plenty of Proxy Servers

13. Features And Options

14. Features ATSNGiNXSquidVarnishApache httpd mod_proxy Reverse ProxyYYYYY Forward ProxyYNYNY Transp. ProxyYNYNY Plugin APIsYYpartialYY CacheYYYYY ESIYNYpartialN ICPYNYNN SSLYYYNY SPDYYYNNpartial

15. SSL Features Source: https://istlsfastyet.com/ - Ilya Grigorik

16. What type of proxy do you need? Of our candidates, only three fully supports all proxy modes

17. HTTP/1.1 Compliance

18. Accept-Encoding - gzip Vary Age If-None-Match

19. How things can go wrong: Vary $ curl -D - -o /dev/null -s --compress http://10.118.73.168/ HTTP/1.1 200 OK Server: nginx/1.3.9 Date: Wed, 12 Dec 2012 18:00:48 GMT Content-Type: text/html; charset=utf-8 Content-Length: 8051 Connection: keep-alive Cache-Control: public, max-age=900 Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000 Expires: Sun, 19 Nov 1978 05:00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip

20. How things can go wrong: Vary $ curl -D - -o /dev/null -s http://10.118.73.168/ HTTP/1.1 200 OK Server: nginx/1.3.9 Date: Wed, 12 Dec 2012 18:00:57 GMT Content-Type: text/html; charset=utf-8 Content-Length: 8051 Connection: keep-alive Cache-Control: public, max-age=900 Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000 Expires: Sun, 19 Nov 1978 05:00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip EPIC FAIL! Note: no gzip request

21. CoAdvisor HTTP protocol quality tests for reverse proxies 49% 81% 51% 68%

22. 25% 6% 27% 15%

23. Architecture

24. Architecture And Process Models Multithreading Events Process Fibers – Co-operative multitasking, getcontext/setcontext

25. Threads

26. Pros – Easy to share memory – Lightweight context switching Cons – Easy to (accidently) share memory Overwriting another threads memory – Locking Deadlocks, race conditions, starvation

27. Event Processing

28. Problems with Event Processing Doesn’t work well with blocking APIs – open(), locking It doesn’t scale on SMP by itself

29. Process Model And Architecture ATSNGiNXSquidVarnishApache httpd mod_proxy ThreadsXXX EventsXXXpartialX ProcessesXXX

30. Caching Architecture

31. Cache Mainly two types – File system – Database like In memory index – Bytes per object Minimize disk seeks and system calls

32. Cache ATSNGiNXSquidVarnishApache httpd mod_cache File systemXXX mmapX Raw disk/direct IOXX Ram cacheXX Memory indexXXX* Persistent cacheXXXX

33. Performance Testing

34. ATS Configuration etc/trafficserver/remap.config: map / http://origin.example.com etc/trafficserver/records.config: CONFIG proxy.config.http.server_ports STRING 80 CONFIG proxy.config.accept_threads INT 3

35. NGiNX Configuration worker_processes 24; access_log logs/access.log main; proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m; proxy_temp_path /mnt/nginx_temp; server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; } location / { proxy_pass http://origin.example.com; proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; } location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }

36. Squid Configuration http_access allow all http_port 80 accel workers 24 cache_mem 4096 MB memory_cache_shared on cache_dir rock /usr/local/squid/cache 1000 max-size=32768 cache_peer origin.example.com parent 80 0 no-query originserver

37. Varnish Configuration backend default {.host = ”origin.example.com”;.port = "80"; }

38. Varnish Configuration (Cont) sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10

39. Apache httpd Configuration LoadModule cache_module modules/mod_cache.so LoadModule cache_disk_module modules/mod_cache_disk.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so Include conf/extra/httpd-mpm.conf ProxyPass / http://origin.example.com/ CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3 MaxKeepAliveRequests 10000

40. Benchmark 1 1,000 clients 8KB response 100% cache hit Keep-alive on 100K rps rate limited

41. Squid used the most CPU and the worst median latency 95 th percentile latency with NiGNX, Squid and httpd

42. Benchmark 2 1,000 clients 8KB response 100% cache hit Keep-alive off

43. Squid used the most CPU again NGiNX had latency issues ATS most throughput

44. ATS Pros – Scales well automatically, little config needed – Best cache implementation Cons – Too many config files – Too many options in the default config files (5.x fixed this)

45. NGiNX Pros – Lots of plugins – FastCGI support Cons – HTTP/1.1 compliance – Latency issues around accepting new connections – Rebuild server for new plugins

46. Squid Pros – Best HTTP/1.1 compliance Cons – Memory index for cache using 10x that of ATS – Least efficient with CPU – Worst median latency for keep-alive benchmarks

47. Varnish Pros – VCL (Varnish Configuration Language) Can do a lot without writing plugins Cons – Thread per connection – mmap for cache Persistence is experimental – No SSL or SPDY support

48. Apache httpd Pros – Lots of plugins – Most used http server – Best 95 th percentile latency for non-keep-alive Cons – SPDY Support

49. Why ATS? Scales well – CPU Usage, auto config Cache scales well – Efficient memory index, minimizes seeks Apache Community Plugin support – Dynamic loading of plugins and easy to port existing plugins over

50. References ATS - http://trafficserver.apache.org/ NGiNX - http://nginx.org/ Squid - http://www.squid-cache.org/ Varnish - https://www.varnish-cache.org/ Apache httpd - http://httpd.apache.org/