Cache Changes Partition Size Was 8GB limit Write pointer per Partition == lots of seeks Ram Cache per Partition == poor LRU CacheDir now 10 bytes (up from 8) 25% more memory/TB disk for directory Supports.5PB Partitions with 512 sectors 2ZB with 4k sectors One/Disk unless partition.config/hosting Sizing now 2.25 GB RAM / TB Disk
Cache Changes II Large Object Support Offsets/sizes ink64 (except fragment size) Add do_io_pread() For efficient Range Incomplete (testing/ability to call repeatedly) Upped Aggregation sizes (4MB) Other Protocol Support Add set_header/get_header
Cache Changes III Lock Contention Fewer partitions means fewer locks So, do not hold the Part lock over SM callbacks Future/Discuss: (for later) take or try+retry locks? Thread Efficiency Schedule on initial (open_XXX) thread if possible AIO now configurable to callback on Calling thread/AIO thread/ANY Ethread Combined with Event changes = <1msec hit/miss
Buffers Support Large Objects > 2GB All sizes/offsets ink64 VIO, MIOBuffer, VConnection Upped max fast buffer size to 64K
Net Edge and Level Triggered IO Support for Libev (optional) Native epoll (linux) Native kqueue (FreeBSD) Native port (Solaris) Use eventfd/pipe to wake threads (>latency) Use new Queue interface with link field offset in template (see List.h)
Regressions We should write them and run them Try traffic_server –R1 New RegressionSM Supports sequential/parallel composition Need to get them run daily and on buildbot Performance Regression We should have them We should track them
Cache Future & Discussion Support for other protocols (?) Efficient support for very large objects Supporting writes w/o copy to Aggregation buffer Efficient support for very small objects Cluster support for changes 4K native block sizes This ties in with small objects Separate Aggregation Header and Object Header Allow multiple Objects in a 4K block
Locking Discussion Locking model was try/reschedule Reason: bad thread libraries/schedulers New thread libraries and OS support better Rescheduling == latency Rescheduling makes contention obvious Switch to take lock over small sections?
Events Standard Libraries Callback-based, no fit with current design Libev Nice clean and very efficient Doesn’t support edge-triggering Libevent A bit messy not quite as efficient 1.X and 2.X-alpha have different interfaces Our current system puts IO events in Net Lacks generality, confusing, messy
Events II Options Fix our system Less risk/changes More flexible Reduces dependencies Adopt a library Let someone else handle the problem Which library? Do we include source and hack it? Do we expose the library API? What interactions? What do we need to decide?
Cleanup Not consistent Lots of #ifdefs Lots of unused variables/functions/etc. Naming Ink everywhere Why ink_xxx and UpperCased.h headers? Directory Structure/Modules Proxy/XXX ? What is there and why? Do we want modules? Where they go? Dynamic linking? API External vs Internal API?