ULC: An Unified Placement and Replacement Protocol in Multi-level Storage Systems Song Jiang and Xiaodong Zhang College of William and Mary

Multi-Level Buffer Caching in Distributed Systems client network Front-tier serverend-tier server disk array

Challenges to Improve Hierarchy Performance LRU L1 L2 L3 L4 LRU (1) Can the hit rate of hierarchical caches achieve the hit rate of a single first level cache with its size equal to the aggregate size of the hierarchy? (2) Can we make caches close to clients contribute more to the hit rate? L1 80% 40% 50%10% 50%

Reason I: Weakened Locality at Low Level Caches Low level caches hold the misses from their upper level buffer caches, and the hits have high latency. The requests with strong locality have been filtered by the high level buffer caches close to clients.

An Existing Solution: Re-designing Low Level Cache Replacement Algorithms To overcome weak locality, MQ is a frequency-based replacement; Once a block is accessed, it is promoted to a higher queue. Periodically, blocks in each queue are checked and low frequency blocks are demoted to lower queues. Multi-Queue Replacement (MQ) [USENIX01]... Q0Q0 Q1Q1 QnQn Q out

Drawbacks of MQ Replacement Inheriting the weakness of frequency-based algorithm – not responsive to access pattern changes Containing workload sensitive parameters; Cannot fully exploit the locality knowledge inherent in applications (accurate information is in high level caches) Motivation: Locality analysis is conducted at clients, where original requests are generated.

Both caches Server cache client Cache Snapshots at every 1000 references Reason II: Undiscerning Redundancy among Levels of Buffer Caches

Another Existing Solution: Extending Existing Replacement into an Unified Replacement For example: Unified LRU (uniLRU) [USENIX02] L1 LRU stack L2 LRU stack Demotion 10 Client Server 10 6

Drawbacks of Unified LRU ……. L1 LRU stack L2 LRU stack All the hits go to this L2 position 1)High level caches are not well utilized 2)Large demotion overhead;

Our Approach: Unified Level-aware Caching (ULC) Blocks with weak locality are placed in the low level buffer caches (1) Locality is analyzed at client. (2) The analysis results are used to direct the placement of blocks in the hierarchy. Minimizing redundancy among levels of the buffer caches by unified replacement based on client information. Locality strength Cache levels

Quantifying Locality Strength LAD-R = max (LAD, R) Next Access Position Last Access Position Current Position LAD LAD-R R Unified LRU Stack Locality strength is characterized by Next Access Distance (NAD); NAD is unknown currently; NAD is quantitatively predicted by Last Access Distance (LAD) and Recency (R). NAD LAD LAD-R R Next Access Position Last Access Position Current Position NAD Advantages of LAD-R over R 1.not change until the next reference of the block 2.Accurate quantification

Multi-Level Buffer Caching Protocol ---- Unified and Level-Aware Caching (ULC) ULC running on the first level client dynamically ranks the accessed blocks according to their LAD-R values. Based on the ranking results, blocks are cached (placed) at levels L1, L2, …, accordingly. Low level caches take actions such as caching or replacing according to the instructions from clients.

LAD-R Based Block Caching Exactly arranging block layout as LAD-R ranking is expensive (at least O(logn)) Efficient two-phase LAD-R Based caching (O(1)): 1)LAD determines block placement at the time of retrieval (R = 0); 2)R is used for block replacement after a block is cached.

LAD-R Based Placement and Replacement L1 LRU stack L2 LRU stack The LRU position at which a block is accessed determines its placement The current LRU position determines its replacement 2 7

ULC Data Structure uniLRU Stack Y1 Y2 Y3 Yardstick Recency Status R1 R2 R3 L L2 L3 Level Status Recency Status is determined by recency Level Status is determined by LAD The placement of a block is determined by its level status The yardstick block is the one for replacement at the corresponding level

Two Operations in the ULC Protocol Two request messages from the client to the low level caches: 1.Retrieve (b, i, j) ( i j ): retrieve block b from level Li, and cache it at level Lj when it passes level Lj on its route to level L1. 2.Demote (b, i, j) (i < j): demote block b from level Li into level Lj.

uniLRU Stack Y1 Y2 Y3 L L2L3 Level Status Retrieve (11, From, To) R2 Block 11 L3L2 R2, accessed Retrieve (11, 3, 2 )

uniLRU Stack Y1 Y2 Y3 L L2L3 Level Status 11 Retrieve (11, 3, 2) Demote (6, 2, 3) 6 6

ULC with Multiple Clients L1 Block Lout block Y1 Yardstick 9 5 Y2 3 6 Y1 Yardstick 9 Client 1 Client 2 Global_LRU at Server L2 Block L1L2 L1L2 Y2

Performance Evaluation: Workload Traces RANDOM: spatially uniform distribution of references; (synthetic traces) ZIPF: highly skewed reference distribution; (synthetic traces) HTTPD collected on a 7-node parallel web-server. (HP) DEV1 collected in an office environment for over 15 consecutive days. (HP) TPCC1 the I/O trace of the TPC-C database benchmark. (IBM DB2)

Performance on a 3-level Structure Block size: 8KB Block transfer time between the client and the server buffer caches: 1ms Block transfer time between the server buffer cache and its RAM cache on disk: 0.2ms Block transfer time between the disk RAM cache and the disk: 10ms Cache size: 100MB each

1)Compared with indLRU, ULC significantly increases hit ratios; 2)Compared with uniLRU, ULC providse better hit distribution;

1)indLRU has high miss penalty; 2)uniLRU has high demotion cost;

Performance on a Multi-client Structure httpd collected on a 7 -node parallel web-server. openmail : collected on 6 HP 9000 K580 servers running HP OpenMail application. db2 collected on an 8 -node IBM SP2 system running an IBM DB2 database. Block size: 8KB Block transfer time between the clients and the server: 1ms Block transfer time between the server buffer cache and the disk: 10ms Cache size: 100MB each (except for workload tpcc1, which is 50MB)

The effect of cache pollution in MQ

Large demotion cost in uniLRU

Summary We propose an effective way to quantify locality in multi-level caches; We design an efficient block placement / replacement protocol (ULC); ULC makes the layout of cached blocks in the hierarchy matches their locality; Experiments show that ULC significantly outperform exiting schemes.