A Preliminary Attempt ECEn 670 Semester Project Wei Dang Jacob Frogget Poisson Processes and Maximum Likelihood Estimator for Cache Replacement

Outline Motivation Cache replacement algorithms Poisson processes in webserver-based cache Poisson processes in microprocessor-based cache Maximum Likelihood estimator Predicting procedures Simulation and evaluation Results compared to LRU Future work

Motivation Memory-processor speed gap is getting larger

One of the Solutions Memory hierarchy L1/L2/L3 closer to chip, less latency, but smaller size Cache replacement Cache is fast but limited in size conflict Good replacement policies needed on contention

Cache Replacement Algorithms Cache conflict in a 4-way associative cache Set index

Common Cache Replacement Algorithms Random Least Recently Used (LRU, most widely used) Least Frequently Used FIFO

Poisson Processes for webserver cache Arrival time of queries to a webserver can be modeled as a Poisson Process Interpretation: the probability of having k queries up to some point of time Assumption Arrivals of queries are independent of each other Not always true but valid for most cases

Poisson Processes for microprocessor cache Independence assumption invalid References to cache are highly correlated (especially to data cache) Temporal locality Spatial locality

Poisson Processes for microprocessor cache cont. One Poisson process for each block within a set Set index An example set from a 4-way associative cache

Poisson Processes for microprocessor cache cont. Correlation between these four random processes Set index Local counters to each block One global counter n for each set

Maximum Likelihood Estimator The estimate is the arithmetic mean of

Predicting Procedures Initially each block has Given a previous calculated for block, the estimated is calculated as: Probabilities for replacement is then Replace block with lowest probability Choose randomly on equal probabilities

Simulation and Evaluation Simulator: MyDLX cache simulator from EE628 Metrics: miss rate for Instruction-cache and Data-Cache Various associativities Five benchmarks Compared to LRU

Results

Results not encouraging Sometimes 0% miss rate for both algorithms (might be due the inherent characteristics of benchmarks) Statistical approach worse than LRU for most cases Getting worse for higher associativity (more blocks to predict)

Analysis of deficiencies of our model Independence model may be inaccurate (even accesses to the same block within a set may not be independent) Local counter is reset to 0 on eviction (history eliminated)

Future work and challenges More accurate model with more correlation parameters for each Poisson process Implementation complexity (hardware expensive; LRU is already expensive at high associativity) May be implemented as a software cache as a supplement to hardware cache.

The End Thank you! Questions?