1. \course\cpeg324-08F\Topic7c
Cache Parameters
Cache size : Scache (lines)
Set number: N (sets)
Line number per set: K (lines/set)
Scache = KN (lines)
= KN * L (bytes)  Here L is line size in bytes
K-way set-associative
2018/9/22
\course\cpeg324-08F\Topic7c

2. Trade-offs in Set-Associativity
Fully-associative:
- Higher hit ratio, concurrent search, but slow access when associativity is large.
Direct mapping:
- Fast access (if hits) and simplicity for comparison.
- Trivial replacement algorithm.
Problem with hit ratio, e.g. in extreme case: if alternatively use 2 blocks which mapped into the same cache block frame: “trash” may happen.
2018/9/22
\course\cpeg324-08F\Topic7c

3. \course\cpeg324-08F\Topic7c
Note
Main memory size: Smain (blocks)
Cache memory Size: Scache (blocks)
Let P = Since P >>1.
Average search length is much greater than 1.
Set-associativity provides a trade-off between:
Concurrency in search.
Average search/access time per block.
You need search!
Smain
Scache
2018/9/22
\course\cpeg324-08F\Topic7c

4. \course\cpeg324-08F\Topic7c
N Scache
<
Full
associative
Set
Direct
Mapped
Number of sets
2018/9/22
\course\cpeg324-08F\Topic7c

5. Important Factors in Cache Design
Address partitioning strategy
(3-dimention freedom).
Total cache size/memory size
Work load
2018/9/22
\course\cpeg324-08F\Topic7c

6. \course\cpeg324-08F\Topic7c
Address Partitioning
M bits
Log N
Log L
Set number byte address in a line
Byte addressing mode
Cache memory size data part = NKL (bytes)
Directory size (per entry)
M - log2N - log2L
Reduce clustering (randomize accesses)
set size
2018/9/22
\course\cpeg324-08F\Topic7c

7. \course\cpeg324-08F\Topic7c
Note: The exists a knee
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Cache Size
0.34
Miss Ratio
General Curve Describing Cache Behavior
2018/9/22
\course\cpeg324-08F\Topic7c

8. \course\cpeg324-08F\Topic7c
…the data are sketchy and highly dependent on the method of gathering...
… designer must make critical choices using a combination of “hunches, skills, and experience” as supplement…
“a strong intuitive feeling concerning a future event or result.”
2018/9/22
\course\cpeg324-08F\Topic7c

9. \course\cpeg324-08F\Topic7c
Basic Principle
Typical workload study + intelligent estimate of others
Good Engineering: small degree over-design
“30% rule”:
Each doubling of the cache size reduces misses by 30% by Alan J. Smith. Cache Memories. Computing Surveys, Vol. 14., No 13, Sep 1982.
It is a rough estimate only.
2018/9/22
\course\cpeg324-08F\Topic7c

10. \course\cpeg324-08F\Topic7c
K: Associativity
Bigger  Miss ratio
Smaller is better in:
Faster
Cheaper
4 ~ 8 get best miss ratio
Simpler
2018/9/22
\course\cpeg324-08F\Topic7c

11. \course\cpeg324-08F\Topic7c
L : Line Size
Atomic unit of transmission
Miss ratio
Smaller
Larger average delay
Less traffic
Larger average hardware cost for associative search
Larger possibility of “Line crossers”
Workload dependent
16 ~ 128 byte
Memory references spanning the boundary between two cache lines
2018/9/22
\course\cpeg324-08F\Topic7c

12. Cache Replacement Policy
FIFO (first-in-first-out) replace the block loaded furthest in the past
LRU (least-recently used) replace the block used furthest in the past
OPT (furthest-future used) replace the block which will be used furthest in the future.
Do not retain lines that have next occurrence in the most distant future
Note: LRU performance is close to OPT for frequently encountered program structures.
2018/9/22
\course\cpeg324-08F\Topic7c

13. Example: Misses and Associativity
Small cache with four one-word blocks. Sequence 0, 8, 0, 6 and 8.
Direct Mapped Cache.
Blue text Data used in time t.
Black text Data used in time t-1.
5 misses for the 5 accesses
2018/9/22
\course\cpeg324-08F\Topic7c

14. Example: Misses and Associativity (cont’d)
Small cache with four one-word blocks. Sequence 0, 8, 0, 6 and 8.
Two-way set-associative. LRU replacement policy
Blue text Data used in time t.
Black text Data used in time t-1.
4 misses for the 5 accesses
2018/9/22
\course\cpeg324-08F\Topic7c

15. Example: Misses and Associativity (cont’d)
Small cache with four one-word blocks. Sequence 0, 8, 0, 6 and 8.
Fully associative Cache.
Any memory block can be stored in any cache block.
Blue text Data used in time t.
Black text Data used in time t-1.
Red text  Data used in time t-2.
3 misses for the 5 accesses
2018/9/22
\course\cpeg324-08F\Topic7c

16. \course\cpeg324-08F\Topic7c
Program Structure
for i = 1 to n
for j = 1 to n
endfor
Last-in-first-out feature makes the recent past likes the near future ….
2018/9/22
\course\cpeg324-08F\Topic7c

17. \course\cpeg324-08F\Topic7c
Problem with LRU
Not good in mimic sequential/cyclic
Example
ABCDEF ABC…… ABC……
Exercise: With a set size of 3, what is the miss ratio assuming all 6 addresses mapped to the same set ?
2018/9/22
\course\cpeg324-08F\Topic7c

18. Performance Evaluation Methods for Workload
Analytical modeling.
Simulation
Measuring
2018/9/22
\course\cpeg324-08F\Topic7c

19. Cache Analysis Methods
Hardware monitoring:
Fast and accurate.
Not fast enough (for high-performance machines).
Cost.
Flexibility/repeatability.
2018/9/22
\course\cpeg324-08F\Topic7c

20. Cache Analysis Methods
cont’d
Cache Analysis Methods
Address traces and machine simulator:
Slow.
Accuracy/fidelity.
Cost advantage.
Flexibility/repeatability.
OS/other impacts - How to put them in?
2018/9/22
\course\cpeg324-08F\Topic7c

21. Trace Driven Simulation for Cache
Workload dependence:
Difficulty in characterizing the load.
No general accepted model.
Effectiveness:
Possible simulation for many parameters.
Repeatability.
2018/9/22
\course\cpeg324-08F\Topic7c

22. Problem in Address Traces
Representative of the actual workload (hard)
Only cover a small fraction of real workload.
Diversity of user programs.
Initialization transient
Use long enough traces to absorb the impact of cold misses
Inability to properly model multiprocessor effects
2018/9/22
\course\cpeg324-08F\Topic7c