1. Caches: reducing miss penalty Prof. Eric Rotenberg
ECE 463/563 Fall `18
Caches: reducing miss penalty
Prof. Eric Rotenberg
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

2. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
Reduce Miss Penalty
L2 cache
Victim cache
Write buffer
Early restart, critical word first
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

3. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
L2 cache
Effects of increasing L1 cache size
Decrease L1 miss rate
Increase L1 hit time
Instead, reduce average miss penalty by “catching” some L1 capacity misses as hits in a larger cache between L1 cache and main memory
AAT = HTL1 + MRL1* MP
Acronyms:
HT: hit time
MR: miss rate
MP: miss penalty
AAT = HTL1 + MRL1 * MPL1
MPL1 = HTL2 + MRL2 * MPL2
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

4. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
L2 cache example
Example HT MR MP L1
1 cycle
0.05
it depends L2
4 cycles
0.01
100 cycles
L1 cache only
L1 and L2 caches
MPL1
100 cycles
5 cycles
( *100)
AAT
6 cycles
( *100)
1.25 cycles
( *5)
L1 cache
main memory
100 cycles
L1 cache
L2 cache
main memory
4 cycles
100 cycles
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

5. ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg
Victim cache
See previous pptx
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

6. Write misses and the processor
Processor doesn’t need to wait for write miss
It’s not asking for data
Don’t stall processor
Do the write in the background
Need a write buffer
Processor delegates responsibility to the write buffer, for performing the write to the memory hierarchy
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

7. Write buffer in WBWA Scenario 1: write hit
L1 cache
WBWA policy
write request from processor
Next level in mem. hier.
(L2 cache or main memory)
write hit
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

8. Write buffer in WBWA Scenario 2: write miss
L1 cache
WBWA policy
data
write request from processor
read block from next level
address
Next level in mem. hier.
(L2 cache or main memory)
write miss
merge
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

9. Write buffer in WTNA Scenario 1: write hit
L1 cache
WTNA policy
data
write request from processor
write to next level
address
Next level in mem. hier.
(L2 cache or main memory)
write hit
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

10. Write buffer in WTNA Scenario 2: write miss
L1 cache
WTNA policy
data
write request from processor
write to next level
address
Next level in mem. hier.
(L2 cache or main memory)
write miss
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

11. Complication: write followed by dependent read
A read that hits in the cache is ok
Since the block is in the cache, any previous writes to the block have completed
A read that misses in the cache needs special treatment
It must check the write buffer for any pending writes that it depends on
If its address matches a pending write’s address, read gets data directly from write buffer
Otherwise it could get stale data from the next level if the younger read completes before the older write
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

12. Read miss checks if it depends on any pending writes in the write buffer
read request from processor
read miss
L1 cache
(WBWA or WTNA)
read address
data
address =? =? =? =?
Next level in mem. hier.
(L2 cache or main memory)
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg

13. Early restart, Critical word first
These techniques shave off a few cycles from the miss penalty by not waiting for all bytes of the block
Early Restart:
As soon as the requested word arrives, forward it to the processor
Processor resumes while the rest of the block is filled into the cache
Early Restart + Critical Word First:
Problem still with Early Restart: What if requested word is in the middle of the block?
Critical Word First: Memory system returns requested word first, followed by other words
Fall 2018
ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg