1. Ka-Ming Keung Swamy D Ponpandi
Trace Cache
Ka-Ming Keung
Swamy D Ponpandi

2. Lecture Topics Trace Cache Block based trace cache Optimizations
CPRE 585 Fall 2004

3. Motivation
Complex superscalar processor design to extract higher ILP -diminishing returns
Issues – Fetch bandwidth
CPRE 585 Fall 2004

4. Producer-Consumer View
Branch outcome/Jump address
Instruction
Fetch & Decode
Instruction
Execution
This is a producer consumer view of superscalar processor, we can see that the entire processor can be divided
Into an fetch unit and execution unit. It is clear that to maintain the instruction execution throughput, fetch unit has to keep the execution unit busy. That is, if execution unit is able to accept instructions at an rate r, then the fetch unit has to provide instructions at the same rate
if fetch rate < r, under utlilization, low IPC
fetch rate > r, fetch unit stalls
Another thing to note is EU actually directs the FU to fetch, hence FU rate is actually dependent on EU’s outcome
Of course FU could speculate.
Instruction Buffers
CPRE 585 Fall 2004

5. Fetch Bandwidth Branch Outcome Cache Hit rate Branch Throughput
Instruction Alignment
Fetch bandwidth depends on many factors including
Branch outcome – lot of research, we have looked at BHT, target address prediction (BTB)etc
Cache hit rate – dependent on memory hierarchy, locality of program etc
We will concentrate on branch throughput and instruction alignment, this is because, with increase in issue width multiple branches have to be predicted in the same cycle to help the FU to fetch more instructions
Multiple branch prediction introduces another problem, fetching from multiple non-contiguous locations in cache memory which necessitates multiple cache banks (may lead to conflicts), multiports (costly). This increases fetch unit latency which in turn has impact on the instruction execution rate (lower clock, pipeline stalls ).
We got to fetch a branch and get the basic block if it is predicted to be taken in the same cycle (how is it possible?)
Alignment – fetch non-contiguous blocks, rotate, shift to create the dynamic sequence of instructions
CPRE 585 Fall 2004

6. The Trace Cache Capture dynamic instruction sequences – Snapshot
Cache line – N instructions and M basic blocks
Dynamic instruction stream
trace{A:taken,taken}
trace{A:taken,taken}
later …. A A t t t t
Fill New Trace from Instruction cache
Access existing trace using A and predictions (t,t)
To alleviate the problems mentioned in the previous slide , trace cache is proposed to capture the dynamic sequence of instructions.
Dynamic instruction sequence – can start at any address and continue down up to M taken branches (why not taken branches)?
Each cache line is fully specified by the starting address and the order of M-1 taken branches.
TRACE CACHE
TRACE CACHE t t t t A A
1st basic block
3rd basic block (still filling)
TO DECODER
2nd basic block
CPRE 585 Fall 2004

7. Pros and Cons
Deliver multiple blocks of instructions in the same cycle without support from a compiler and modifying instruction set
Not on critical path, complexity is moved out of the fetch-issue pipeline where additional latency impacts performance
CPRE 585 Fall 2004

8. Pros and Cons
No need to rotate, shift basic blocks to create dynamic instruction sequence
Redundant Information
CPRE 585 Fall 2004

9. Trace Cache – Dissection
Fetch Address
Trace Cache
Core Fetch Unit
n instructions
n instructions
Line fill
2:1 Mux
Instruction latch
To Decoder
CPRE 585 Fall 2004

10. Trace Cache – More details
Fetch address
from decoder
Trace Cache
Core fetch unit
Branch flags
Target addr
Merge logic
Tag
Branch mask
Instruction Cache
Fill
Control
Branch
Target
Buffer
Line fill buffer A 11
11,1 X Y
Return
Address
Stack
Predictor
1st branch
2nd branch
BTB Logic
fall through
3rd branch
Hit Logic
mask/shift/interchange
To TC
next fetch
address
n instructions
n instructions
Fetch address
from predictor
CPRE 585 Fall 2004
To instruction latch for decoding

11. Trace Cache Component Valid bit Tag Branch flags Branch mask
Trace fall-through address
Trace target address
Line-Fill Buffer
Line Fill Buffer
It services trace cache misses
The control logic merges each incoming block of instructions with preceding instructions in the line-fill buffer
Filling is complete when
1. n instructions have benn traced, or
2. m branches have been detected
After filling, contents are moved to the trace cache
Branch flags and branch mask are generated during the line-fill process
The trace target and fall-through address are computed at the end of the line-fill
CPRE 585 Fall 2004

12. Trace Cache Hit Conditions
1. The fetch address match the tag
2. The branch predictions match the branch flags
If Hits, an entire trace of instructions is fed into the instruction
CPRE 585 Fall 2004

13. Trace Cache Miss
If Misses, fetching proceeds normally from the instruction cache
CPRE 585 Fall 2004

14. Design Issues Associativity Multiple paths Partial matches
Indexing methods
Cache line size
Filling
Trace selection
Victim trace cache
CPRE 585 Fall 2004

15. Results
CPRE 585 Fall 2004

16. CPRE 585 Fall 2004

17. Trace cache effectiveness
CPRE 585 Fall 2004

18. Pentium 4 Trace Cache
6 uops per line (what is the benefit of storing as uops?)
Max 12 k uops (21 KB)
Uses virtual addresses so no need for address translation
CPRE 585 Fall 2004

19. Block-Based Trace Cache
Fetch Address renaming
Advantage: Lesser bits
Unique renamed pointer is assigned to each address
No Tag Comparison, Faster instruction fetch
Rename table which is maintained at completion
Move the complexity and latency from instruction fetch time to completion time
CPRE 585 Fall 2004

20. Block Instructions are stored on a block.
Each block contains a trace of instruction
Block is updated at the end of each instruction execution
Each block has a block ID
CPRE 585 Fall 2004

21. How to find out the trace?
Original: Fetched address
Now: Trace id
Trace id is determined by
the branch history, and
Past block id predictions
Trace table stores the block id predictions
CPRE 585 Fall 2004

22. Rename Table Check if the predicted block exists
Give a block id to a new trace
CPRE 585 Fall 2004

23. Dynamic Optimizations
Register Moves
Register move does not require calculation
Can be performed in ROB and renaming unit
Trace Cache can do it in a better way
CPRE 585 Fall 2004

24. Register Moves Advantages
By Placing detection logic in fill unit, the decode and rename logic execute move instruction without having to pay for latency of detecting them
Requires no execution resources, thus incur no delays due to artifacts.
CPRE 585 Fall 2004

25. Re-association Decreases the dependency addi Rx,Ry,4 addi Rz,Rx,4
addi Rz, Ry,8
CPRE 585 Fall 2004

26. Scaled Adds Decreases the dependency
shifti Rw, Rx << shifti Rw, Rx << 1
add Ry, Rw, Rz scaled Add Ry, (Rx<<1), Rz
Decreases the dependency
CPRE 585 Fall 2004

27. Instruction Placement
Many instructions are often unable to execute the cycle after their source operands are produced.
Reorder the instructions within the trace cache
Because they must wait for the value to be forwarded from the producting functional unit to the awaiting instruction’s FU
CPRE 585 Fall 2004

28. Conclusion Increases IPC New area for code optimization
CPRE 585 Fall 2004