1. Lecture 8: ILP and Speculation Contd. Chapter 2, Sections 2. 6, 2
Lecture 8: ILP and Speculation Contd. Chapter 2, Sections 2.6, 2.8 Section 2.7-VLIW Done earlier

2. Superscalar with Speculation
Speculative execution – execute control dependent instructions even when we are not sure if they should be executed
With branch prediction, we speculate on the outcome of the branches and execute the program as if our guesses were correct. Misprediction? Hardware undo
Instructions after the branch can be fetched and issued, but can not execute before the branch is resolved
Speculation allows them to execute with care.
Multi-issue + branch prediction + Tomasulo
Implemented in a number of processors:
PowerPC 603/604/G3/G4, Pentium II/III/4, Alpha 21264, AMD K5/K6/Athlon, MIPS R10k/R12k

3. Hardware Modifications
Speculated instructions execute and generate results. Should they be written into register file? Should they be passed onto dependent instructions (in reservation stations)?
Separate the bypassing paths from actual completion of an instruction. Do not allow speculated instructions to perform any updates that cannot be undone.
When instructions are no longer speculative, allow them to update register or memory – instruction commit.
Out-of-order execution, in-order commit (provide precise exception handling)
Then where are the instructions and their results between execution completion and instruction commit? Instructions may finish considerably before their commit.
Reorder buffer (ROB) holds the results of instructions that have finished execution but have not committed.
ROB is a source of operands for instructions, much like the store buffer

4. HW support for More ILP HW support for More ILP
Speculation: allow an instruction to issue that is dependent on branch predicted to be taken without any consequences (including exceptions) if branch is not actually taken (“HW undo”); called “boosting”
Combine branch prediction with dynamic scheduling to execute before branches resolved
Separate speculative bypassing of results from real bypassing of results
When instruction no longer speculative, write boosted results (instruction commit) or discard boosted results
execute out-of-order but commit in-order to prevent irrevocable action (update state or exception) until instruction commits

5. HW support for More ILP
Need HW buffer for results of uncommitted instructions: reorder buffer
3 fields: instr, destination, value
Reorder buffer can be operand source => more registers like RS
No more store buffers beforeMemory (Fig. 3.29)
Use reorder buffer number instead of reservation station when execution completes
Supplies operands between execution complete & commit
Once operand commits, result is put into register
Instructions commit in order
As a result, its easy to undo speculated instructions on mispredicted branches or on exceptions
Reorder
Buffer FP Op
Queue
FP Regs
Res Stations
Res Stations
FP Adder
FP Adder

6. Four Steps of Speculative Tomasulo Algorithm
1. Issue— get instruction from FP Op Queue
If reservation station and reorder buffer slot free, issue instr & send operands & reorder buffer no. for destination (this stage sometimes called “dispatch”)
2. Execution— operate on operands (EX)
When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute; checks RAW (sometimes called “issue”)
3. Write result— finish execution (WB)
Write on Common Data Bus to all awaiting FUs & reorder buffer; mark reservation station available.
4. Commit— update register with reorder result
When instr. at head of reorder buffer & result present, update register with result (or store to memory) and remove instr from reorder buffer. Mispredicted branch flushes reorder buffer (sometimes called “graduation”)

8. With Hardware Speculation

9. Additional Functionalities of ROB
Dynamically execute instructions while maintaining precise interrupt model.
In-order commit allows handling interrupts in-order at commit time
Undo speculative actions when a branch is mispredicted
In reality, misprediction is expected to be handled as soon as possible. Flushing all the entries that appear after the branch, allowing those preceding instructions to continue.
Performance is very sensitive to branch-prediction mechanism
Prediction accuracy, misprediction detection and recovery
Avoids hazards through memory (memory disambiguation)
WAW and WAR are removed since updating memory is done in order
RAW hazards are maintained by 2 restrictions:
A load’s effective address is computed after all earlier stores
A load can not read from memory if there is an earlier store in ROB having the same effective address (some machine simply bypass the value from store to the load)

10. Getting CPI < 1 Multiple Issue with Speculation

11. Example

12. Timing of Multiple Issue without Speculation with two CDBs

13. Timing of Multiple Issue with Speculation – with multiple CDBs and commit operations!
Note: Perfect branch prediction and speculation is assumed in Fig Otherwise the performance will be lower.

14. Register renaming, virtual registers versus Reorder Buffers
Alternative to Reorder Buffer is a larger virtual set of registers and register renaming
Virtual registers hold both architecturally visible registers + temporary values
replace functions of reorder buffer and reservation station
Renaming process maps names of architectural registers to registers in virtual register set
Changing subset of virtual registers contains architecturally visible registers
Simplifies instruction commit: mark register as no longer speculative, free register with old value
Adds extra registers: Alpha, Pentium,…
Size limits no. instructions in execution (used until commit)

15. How much to speculate?
Speculation Pro: uncover events that would otherwise stall the pipeline (cache misses)
Speculation Con: speculate costly if exceptional event occurs when speculation was incorrect
Typical solution: speculation allows only low-cost exceptional events (1st-level cache miss)
When expensive exceptional event occurs, (2nd-level cache miss or TLB miss) processor waits until the instruction causing event is no longer speculative before handling the event
Assuming single branch per cycle: future may speculate across multiple branches!

16. Pentium $ Microarchitecture, Fig. 2.26

17. Performance of 5 Integer and 5 Floating Point SPEC 2000 Benchmarks

20. 2.6 GHz AMD Opteron Vs 3.2 GHz Pentium 4