1. Lecture 6 Score Board And Tomasulo’s Algorithm
Nov. 2, 2004
Lec. 7

2. Three Parts of the Scoreboard
1. Instruction status—which of 4 steps the instruction is in (Issue, Operand Read, EX, Write)
2. Functional unit status—Indicates the state of the functional unit (FU). 9 fields for each functional unit
Busy—Indicates whether the unit is busy or not
Op—Operation to perform in the unit (e.g., + or –)
Fi—Destination register
Fj, Fk—Source-register numbers
Qj, Qk—Functional units producing source registers Fj, Fk
Rj, Rk—Flags indicating when Fj, Fk are ready and not yet read. Set to
No after operand are read.
3. Register result status—Indicates which functional unit will write each register, if one exists. Blank when no pending instructions will write that register
What you might have thought
1. 4 stages of instruction executino
2.Status of FU: Normal things to keep track of (RAW & structura for busyl):
Fi from instruction format of the mahine (Fi is dest)
Add unit can Add or Sub
Rj, Rk - status of registers (Yes means ready)
Qj,Qk - If a no in Rj, Rk, means waiting for a FU to write result; Qj, Qk means wihch FU waiting for it
3.Status of register result (WAW &WAR)s:
which FU is going to write into registers
Scoreboard on 6600 = size of FU
6.7, 6.8, 6.9, 6.12, 6.13, 6.16, 6.17
FU latencies: Add 2, Mult 10, Div 40 clocks
Nov. 2, 2004
Lec. 7

3. Detailed Scoreboard Pipeline Control
Read operands
Execution complete
Instruction status
Write result
Issue
Wait until
Bookkeeping
Not busy (FU) and not result(D)
Busy(FU)¬ yes; Op(FU)¬ op; Fi(FU)¬ `D’; Fj(FU)¬ `S1’; Fk(FU)¬ `S2’; Qj¬ Result(‘S1’); Qk¬ Result(`S2’); Rj¬ not Qj; Rk¬ not Qk; Result(‘D’)¬ FU;
WAW
Rj and Rk
Rj¬ No; Rk¬ No
Functional unit done
"f((Fj( f )!=Fi(FU) or Rj( f )=No) & (Fk( f )!=Fi(FU) or Rk( f )=No))
"f(if Qj(f)=FU then Rj(f)¬ Yes); "f(if Qk(f)=FU then Rj(f)¬ Yes); Result(Fi(FU))¬ 0; Busy(FU)¬ No
Rj,Rk==No  1. Source operands are not ready. OR
2. Source operands are ready and have been read
WAR
A.55 on page A-76
Nov. 2, 2004
Lec. 7

4. Scoreboard Example
The following numbers are to illustrate behavior, not representative
LD – 1 cycle
(compute address + data cache access)
ADDDs and SUBs are 2 cycles
Multiply is 10 cycles
Divide is 40 cycles
Nov. 2, 2004
Lec. 7

5. Scoreboard Example
Nov. 2, 2004
Lec. 7

6. Scoreboard Example Cycle 1
Nov. 2, 2004
Lec. 7

7. Scoreboard Example Cycle 2
Note: Can’t issue I2 because Integer unit is busy. Can’t issue next instruction due to in-order issue
Nov. 2, 2004
Lec. 7

8. Scoreboard Example Cycle 3
Nov. 2, 2004
Lec. 7

9. Scoreboard Example Cycle 4
Nov. 2, 2004
Lec. 7

10. Scoreboard Example Cycle 5
Now I2 is issued
Nov. 2, 2004
Lec. 7

11. Scoreboard Example Cycle 6
Nov. 2, 2004
Lec. 7

12. Scoreboard Example Cycle 7
I3 stalled at read because I2 isn’t complete
Nov. 2, 2004
Lec. 7

13. Scoreboard Example Cycle 8
Nov. 2, 2004
Lec. 7

14. Scoreboard Example Cycle 9
Note: I3 and I4 read operands because F2 is now available. ADDD (I6) can’t be issued because SUBD (I4) uses the adder
Nov. 2, 2004
Lec. 7

15. Scoreboard Example Cycle 11
Note: Add takes 2 cycles, so nothing happens in cycle 10. MUL continues.
Nov. 2, 2004
Lec. 7

16. Scoreboard Example Cycle 12
Nov. 2, 2004
Lec. 7

17. Scoreboard Example Cycle 13
Now ADDD is issued because SUBD has completed
Nov. 2, 2004
Lec. 7

18. Scoreboard Example Cycle 14
Nov. 2, 2004
Lec. 7

19. Scoreboard Example Cycle 15
Note: ADDD takes 2 cycles, so no change
Nov. 2, 2004
Lec. 7

20. Scoreboard Example Cycle 16
ADDD completes, but MULTD and DIVD go on
Nov. 2, 2004
Lec. 7

21. Scoreboard Example Cycle 17
ADDD stalls, can’t write back due to WAR with DIVD. MULT and DIV continue
Nov. 2, 2004
Lec. 7

22. Scoreboard Example Cycle 18
MULT and DIV continue
Nov. 2, 2004
Lec. 7

23. Scoreboard Example Cycle 19
MULT completes after 10 cycles
Nov. 2, 2004
Lec. 7

24. Scoreboard Example Cycle 20
MULTD completes and writes to F0
Nov. 2, 2004
Lec. 7

25. Scoreboard Example Cycle 21
Now DIVD reads because F0 is available
Nov. 2, 2004
Lec. 7

26. Scoreboard Example Cycle 22
ADDD writes result because WAR is removed.
Nov. 2, 2004
Lec. 7

27. Scoreboard Example Cycle 61
DIVD completes execution
Nov. 2, 2004
Lec. 7

28. Scoreboard Example Cycle 62
Execution is finished
Nov. 2, 2004
Lec. 7

29. Review: Scoreboard Limitations of 6600 scoreboard DIV.D F0, F2, F4
No forwarding
Limited to instructions in basic block (small window)
Large number of functional units (structural hazards)
Stall on WAR hazards
Stall on WAW hazards
DIV.D F0, F2, F4
ADD.D F6, F0, F8
S.D F6, 0(R1)
SUB.D F8, F10, F14
MUL.D F6, F10, F8
WAR
WAW
Output dependence
Antidependence
Name dependence
Nov. 2, 2004
Lec. 7

30. Another Dynamic Algorithm: Tomasulo Algorithm
For IBM 360/91 about 3 years after CDC 6600
Goal: High Performance without special compilers
Differences between Tomasulo Algorithm & Scoreboard
Control & buffers distributed with Function Units vs. centralized in scoreboard; called “reservation stations”
Registers in instructions replaced by pointers to reservation station buffer
HW renaming of registers to avoid WAW hazards
Buffer operand values to avoid WAR hazards
Common Data Bus broadcasts results to all FUs
Load and Stores treated as FUs as well
Why study? Lead to Alpha 21264, HP 8000, MIPS 10000, Pentium II, Power PC 604 …
Nov. 2, 2004
Lec. 7

31. FP unit and load-store unit using Tomasulo’s alg.
Nov. 2, 2004
Lec. 7

32. Another Dynamic Algorithm: Tomasulo Algorithm
DIV.D F0, F2, F4
ADD.D S, F0, F8
S.D S, 0(R1) register renaming
SUB.D T, F10, F14
MUL.D F6, F10, T
Implemented through reservation stations (rs) per functional unit
Buffers an operand as soon as it is available – avoids WAR hazards.
Pending instr. designate rs that will provide their inputs – avoids WAW hazards.
The last write in a sequence of same-register-writing actually updates the register
Decentralize hazard detection and execution control
Instruction results are passed directly to the FU from rs rather than from registers
Through common data bus (CDB)
Nov. 2, 2004
Lec. 7

33. Three Stages of Tomasulo Algorithm
1. Issue—get instruction from FP Op Queue
Stall if structural hazard, ie. no space in the rs. If reservation station (rs) is free, the issue logic issues instr to rs & read operands into rs if ready (Register renaming => Solves WAR). Make status of destination register waiting for this latest instn even if the previous instn writing to this register hasn’t completed => Solves WAW hazards.
2. Execution—operate on operands (EX)
When both operands are ready then execute; if not ready, watch CDB for result – Solves RAW
3. Write result—finish execution (WB)
Write on Common Data Bus to all awaiting units; mark reservation station available. Write result into dest. reg. if its status is r. => Solves WAW.
Normal data bus: data + destination (“go to” bus)
CDB: data + source (“come from” bus)
64 bits of data + 4 bits of Functional Unit source address
Write if matches expected Functional Unit (produces result)
Does broadcast
Nov. 2, 2004
Lec. 7

34. Reservation Station Components
Op—Operation to perform in the unit (e.g., + or –)
Vj, Vk— Value of the source operand.
Qj, Qk— Name of the RS that would provide the source operands. Value zero means the source operands already available in Vj or Vk, or is not necessary.
Busy—Indicates reservation station or FU is busy
Register File Status Qi:
Qi —Indicates which functional unit will write each register, if one exists. Blank (0) when no pending instructions that will write that register meaning that the value is already available.
Nov. 2, 2004
Lec. 7

35. Tomasulo Example Cycle 0
Nov. 2, 2004
Lec. 7

36. Tomasulo Example Cycle 1
Nov. 2, 2004
Lec. 7

37. Tomasulo Example Cycle 2
Nov. 2, 2004
Lec. 7

38. Tomasulo Example Cycle 3
Nov. 2, 2004
Lec. 7

39. Tomasulo Example Cycle 4
Nov. 2, 2004
Lec. 7

40. Tomasulo Example Cycle 5
Nov. 2, 2004
Lec. 7

41. Tomasulo Example Cycle 6
Nov. 2, 2004
Lec. 7

42. Tomasulo Example Cycle 7
Nov. 2, 2004
Lec. 7

43. Tomasulo Example Cycle 8
Nov. 2, 2004
Lec. 7

44. Tomasulo Example Cycle 9
Nov. 2, 2004
Lec. 7

45. Tomasulo Example Cycle 10
Nov. 2, 2004
Lec. 7

46. Tomasulo Example Cycle 11
Nov. 2, 2004
Lec. 7

47. Tomasulo Example Cycle 12
Nov. 2, 2004
Lec. 7

48. Tomasulo Example Cycle 15
Nov. 2, 2004
Lec. 7

49. Tomasulo Example Cycle 16
Nov. 2, 2004
Lec. 7

50. Tomasulo Example Cycle 56
Nov. 2, 2004
Lec. 7

51. Tomasulo Example Cycle 57
Nov. 2, 2004
Lec. 7

52. Branch Prediction (3.4, 3.5)
Nov. 2, 2004
Lec. 7

53. Branch Prediction Easiest (static prediction) Next easiest
Always taken, always not taken
Opcode based
Displacement based (forward not taken, backward taken)
Compiler directed (branch likely, branch not likely)
Next easiest
1 bit predictor – remember last taken/not taken per branch
Use a branch-prediction buffer or branch-history table
Use part of the PC (low-order bits) to index buffer/table
Multiple branches may share the same bit
Invert the bit if the prediction is wrong
Backward branches for loops will be mispredicted twice
Nov. 2, 2004
Lec. 7

54. Q: Assume a loop branch is taken nine times in a row, then not taken once. What is the prediction accuracy using 1-bit predictor?
A: After first loop, the predictor will say not to take because the last time the execution came out of loop, it set a “0” in the predictor. So, it’s a misprediction. The bit will now be set to “1”. Works fine until the last loop when it is predicted as taken. So, 2 mispredictions in in 10 loop executions => 80% accuracy.
How about a 2-bit predictor? Let the prediction be changed only after it misses twice in a row.
Nov. 2, 2004
Lec. 7

55. 2-bit Branch Prediction
Has 4 states instead of 2, allowing for more information about tendencies
A prediction must miss twice before it is changed
Good for backward branches of loops
Nov. 2, 2004
Lec. 7

56. Branch History Table BHT Has limited size 2 bits by N (e.g. 4K)
4K same as infinite, see Fig. 3.9
Uses low-order bits of branch PC to choose entry 01
BHT
branch PC
Nov. 2, 2004
Lec. 7

57. Can we do better ?
Correlating branch predictors also look at other branches for clues
if (aa==2) T
aa = 0
if (bb==2) T
bb = 0
if(aa!=bb) { … NT
Prediction if the last branch is NT
Prediction if the last branch is T
(1,1) predictor – uses history of 1 branch and uses a 1-bit predictor
Nov. 2, 2004
Lec. 7

58. Correlating Branch Predictor
If we use 2 branches as histories, then there are 4 possibilities (T-T, NT-T, NT-NT, NT-T).
For each possibility, we need to use a predictor (1-bit, 2-bit).
And this repeats for every branch.
(2,2) branch prediction
Nov. 2, 2004
Lec. 7

59. Performance of Correlating Branch Prediction
With same number of state bits, (2,2) performs better than noncorrelating 2-bit predictor.
Outperforms a 2-bit predictor with infinite number of entries
Nov. 2, 2004
Lec. 7

60. General (m,n) Branch Predictors
The global history register is an m-bit shift register that records the last m branches encountered by the processor
Usually use both the PC address and the GHR (2-level)
m-bit ghr PC
Combining
funciton 01
n-bit predictors 00
Nov. 2, 2004
Lec. 7

61. Is Branch Predictor Enough?
When is using branch prediction beneficial?
When the outcome is known later than the target
For example, in our standard MIPS pipeline, we compute the target in ID stage but testing the branch condition incur a structure hazard in register file.
If we predict the branch is taken and suppose it is correct, what is the target address?
Need a mechanism to provide target address as well
Can we eliminate the one cycle delay for the 5-stage pipeline?
Need to fetch from branch target immediately after branch
Nov. 2, 2004
Lec. 7

62. Branch Target Buffer (BTB)
Is the current instruction a branch ?
• BTB provides the answer before the current instruction is decoded and therefore enables fetching to begin after IF-stage .
What is the branch target ?
• BTB provides the branch target if the prediction is a taken direct branch (for not taken branches the target is simply PC+4 ) .
Nov. 2, 2004
Lec. 7