1. MS108 Computer System I Lecture 7 Tomasulos Algorithm Prof. Xiaoyao Liang 2014/3/24 1

2. 2 The Tomasulos Algorithm From IBM 360/91 Goal: High Performance using a limited number of registers without a special compiler – 4 double-precision FP registers on 360 – Uses register renaming Why Study a 1966 Computer? – The descendants of this include: Alpha 21264, HP 8000, MIPS 10000, Pentium III, PowerPC 604, …

3. 3 Tomasulo Algorithm Control & buffers are distributed with Function Units (FU) – FU buffers called reservation stations (RS) – Contain information about instructions, including operands – More reservation stations than registers, so can do optimizations compilers cant Registers in instructions replaced by values or pointers to reservation stations – form of register renaming – avoids WAR, WAW hazards Results to FU from RS, not through registers (equivalent of forwarding). A Common Data Bus (CDB) broadcasts results to all FUs (their RSes) Loads and Stores treated as FUs with RSes as well

4. 4 Tomasulo Organization FP adders Add1 Add2 Add3 FP multipliers Mult1 Mult2 From Mem FP Registers Reservation Stations Common Data Bus (CDB) To Mem FP Op Queue Load Buffers Store Buffers Load1 Load2 Load3 Load4 Load5 Load6

5. 5 Reservation Station Components Busy: Indicates reservation station or FU is busy Op: Operation to perform in the unit (e.g., + or –) Vj, Vk: Value of Source operands Qj, Qk: Reservation stations producing source registers (value to be written) – Note: Qj,Qk=0 => ready A: effective address Tomasulo Organization

6. 6 Register result status Qi – Indicates which functional unit will write each register, if one exists. Blank when no pending instructions that will write that register Common data bus – Normal data bus: data + destination (go to bus) – CDB: data + source (come from bus) 64 bits of data + 4 bits of Functional Unit source address Write if matches expected Functional Unit (produces result) Does the broadcast Tomasulo Organization

7. 7 Three Stages of Tomasulo Algorithm 1.Issueget instruction from FP Op Queue – If reservation station free (no structural hazard), control issues the instruction & sends operands (renames registers). 2.Executeoperate on operands (EX) – When both operands ready then execute; if not ready, watch Common Data Bus for result 3.Write resultfinish execution (WB) – Write on Common Data Bus to all awaiting units; mark reservation station available

8. 8 Tomasulo Loop Example Loop:LDF0,0(R1) MULTDF4,F0,F2 SDF4,0(R1) SUBIR1,R1,#8 BNEZR1,Loop This time assume multiply takes 4 clock cycles in the execution stage Assume 1st load takes 8 clock cycles (L1 cache miss) in the execution stage, 2nd load takes 1 extra cycle (hit) Assume store takes 3 cycles in the execution stage To be clear, will show clocks for SUBI, BNEZ Show about 2 iterations

9. 9 Loop Example using simplified presentation for load/store components Added Store Buffers Value of Register used for address, iteration control Instruction Loop Iter- ation Count Instruction status: ExecWrite ITERInstructionjkIssueCompResultBusyAddr Qk 1LDF00R1Load1No 1MULTDF4F0F2Load2No 1SDF40R1Load3No 2LDF00R1Store1No 2MULTDF4F0F2Store2No 2SDF40R1Store3No Reservation Stations: S1S2RS TimeNameBusyOpVjVkQjQkCode: Add1NoLDF00R1 Add2NoMULTDF4F0F2 Add3NoSDF40R1 Mult1NoSUBIR1 #8 Mult2NoBNEZR1Loop Register result status Clock R1 F0F2F4F6F8F10F12...F30 080 Qi

10. 10 Loop Example Cycle 1

11. 11 Loop Example Cycle 2

12. 12 Loop Example Cycle 3

13. 13 Loop Example Cycle 4 Dispatching SUBI Instruction (not in FP queue)

14. 14 Loop Example Cycle 5 And, BNEZ instruction (not in FP queue)

15. 15 Loop Example Cycle 6 Notice that F0 never sees Load from location 80

16. 16 Loop Example Cycle 7 Register file completely detached from computation First and Second iteration completely overlapped

17. 17 Loop Example Cycle 8

18. 18 Loop Example Cycle 9 Load1 completing: who is waiting? Note: Dispatching SUBI

19. 19 Loop Example Cycle 10 Load2 completing: who is waiting? Note: Dispatching BNEZ Instruction status: ExecWrite ITERInstructionjkIssueCompResultBusyAddr Qk 1LDF00R11910Load1No 1MULTDF4F0F22Load2Yes72 1SDF40R13Load3No 2LDF00R1610Store1Yes80Mult1 2MULTDF4F0F27Store2Yes72Mult2 2SDF40R18Store3No Reservation Stations: S1S2RS TimeNameBusyOpVjVkQjQkCode: Add1NoLDF00R1 Add2NoMULTDF4F0F2 Add3NoSDF40R1 4Mult1YesMultdM[80]R(F2)SUBIR1 #8 Mult2YesMultdR(F2)Load2BNEZR1Loop Register result status Clock R1 F0F2F4F6F8F10F12...F30 1064 Qi Load2Mult2

20. 20 Tomasulo Organization FP adders Add1 Add2 Add3 FP multipliers Mult1 Mult2 From Mem FP Registers Reservation Stations Common Data Bus (CDB) To Mem FP Op Queue Load Buffers Store Buffers Load1 Load2 Load3 Load4 Load5 Load6

21. 21 Loop Example Cycle 11 Next load in sequence

22. 22 Loop Example Cycle 12 Why not issue third multiply?

23. 23 Loop Example Cycle 13 Why not issue third store?

24. 24 Loop Example Cycle 14 Mult1 completing. Who is waiting?

25. 25 Loop Example Cycle 15 Mult2 completing. Who is waiting?

26. 26 Loop Example Cycle 16

27. 27 Loop Example Cycle 17

28. 28 Loop Example Cycle 18

29. 29 Loop Example Cycle 19

30. 30 Loop Example Cycle 20 Once again: In-order issue, out-of-order execution and out-of-order completion.

31. 31 Why can Tomasulo overlap iterations of loops? Register renaming – Multiple iterations use different physical destinations for registers (dynamic loop unrolling). Reservation stations – Buffer old values of registers - avoiding the WAR stall that we saw in the scoreboard. Other perspective: Tomasulo builds data flow dependency graph on the fly.

32. 32 Tomasulos scheme offers 2 major advantages (1)the distribution of the hazard detection logic – Distributed reservation stations and the CDB – If multiple instructions waiting on single result, the instructions can be released simultaneously by broadcast on CDB – If a centralized register file were used, the units would have to read their results from the registers when register buses are available. (2) the elimination of stalls for WAW and WAR hazards