1. IBM System 360. Common architecture for a set of machines
IBM System Common architecture for a set of machines. Robert Tomasulo worked on a high-end machine, the Model 91 (1967), on which they implemented his algorithm (today’s topic).

2. COMP 740: Computer Architecture and Implementation
Montek Singh
Oct 24, 2016
Topic: Instruction-Level Parallelism III
(Dynamic Scheduling: Tomasulo’s Algorithm)

3. Today’s Topic Tomasulo’s algorithm for dynamic scheduling
more sophisticated than scoreboarding
will enable scheduling multiple iterations of a loop (but still limited, more to come)
Reading: Ch

4. Dynamic Scheduling: Tomasulo’s Algorithm
For IBM 360/91 (about three years after CDC 6600)
long mem latency (before caches!)
Goal: High performance without special compilers
Small number of floating point registers (4 in 360 architecture) prevented interesting compiler scheduling of operations
Led Tomasulo to try to figure out how to get more effective registers — renaming in hardware!
Why Study 1967 Computer?
Technique languished until 1990s, then rediscovered
The descendants have flourished
Alpha 21264, Pentium 4, AMD Opteron, Power 5, Intel Core i3/i5/i7 …

5. Tomasulo’s Algorithm Differences between IBM 360 and CDC 6600 ISA
IBM has only 2 register specifiers/instruction versus 3 in CDC 6600
IBM has 4 FP registers versus 8 in CDC 6600
Differences between Tomasulo Algorithm and Scoreboard
Control and buffers distributed with Function Units versus centralized in scoreboard; called “reservation stations”
Registers in instructions replaced by pointers to reservation stations
this is called register renaming
renaming helps avoid WAR and WAW hazards
more reservation stations than registers; so allow optzns compilers can’t do
Common Data Bus broadcasts results to all FUs (forwarding!)
Load and Stores treated as FUs as well, with reservation stations for stores

6. Tomasulo: Organization
FP Registers
From Mem
FP Op
Queue
Load Buffers
Load1
Load2
Load3
Load4
Load5
Load6
Store
Buffers
Add1
Add2
Add3
Mult1
Mult2
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Reservation
Stations
To Mem
FP adders
FP multipliers
Common Data Bus (CDB)

7. More Details of Tomasulo Organization
Entities that produce values are assigned 4-bit tags
1, 2, 3, 4, 5, 6 for load buffers
8, 9 for multiplier reservation stations
10, 11, 12 for adder reservation stations
Tag 0 indicates presence of valid data
FP registers have “busy bits”
0 means that register holds valid data
1 means that it is waiting to receive value from source identified by its tag field

8. Reservation Station Components
Each reservation station (RS) has the following fields:
Op: Operation to perform (e.g., + or –)
Vj, Vk: Value of Source operands
Buffer for Store Inst. has one V field, result to be stored
Qj, Qk: Reservation stations producing source registers (value to be written)
Note: Qj,Qk ==0  ready
Buffer for Store Inst. has single Q field, source of value
Busy: Indicates reservation station or FU is busy
Register result status
Indicates which functional unit will write each register (if any)
Blank when no pending instructions will write that register
What you might have thought
1. 4 stages of instruction executino
2.Status of FU: Normal things to keep track of (RAW & structura for busyl):
Fi from instruction format of the mahine (Fi is dest)
Add unit can Add or Sub
Rj, Rk - status of registers (Yes means ready)
Qj,Qk - If a no in Rj, Rk, means waiting for a FU to write result; Qj, Qk means wihch FU waiting for it
3.Status of register result (WAW &WAR)s:
which FU is going to write into registers
Scoreboard on 6600 = size of FU
6.7, 6.8, 6.9, 6.12, 6.13, 6.16, 6.17
FU latencies: Add 2, Mult 10, Div 40 clocks

9. Tomasulo: Representing Data Dependences
Inputs
Operand is a register with busy bit = 0
Data copied immediately (through register bus) into RS
Tag field of RS set to 0
Operand is a register with busy bit = 1
Tag field of RS receives a copy of the register tag field
Operand is a load buffer that contains valid data
Data copied into RS
Operand is a load buffer that is awaiting data
Tag field of RS receives tag of load buffer
Outputs
Output is a register
Busy bit set to 1, tag set to RS tag
Output is a store buffer
Tag set to RS tag, destination address set

10. Three Stages of Tomasulo Algorithm
Issue: get instruction from FP operation queue
If reservation station free, the scoreboard issues instruction and sends operands (renames registers)
Execution: operate on operands (EX)
When both operands ready then execute; if not ready, watch CDB for result
Write Result: finish execution (WB)
Write on Common Data Bus to all awaiting units; mark reservation station available
Common Data Bus: data + source (“came from”)
64 bits of data + 4 bits of RS source address => broadcast
Different from “normal” bus: data + destination (“go to” bus)

11. Tomasulo Example Instruction stream 3 Load Buffers FU count
down
3 FP Adder R.S.
2 FP Mult R.S.
Clock cycle counter

12. Tomasulo Example Cycle 1

13. Tomasulo Example Cycle 2
Note: Can have multiple loads outstanding

14. Tomasulo Example Cycle 3
Note: registers names are removed (“renamed”) in Reservation Stations; MULT issued
Load1 completing; what is waiting for Load1?

15. Tomasulo Example Cycle 4
Load2 completing; what is waiting for Load2?

16. Tomasulo Example Cycle 5
Timer starts down for Add1, Mult1

17. Tomasulo Example Cycle 6
Issue ADDD here despite name dependency on F6?

18. Tomasulo Example Cycle 7
Add1 (SUBD) completing; what is waiting for it?

19. Tomasulo Example Cycle 8

20. Tomasulo Example Cycle 9

21. Tomasulo Example Cycle 10
Add2 (ADDD) completing; what is waiting for it?

22. Tomasulo Example Cycle 11
Write result of ADDD here?
All quick instructions complete in this cycle!

23. Tomasulo Example Cycle 12

24. Tomasulo Example Cycle 13

25. Tomasulo Example Cycle 14

26. Tomasulo Example Cycle 15
Mult1 (MULTD) completing; what is waiting for it?

27. Tomasulo Example Cycle 16
Just waiting for Mult2 (DIVD) to complete

28. Skip some cycles…

29. Tomasulo Example Cycle 55

30. Tomasulo Example Cycle 56
Mult2 (DIVD) is completing; what is waiting for it?

31. Tomasulo Example Cycle 57
Once again: In-order issue, out-of-order execution and out-of-order completion.

32. Observations on Tomasulo’s Algorithm
Instructions: move from decoder to reservation stations
in program order
dependences can be correctly recorded
Data Flow Graph: The graph of pointers connecting the RS, registers, and memory buffers
helps accomplish out-of-order sequencing of instructions
Chief cost of this scheme: high-speed associative hardware
RS hardware has to search for tags when CDB broadcasts some value with its tag
Full load bypassing is supported
load and store buffers are treated just like functional units
additional hardware on 360/91 also supported load forwarding

33. Tomasulo: Example of Load Bypassing
Instruction 202 depends on instructions 200 and 201, so instruction 203 will start executing much before 202 (assuming C and D are found to be different memory addresses)
Work out details off-line
200: F0 ← A
201: F0 ← F0 / F1
202: C ← F0
203: F0 ← D
204: F0 ← F0 * F2

34. Tomasulo: “Loop Unrolling in Hardware”
360/91 supported limited kind of speculation
Small loops could be held in a loop buffer
Loop closing branches were predicted as taken
This has the effect of loop unrolling at run-time
Given the small number of FP registers in machine, software loop unrolling was not a viable option
Why exactly can it unroll loops?
Register renaming
Multiple iterations use different physical destinations for registers (dynamic loop unrolling).
Reservation stations
Permit instruction issue to advance past integer control flow operations
Also buffer old values of registers - totally avoiding the WAR stall
Other perspective: it’s building data flow dependency graph on the fly

35. Tomasulo Loop Example Multiply takes 4 clocks Loads slow (no caches!)
Loop: L.D F0 0 R1
MULT.D F4 F0 F2
S.D F4 0 R1
SUBI R1 R1 #8
BNEZ R1 Loop
Multiply takes 4 clocks
Loads slow (no caches!)

36. Loop Example Cycle 0

37. Loop Example Cycle 1

38. Loop Example Cycle 2

39. Loop Example Cycle 3

40. Loop Example Cycle 4

41. Loop Example Cycle 5

42. Loop Example Cycle 6
Load2

43. Loop Example Cycle 7

44. Loop Example Cycle 8

45. Loop Example Cycle 9

46. Loop Example Cycle 10

47. Loop Example Cycle 11

48. Loop Example Cycle 12
Structural hazard – no MULT unit available

49. Loop Example Cycle 13

50. Loop Example Cycle 14

51. Loop Example Cycle 15

52. Loop Example Cycle 16

53. Loop Example Cycle 17

54. Loop Example Cycle 18

55. Loop Example Cycle 19 …

56. Loop Example Cycle 20

57. Loop Example Cycle 21

58. Summary of Tomasulo’s Algorithm
Reservations stations: renaming to larger set of registers + buffering source operands
Registers not bottleneck
Avoids WAR, WAW hazards of scoreboarding
Allows loop unrolling in hardware
Not limited to basic blocks
provided we have branch prediction
Lasting contributions
Dynamic scheduling
Register renaming
Load/store disambiguation
360/91 descendants are Intel Pentium 4, IBM Power 5, AMD Athlon/Opteron, …
Next: Compiler techniques