IBM System 360. Common architecture for a set of machines IBM System 360. Common architecture for a set of machines. Robert Tomasulo worked on a high-end machine, the Model 91 (1967), on which they implemented his algorithm (today’s topic).

COMP 740: Computer Architecture and Implementation Montek Singh Oct 24, 2016 Topic: Instruction-Level Parallelism III (Dynamic Scheduling: Tomasulo’s Algorithm)

Today’s Topic Tomasulo’s algorithm for dynamic scheduling more sophisticated than scoreboarding will enable scheduling multiple iterations of a loop (but still limited, more to come) Reading: Ch. 3.4-3.5

Dynamic Scheduling: Tomasulo’s Algorithm For IBM 360/91 (about three years after CDC 6600) long mem latency (before caches!) Goal: High performance without special compilers Small number of floating point registers (4 in 360 architecture) prevented interesting compiler scheduling of operations Led Tomasulo to try to figure out how to get more effective registers — renaming in hardware! Why Study 1967 Computer? Technique languished until 1990s, then rediscovered The descendants have flourished Alpha 21264, Pentium 4, AMD Opteron, Power 5, Intel Core i3/i5/i7 …

Tomasulo’s Algorithm Differences between IBM 360 and CDC 6600 ISA IBM has only 2 register specifiers/instruction versus 3 in CDC 6600 IBM has 4 FP registers versus 8 in CDC 6600 Differences between Tomasulo Algorithm and Scoreboard Control and buffers distributed with Function Units versus centralized in scoreboard; called “reservation stations” Registers in instructions replaced by pointers to reservation stations this is called register renaming renaming helps avoid WAR and WAW hazards more reservation stations than registers; so allow optzns compilers can’t do Common Data Bus broadcasts results to all FUs (forwarding!) Load and Stores treated as FUs as well, with reservation stations for stores

Tomasulo: Organization FP Registers From Mem FP Op Queue Load Buffers Load1 Load2 Load3 Load4 Load5 Load6 Store Buffers Add1 Add2 Add3 Mult1 Mult2 Resolve RAW memory conflict? (address in memory buffers) Integer unit executes in parallel Reservation Stations To Mem FP adders FP multipliers Common Data Bus (CDB)

More Details of Tomasulo Organization Entities that produce values are assigned 4-bit tags 1, 2, 3, 4, 5, 6 for load buffers 8, 9 for multiplier reservation stations 10, 11, 12 for adder reservation stations Tag 0 indicates presence of valid data FP registers have “busy bits” 0 means that register holds valid data 1 means that it is waiting to receive value from source identified by its tag field

Reservation Station Components Each reservation station (RS) has the following fields: Op: Operation to perform (e.g., + or –) Vj, Vk: Value of Source operands Buffer for Store Inst. has one V field, result to be stored Qj, Qk: Reservation stations producing source registers (value to be written) Note: Qj,Qk ==0  ready Buffer for Store Inst. has single Q field, source of value Busy: Indicates reservation station or FU is busy Register result status Indicates which functional unit will write each register (if any) Blank when no pending instructions will write that register What you might have thought 1. 4 stages of instruction executino 2.Status of FU: Normal things to keep track of (RAW & structura for busyl): Fi from instruction format of the mahine (Fi is dest) Add unit can Add or Sub Rj, Rk - status of registers (Yes means ready) Qj,Qk - If a no in Rj, Rk, means waiting for a FU to write result; Qj, Qk means wihch FU waiting for it 3.Status of register result (WAW &WAR)s: which FU is going to write into registers Scoreboard on 6600 = size of FU 6.7, 6.8, 6.9, 6.12, 6.13, 6.16, 6.17 FU latencies: Add 2, Mult 10, Div 40 clocks

Tomasulo: Representing Data Dependences Inputs Operand is a register with busy bit = 0 Data copied immediately (through register bus) into RS Tag field of RS set to 0 Operand is a register with busy bit = 1 Tag field of RS receives a copy of the register tag field Operand is a load buffer that contains valid data Data copied into RS Operand is a load buffer that is awaiting data Tag field of RS receives tag of load buffer Outputs Output is a register Busy bit set to 1, tag set to RS tag Output is a store buffer Tag set to RS tag, destination address set

Three Stages of Tomasulo Algorithm Issue: get instruction from FP operation queue If reservation station free, the scoreboard issues instruction and sends operands (renames registers) Execution: operate on operands (EX) When both operands ready then execute; if not ready, watch CDB for result Write Result: finish execution (WB) Write on Common Data Bus to all awaiting units; mark reservation station available Common Data Bus: data + source (“came from”) 64 bits of data + 4 bits of RS source address => broadcast Different from “normal” bus: data + destination (“go to” bus)

Tomasulo Example Instruction stream 3 Load Buffers FU count down 3 FP Adder R.S. 2 FP Mult R.S. Clock cycle counter

Tomasulo Example Cycle 1

Tomasulo Example Cycle 2 Note: Can have multiple loads outstanding

Tomasulo Example Cycle 3 Note: registers names are removed (“renamed”) in Reservation Stations; MULT issued Load1 completing; what is waiting for Load1?

Tomasulo Example Cycle 4 Load2 completing; what is waiting for Load2?

Tomasulo Example Cycle 5 Timer starts down for Add1, Mult1

Tomasulo Example Cycle 6 Issue ADDD here despite name dependency on F6?

Tomasulo Example Cycle 7 Add1 (SUBD) completing; what is waiting for it?

Tomasulo Example Cycle 8

Tomasulo Example Cycle 9

Tomasulo Example Cycle 10 Add2 (ADDD) completing; what is waiting for it?

Tomasulo Example Cycle 11 Write result of ADDD here? All quick instructions complete in this cycle!

Tomasulo Example Cycle 12

Tomasulo Example Cycle 13

Tomasulo Example Cycle 14

Tomasulo Example Cycle 15 Mult1 (MULTD) completing; what is waiting for it?

Tomasulo Example Cycle 16 Just waiting for Mult2 (DIVD) to complete

Skip some cycles…

Tomasulo Example Cycle 55

Tomasulo Example Cycle 56 Mult2 (DIVD) is completing; what is waiting for it?

Tomasulo Example Cycle 57 Once again: In-order issue, out-of-order execution and out-of-order completion.

Observations on Tomasulo’s Algorithm Instructions: move from decoder to reservation stations in program order dependences can be correctly recorded Data Flow Graph: The graph of pointers connecting the RS, registers, and memory buffers helps accomplish out-of-order sequencing of instructions Chief cost of this scheme: high-speed associative hardware RS hardware has to search for tags when CDB broadcasts some value with its tag Full load bypassing is supported load and store buffers are treated just like functional units additional hardware on 360/91 also supported load forwarding

Tomasulo: Example of Load Bypassing Instruction 202 depends on instructions 200 and 201, so instruction 203 will start executing much before 202 (assuming C and D are found to be different memory addresses) Work out details off-line 200: F0 ← A 201: F0 ← F0 / F1 202: C ← F0 203: F0 ← D 204: F0 ← F0 * F2

Tomasulo: “Loop Unrolling in Hardware” 360/91 supported limited kind of speculation Small loops could be held in a loop buffer Loop closing branches were predicted as taken This has the effect of loop unrolling at run-time Given the small number of FP registers in machine, software loop unrolling was not a viable option Why exactly can it unroll loops? Register renaming Multiple iterations use different physical destinations for registers (dynamic loop unrolling). Reservation stations Permit instruction issue to advance past integer control flow operations Also buffer old values of registers - totally avoiding the WAR stall Other perspective: it’s building data flow dependency graph on the fly

Tomasulo Loop Example Multiply takes 4 clocks Loads slow (no caches!) Loop: L.D F0 0 R1 MULT.D F4 F0 F2 S.D F4 0 R1 SUBI R1 R1 #8 BNEZ R1 Loop Multiply takes 4 clocks Loads slow (no caches!)

Loop Example Cycle 0

Loop Example Cycle 1

Loop Example Cycle 2

Loop Example Cycle 3

Loop Example Cycle 4

Loop Example Cycle 5

Loop Example Cycle 6 Load2

Loop Example Cycle 7

Loop Example Cycle 8

Loop Example Cycle 9

Loop Example Cycle 10

Loop Example Cycle 11

Loop Example Cycle 12 Structural hazard – no MULT unit available

Loop Example Cycle 13

Loop Example Cycle 14

Loop Example Cycle 15

Loop Example Cycle 16

Loop Example Cycle 17

Loop Example Cycle 18

Loop Example Cycle 19 …

Loop Example Cycle 20

Loop Example Cycle 21

Summary of Tomasulo’s Algorithm Reservations stations: renaming to larger set of registers + buffering source operands Registers not bottleneck Avoids WAR, WAW hazards of scoreboarding Allows loop unrolling in hardware Not limited to basic blocks provided we have branch prediction Lasting contributions Dynamic scheduling Register renaming Load/store disambiguation 360/91 descendants are Intel Pentium 4, IBM Power 5, AMD Athlon/Opteron, … Next: Compiler techniques