1. Lecture 7 Dynamic Scheduling
This lecture will cover sections : introduction, application, technology trends, cost and price.
Assoc. Prof Dr. Norafida Ithnin
Advanced Computer System & Architecture
MCC 2313

2. Dynamic vs. Static Scheduling
Data hazards in a program to cause a processor to stall.
With static scheduling the compiler tries to reorder these instructions during compile time to reduce pipeline stalls.
Uses less hardware
Can use more powerful algorithms
With dynamic scheduling the hardware tries to rearrange the instructions during run-time to reduce pipeline stalls.
Simpler compiler
Handles dependencies not known at compile time
Allows code compiled for a different machine to run efficiently.

3. Out-Of-Order Execution
In our previous model of DLX, all instructions executed in the order that they appear
This can lead to unnecessary stalls
DIVD FO, F2, F4
ADDD F10, F0, F8
SUBD F12, F8, F14
SUBD stalls waiting for the ADDD to go first, even though SUBD does not have a data dependency.
With out-of-order execution, the SUBD is allowed to executed before the add
This can lead to out-of order completion, which can cause WAW and WAR hazards

4. Scoreboarding
The scoreboard implements a centralized control scheme that
Detects all resource and data hazards
Allows instructions to execute out-of-order when no resource hazards or data dependencies
First implemented in 1964 by the CDC 6600, which had 18 separate functional units
4 FP units (2 multiply, 1 add, 1 divide)
7 memory units (5 loads, 2 stores)
7 integer units (add, shift, logical, compare, etc.)
Dynamic DLX (much simpler)
2 FP multiply (10 EX cycles)
1 FP add (2 EX cycles)
1 FP divide (40 EX cycles)
1 integer unit (1 EX cycle)

5. Out-of-Order Execution
Out-of-order execution divides ID stage:
1. Issue—decode instructions, check for structural hazards
2. Read operands—wait until no data hazards, then read operands
Scoreboards allow instruction to execute whenever 1 & 2 hold, not waiting for prior instructions
CDC 6600: In order issue, out of order execution, out of order commit (also called completion)

6. Scoreboard Implications
Out-of-order completion can lead to WAR and WAW hazards?
Solution for WAW
Detect WAW hazard before reading operands
Stall write until other instruction completes
Solutions for WAR
Detect WAR hazards before writing back to the register files and stall the write back
This scoreboard does not take advantage of forwarding, since it waits until both results are written back to the register file
Scoreboard replaces ID, EX, WB with 4 stages

7. Four Stages of Scoreboard Control
Issue (ID1)
decode instructions
check for structural and WAW hazards
stall until structural and WAW hazards are resolved
Read operands (ID2)
wait until no RAW hazards
then read operands
Execution (EX)
operate on operands
may be multiple cycles - notify scoreboard when done
Write result (WB)
finish execution
stall if WAR hazard

8. Three Parts of the Scoreboard
1. Instruction status—which of 4 steps the instruction is in: ID1, ID1, EX, or WB.
2. Functional unit status—Indicates the state of the functional unit (FU). 9 fields for each functional unit
Busy—Indicates whether the unit is busy or not
Op—Operation to perform in the unit (e.g., + or –)
Fi—Destination register
Fj, Fk—Source-register numbers
Qj, Qk—Functional units producing source registers Fj, Fk
Rj, Rk—Flags indicating when Fj, Fk are ready
3. Register result status—Indicates which functional unit will write each register, if one exists. Blank when no pending instructions will write that register

9. Scoreboard Example for DLX

10. CDC 6000 Scoreboard Summary
Speedup from scoreboard
1.7 for FORTRAN programs
2.5 for hand-coded assembly language programs
Effects of modern compilers?
Hardware
Scoreboard hardware approximately same as one FPU
Main cost was buses (4x’s normal amount)
Could be more severe for modern processors
Limitations
No forwarding logic
Limited to instructions in basic block
Stalls for WAW hazards
Wait for WAR hazards before WB

11. Tomasulo Algorithm for Dynamic Scheduling
For IBM 360/91 in about 3 years after CDC 6600
Goal: High performance without special compilers
Differences between IBM 360 & CDC 6600
IBM has only 2 register specifiers/instr vs. 3 in CDC 6600
IBM has register-memory instructions
IBM has 4 FP registers vs. 8 in CDC 6600
IBM has pipelined functional units (3 adds, 2 multiplies)
Tomasulo algorithm is designed to handle name dependencies (WAW and WAR hazards) efficiently
SUB F1, F2, F0
DIVF F2, F3 , F2
ADDF F3, F0, F0
MULF F3, F1, F1

12. Tomasulo Algorithm Differences from Scoreboarding
Distributed hazard detection and control (through reservation stations)
Results are bypassed to function units
Common data bus (CDB) broadcasts results to all FUs.
HW renaming of registers to avoid WAR, WAW hazards
Load and Stores treated as FUs as well
Registers in instructions replaced by pointers to reservation station buffers
Lead to concepts used in Alpha 21264, HP 8000, MIPS 10000, Pentium II, PowerPC 604, …

13. Tomasulo Organization
FP Op Queue
FP Registers
Load Buffer
Store Buffer
Common Data Bus
FP Add Res. Station
FP Mul Res. Station

14. Reservation Station Components
Op—Operation to perform in the unit (e.g., + or –)
Qj, Qk—Reservation stations producing source registers
Vj, Vk—Value of Source operands
Rj, Rk—Flags indicating when Vj, Vk are ready
Busy—Indicates reservation station and FU is busy
Register result status—Indicates which functional unit will write each register, if one exists. Blank when no pending instructions that will write that register.
What you might have thought
1. 4 stages of instruction executino
2.Status of FU: Normal things to keep track of (RAW & structura for busyl):
Fi from instruction format of the mahine (Fi is dest)
Add unit can Add or Sub
Rj, Rk - status of registers (Yes means ready)
Qj,Qk - If a no in Rj, Rk, means waiting for a FU to write result; Qj, Qk means wihch FU waiting for it
3.Status of register result (WAW &WAR)s:
which FU is going to write into registers
Scoreboard on 6600 = size of FU
6.7, 6.8, 6.9, 6.12, 6.13, 6.16, 6.17
FU latencies: Add 2, Mult 10, Div 40 clocks

15. Three Stages of Tomasulo Algorithm
1. Issue—get instruction from FP Op Queue
If reservation station free, the scoreboard issues instr & sends operands (renames registers).
2. Execution—operate on operands (EX)
When both operands ready then execute; if not ready, watch CDB for result
3. Write result—finish execution (WB)
Write on Common Data Bus to all awaiting units; mark reservation station available.

16. Tomasulo Example Cycle 0

17. Tomasulo Summary Advantages
Prevents register from being the bottleneck
Eliminates WAR, WAW hazards
Allows loop unrolling in HW
Common Data Bus
Broadcasts results to multiple instructions
Central bottleneck
Lasting Contributions
Dynamic scheduling
Register renaming
Load/store disambiguation

18. Discussions