ELEC 5200-001/6200-001 Computer Architecture and Design Spring 2015 Instruction-Level Parallelism Vishwani D. Agrawal James J. Danaher Professor Department of Electrical and Computer Engineering Auburn University, Auburn, AL 36849 http://www.eng.auburn.edu/~vagrawal vagrawal@eng.auburn.edu Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

A Computer System Processor Interrupts Cache Memory – I/O bus I/O controller I/O controller I/O controller Main memory Disk Disk Graphics output Network Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Advanced Architectures – ILP Instruction level parallelism (ILP): multiple instructions fetched and executed simultaneously. ILP is used in addition to pipelining. Processors with ILP are called multiple-issue processors – multiple instructions launched in 1 clock cycle. Two ways: MIMD: Multiple Instructions Multiple Data Superpipeline Superscalar – dynamic multiple issue Very long instruction word (VLIW) – static multiple issue SIMD: Single Instruction Multiple Data Vector processor Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Superpipeline and Superscalar IF ID EX MEM WB Pipeline 1 instruction/cycle IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Superpipeline (Pipeline clock is twice as fast as the system clock) 2 instructions per cycle IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Superscalar 2 (or more) instructions/cycle IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB System clock cycles 0 1 2 3 4 5 6 7 8 Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

A Static Two-Issue MIPS Pipeline Read two instructions per cycle: An ALU or branch instruction, and A load or store instruction Insert one nop if above pair is not available Added hardware (Figure 4.69, page 336): A second instruction memory Additional input/output ports in register file Additional ALU in execute stage for address calculation Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

An Example (Page 337) Loop: lw $t0, 0($s1) addu $t0, $t0, $s2 sw $t0, 0(s1) addi $s1, $s1, – 4 bne $s1, $0, Loop Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Static Two-Issue Execution ALU or branch instruction Data transfer instruction Clock cycle Loop: nop lw $t0, 0($s1) 1 addi $s1, $s1, – 4 2 addu $t0, $t0, $s2 3 bne $s1, $0, Loop sw $t0, 4($s1) 4 Note code reordering and change in sw argument. CPI = 4/5 = 0.8 < 0.5 (ideal) Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Loop Unrolling (Index Multiple of 4) ALU or branch instruction Data transfer instruction Clock cycle Loop: addi $s1, $s1, – 16 lw $t0, 0($s1) 1 nop lw $t1, 12($s1) 2 addu $t0, $t0, $s2 lw $t2, 8($s1) 3 addu $t1, $t1, $s2 lw $t3, 4($s1) 4 addu $t2, $t2, $s2 sw $t0, 16($s1) 5 addu $t3, $t3, $s2 sw $t1, 12($s1) 6 sw $t2, 8($s1) 7 bne $s1, $0, Loop sw $t3, 4($s1) 8 CPI = 8/14 = 0.57 < 0.5 (ideal) Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

VLIW: Very Long Instruction Word Static multiple issue, ILP determined by compiler. Datapath contains multiple execution units. Compiler groups instructions that have no data or resource conflicts for parallel execution. Grouped instructions are packed in very long words of a wide instruction memory. Speedup benefit of VLIW is highly program dependent. J. A. Fisher, “Very Long Instruction Word Architecture and ELI-512,” Proc. 10th Symp. on Computer Architecture, Stockholm, June 1983, pp. 478-490. J. A. Fisher, P. Faraboschi and C. Young, Embedded Computing: A VLIW Approach to Architecture, Compilers and Tools, Morgan Kaufmann. Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Superscalar: Dynamic Scheduling and Out-of-Order Execution Instruction fetch and decode unit In-order issue Out-of-order issue Reservation station Reservation station Reservation station Reservation station Functional units integer integer Floating point Load/ store Out-of-order execution Commit unit In-order commit Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Out of Order Execution (OOE) A procedural programming language sequences instructions. Sequencing assumes an order of execution – no parallelism. OOE must preserve correctness of result. Principle: Two instructions can be executes in parallel if they do not have dependences. Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

RAW Dependence Read after write (RAW): A dependent instruction reads from a register being written to by another instruction. Example: add $s1, $s2, $s3 sub $s2, $s1, $s3 sub has RAW dependence on add Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

WAR Dependence Write after read (WAR): A dependent instruction writes to a register being read by another instruction. Example: add $s1, $s2, $s3 sub $s2, $s1, $s3 sub has WAR dependence on add Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

WAW Dependence Read after write (RAW): One instruction writes to a register to being written to by another instruction. Example: add $s2, $s2, $s3 sub $s2, $s1, $s3 sub has WAW dependence on add Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Superscalar Instruction Issue Rules: RAW dependence – If any operand is being written, do not issue. WAR dependence – If the result register is being read, do not issue. WAW dependence – If the result register is being written, do not issue. Scoreboard: Cycle by cycle record of registers and execution units showing how many instructions are using them. Example 1: In-order issue (next 2 slides). Example 2: Out-of-order issue (3rd slide). Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Dynamic Scheduling Consider an example: Assume: First with in-order issue Then with out-of-order issue Assume: Up to two instructions are fetched in a cycle Instruction register can hold two instructions An Instruction is issued in decode cycle, or must wait until there is no RAW, WAR or WAW dependence An instruction can retire two or three cycles after it is issued Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

# Ck Decoded Issue Inst# Retire Reg. to read Reg. to write 1 2 3 4 5 6 cycle Inst # Decoded Issue Inst# Retire Reg. to read Reg. to write 1 2 3 4 5 6 7 R3 = R0 * R1 R4 = R0 + R2 R5 = R0 + R1 R6 = R1 + R4 - R7 = R1 * R2 R1 = R0 – R2 8 9 R3 = R3 * R1 Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

In-order Issue scoreboard (Continued) # Ck cycle Instr # Decoded Issue Inst# Retire Reg. to read Reg. to write 1 2 3 4 5 6 7 10 11 12 8 R1 = R4 + R4 - 13 14 15 16 17 18 Out-of-order scoreboard (Next 2 Slides) Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Questions? RAW dependence: Inst# 4 (R6 = R1 + R4) could not be issued until cycle 5. Should Inst# 5 (R7 = R1 * R2) wait in queue? Answer: No. Inst# 5 can be issued in cycle 3 as there is no register conflict (out-of-order issue). WAR dependence: Must the issue of Inst#6 (R1 = R0 – R2) waits until cycle 9 when all instructions reading R1 have retired? Answer: No. Provided new result of Inst#6 does not affect R1 being used by previous instructions (register renaming). Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Inst # Ck Decoded Issue Inst# Retire Reg. to read Reg. to write 1 2 3 cycle Inst # Decoded Issue Inst# Retire Reg. to read Reg. to write 1 2 3 4 5 6 7 R3 = R0 * R1 R4 = R0 + R2 R5 = R0 + R1 R6 = R1 + R4 - R7 = R1 * R2 S1 = R0 – R2 8 R3 = R3 * S1 S2 = R4 + R4 11 9 Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

References Previous example is from: Further reading: A. S. Tanenbaum, Structured Computer Organization, Fifth Edition, Prentice-Hall, 2006, pp. 304-309, Section 4.5.3. Further reading: D. W. Anderson, F. J. Sparacio and R. M. Tomasulo, “The IBM 360 Model 91: Processor Philosophy and Instruction Handling,” IBM J. Res. & Dev., vol. 11, no. 1, pp. 8-24, Jan. 1967. Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Power Reduction by Slack Scheduling Application: Superscalar, out-of-order execution: An instruction is executed as soon as the required data and resources become available. A commit unit reorders the results. Delay the completion of instructions whose result is not immediately needed. Example of RISC instructions: add r0, r1, r2; (A) sub r3, r4, r5; (B) and r9, r1, r9; (C) or r5, r9, r10; (D) xor r2, r10, r11; (E) J. Casmira and D. Grunwald, “Dynamic Instruction Scheduling Slack,” Proc. ACM Kool Chips Workshop, Dec. 2000. Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Slack Scheduling Example Standard scheduling A B C D E Slack scheduling A B C D E Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Slack Scheduling Re-order buffer Low-power execution units Slack bit Scheduling logic Low-power execution units (Reduced voltage) Slack bit Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Superscalar Design of P4 (CISC) CISC shell: Processor fetches instructions from memory in the order of static program. Each instruction is translated into one or more fixed-length RISC instructions, known as micro-operations (micro-ops). RISC core: Micro-ops are executed out-of-order in a dynamically scheduled pipeline. Processor commits the result of each micro-op execution to register file in the order of original program flow. Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Superscalars 3 or more instruction issues per clock: References: Intel P6 AMD K5 Sun UltraSPARC Alpha 21164 MIPS R10000 PowerPC 604/620 HP 8000 References: D. W. Anderson, F. J. Sparacio and R. M. Tomasulo, “The IBM 360 Model 91: Processor Philosophy and Instruction Handling,” IBM J. Res. Dev., vol. 11, pp. 8-24, January 1967. T. Agerwala and J. Cocke, “Reduced Instruction Set Processors,” Technical Report RC12434 (#55845), Yorktown Heights, NY: IBM T. J. Watson Research Center, January 1987. Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

Topics in Computer Architecture Instruction set Program execution through register transfer See Lectures 13-14. Computer arithmetic (2’s complement, IEEE 754 floating point standard, addition, multiplication) Datapaths (single-cycle, multicycle, pipeline) Control (combinational logic, FSM, microcode) Pipelining (throughput, hazards, forwarding, stall, branch prediction) Memory organization (cache, virtual memory) Performance (benchmarks, energy efficiency, Amdal’s law) Advanced architectures (ILP, OOE, superscalar, etc.) Not discussed in this course: Multiprocessors Compiler and software techniques – loop unrolling, trace execution, etc. Input and output Power management Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12

One who claims to know much about computer architecture speaks from ignorance . . . because a lot is going to happen in the future, which is . . . http://www.youtube.com/watch?v=xZbKHDPPrrc Doris Day in Hitchcock’s 1956 Movie “The Man Who Knew Too Much” Spr 2015, Apr 22 . . . ELEC 5200-001/6200-001 Lecture 12