Krste Asanovic Electrical Engineering and Computer Sciences

Slides:



Advertisements
Similar presentations
Out-of-Order Execution & Register Renaming
Advertisements

Krste Asanovic Electrical Engineering and Computer Sciences
© Krste Asanovic, 2014CS252, Spring 2014, Lecture 5 CS252 Graduate Computer Architecture Spring 2014 Lecture 5: Out-of-Order Processing Krste Asanovic.
2/28/2013 CS152, Spring 2013 CS 152 Computer Architecture and Engineering Lecture 11 - Out-of-Order Issue, Register Renaming, & Branch Prediction Krste.
ECE 552 / CPS 550 Advanced Computer Architecture I Lecture 8 Instruction-Level Parallelism – Part 1 Benjamin Lee Electrical and Computer Engineering Duke.
COMP25212 Advanced Pipelining Out of Order Processors.
CSE 490/590, Spring 2011 CSE 490/590 Computer Architecture Complex Pipelining II Steve Ko Computer Sciences and Engineering University at Buffalo.
February 28, 2011CS152, Spring 2011 CS 152 Computer Architecture and Engineering Lecture 10 - Complex Pipelines, Out-of-Order Issue, Register Renaming.
Spring 2003CSE P5481 Reorder Buffer Implementation (Pentium Pro) Hardware data structures retirement register file (RRF) (~ IBM 360/91 physical registers)
CPE 731 Advanced Computer Architecture ILP: Part IV – Speculative Execution Dr. Gheith Abandah Adapted from the slides of Prof. David Patterson, University.
CSE 490/590, Spring 2011 CSE 490/590 Computer Architecture ILP II Steve Ko Computer Sciences and Engineering University at Buffalo.
February 28, 2012CS152, Spring 2012 CS 152 Computer Architecture and Engineering Lecture 11 - Out-of-Order Issue, Register Renaming, & Branch Prediction.
1 IF IDEX MEM L.D F4,0(R2) MUL.D F0, F4, F6 ADD.D F2, F0, F8 L.D F2, 0(R2) WB IF IDM1 MEM WBM2M3M4M5M6M7 stall.
CS 152 Computer Architecture and Engineering Lecture 13 - Out-of-Order Issue, Register Renaming, & Branch Prediction Krste Asanovic Electrical Engineering.
CSE 490/590, Spring 2011 CSE 490/590 Computer Architecture ILP III Steve Ko Computer Sciences and Engineering University at Buffalo.
March 11, 2010CS152, Spring 2010 CS 152 Computer Architecture and Engineering Lecture 14 - Advanced Superscalars Krste Asanovic Electrical Engineering.
CSE 490/590, Spring 2011 CSE 490/590 Computer Architecture ILP I Steve Ko Computer Sciences and Engineering University at Buffalo.
March 2, 2010CS152, Spring 2010 CS 152 Computer Architecture and Engineering Lecture 12 - Complex Pipelines Krste Asanovic Electrical Engineering and Computer.
CS 152 Computer Architecture and Engineering Lecture 14 - Advanced Superscalars Krste Asanovic Electrical Engineering and Computer Sciences University.
CS 152 Computer Architecture and Engineering Lecture 15 - Advanced Superscalars Krste Asanovic Electrical Engineering and Computer Sciences University.
March 4, 2010CS152, Spring 2010 CS 152 Computer Architecture and Engineering Lecture 13 - Out-of-Order Issue, Register Renaming, & Branch Prediction Krste.
CS 152 Computer Architecture and Engineering Lecture 12 - Complex Pipelines Krste Asanovic Electrical Engineering and Computer Sciences University of California.
CS 152 Computer Architecture and Engineering Lecture 13 - Out-of-Order Issue and Register Renaming Krste Asanovic Electrical Engineering and Computer Sciences.
CS 152 Computer Architecture and Engineering Lecture 12 - Complex Pipelines Krste Asanovic Electrical Engineering and Computer Sciences University of California.
March 9, 2011CS152, Spring 2011 CS 152 Computer Architecture and Engineering Lecture 12 - Advanced Out-of-Order Superscalars Krste Asanovic Electrical.
1 COMP 206: Computer Architecture and Implementation Montek Singh Wed, Oct 5, 2005 Topic: Instruction-Level Parallelism (Dynamic Scheduling: Scoreboarding)
CS 252 Graduate Computer Architecture Lecture 4: Instruction-Level Parallelism Krste Asanovic Electrical Engineering and Computer Sciences University of.
March 2, 2011CS152, Spring 2011 CS 152 Computer Architecture and Engineering Lecture 11 - Out-of-Order Issue, Register Renaming, & Branch Prediction Krste.
ECE 552 / CPS 550 Advanced Computer Architecture I Lecture 9 Instruction-Level Parallelism – Part 2 Benjamin Lee Electrical and Computer Engineering Duke.
1 Lecture 6 Tomasulo Algorithm CprE 581 Computer Systems Architecture, Fall 2009 Zhao Zhang Reading:Textbook 2.4, 2.5.
CS 152 Computer Architecture and Engineering Lecture 15 - Out-of-Order Memory, Complex Superscalars Review Krste Asanovic Electrical Engineering and Computer.
Professor Nigel Topham Director, Institute for Computing Systems Architecture School of Informatics Edinburgh University Informatics 3 Computer Architecture.
04/03/2016 slide 1 Dynamic instruction scheduling Key idea: allow subsequent independent instructions to proceed DIVDF0,F2,F4; takes long time ADDDF10,F0,F8;
March 1, 2012CS152, Spring 2012 CS 152 Computer Architecture and Engineering Lecture 12 - Advanced Out-of-Order Superscalars Krste Asanovic Electrical.
IBM System 360. Common architecture for a set of machines
CS 152 Computer Architecture and Engineering Lecture 10 - Complex Pipelines, Out-of-Order Issue, Register Renaming John Wawrzynek Electrical Engineering.
Dynamic Scheduling Why go out of style?
Images from Patterson-Hennessy Book
CS 152 Computer Architecture and Engineering Lecture 11 - Out-of-Order Issue, Register Renaming, & Branch Prediction John Wawrzynek Electrical Engineering.
Lecture 10 - Complex Pipelines, Out-of-Order Issue, Register Renaming
/ Computer Architecture and Design
Out of Order Processors
Step by step for Tomasulo Scheme
Dr. George Michelogiannakis EECS, University of California at Berkeley
CS203 – Advanced Computer Architecture
Microprocessor Microarchitecture Dynamic Pipeline
High-level view Out-of-order pipeline
John Kubiatowicz Electrical Engineering and Computer Sciences
CMSC 611: Advanced Computer Architecture
Lecture 6: Advanced Pipelines
Out of Order Processors
Dept. of Info. Sci. & Elec. Engg.
Lecture 8: ILP and Speculation Contd. Chapter 2, Sections 2. 6, 2
CS252 Graduate Computer Architecture Lecture 7 Dynamic Scheduling 2: Precise Interrupts February 9th, 2010 John Kubiatowicz Electrical Engineering.
Electrical and Computer Engineering
CS 152 Computer Architecture and Engineering Lecture 13 - Out-of-Order Issue, Register Renaming, & Branch Prediction Krste Asanovic Electrical Engineering.
Krste Asanovic Electrical Engineering and Computer Sciences
CS 704 Advanced Computer Architecture
Lecture 8: Dynamic ILP Topics: out-of-order processors
Adapted from the slides of Prof
Krste Asanovic Electrical Engineering and Computer Sciences
Krste Asanovic Electrical Engineering and Computer Sciences
Lecture 7: Dynamic Scheduling with Tomasulo Algorithm (Section 2.4)
Tomasulo Organization
Adapted from the slides of Prof
Scoreboarding ENGS 116 Lecture 7 Vincent H. Berk October 5, 2005
September 20, 2000 Prof. John Kubiatowicz
CS 152 Computer Architecture and Engineering CS252 Graduate Computer Architecture Lecture 11 – Out-of-Order Execution Krste Asanovic Electrical Engineering.
Lecture 9: Dynamic ILP Topics: out-of-order processors
Conceptual execution on a processor which exploits ILP
Presentation transcript:

CS 152 Computer Architecture and Engineering Lecture 13 - Out-of-Order Issue and Register Renaming Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste http://inst.eecs.berkeley.edu/~cs152 CS252 S05

Last time in Lecture 12 Pipelining is complicated by multiple and/or variable latency functional units Out-of-order and/or pipelined execution requires tracking of dependencies RAW WAR WAW Dynamic issue logic can support out-of-order execution to improve performance Last time, looked at simple scoreboard to track out-of-order completion 3/13/2008 CS152-Spring’08 CS252 S05

In-Order Issue Pipeline ALU Mem IF WB ID Issue Fadd GPR’s FPR’s Fmul Fdiv 3/13/2008 CS152-Spring’08 CS252 S05

Scoreboard for In-order Issues Busy[FU#] : a bit-vector to indicate FU’s availability. (FU = Int, Add, Mult, Div) These bits are hardwired to FU's. WP[reg#] : a bit-vector to record the registers for which writes are pending. These bits are set to true by the Issue stage and set to false by the WB stage Issue checks the instruction (opcode dest src1 src2) against the scoreboard (Busy & WP) to dispatch FU available? RAW? WAR? WAW? Busy[FU#] WP[src1] or WP[src2] cannot arise WP[dest] 3/13/2008 CS152-Spring’08 CS252 S05

In-Order Issue Limitations: an example latency 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 DIVD F4, F2, F8 4 6 ADDD F10, F6, F4 1 1 2 3 4 5 6 In-order: 1 (2,1) . . . . . . 2 3 4 4 3 5 . . . 5 6 6 In-order restriction prevents instruction 4 from being dispatched 3/13/2008 CS152-Spring’08 CS252 S05

Out-of-Order Issue IF ID WB ALU Mem Fadd Fmul Issue Issue stage buffer holds multiple instructions waiting to issue. Decode adds next instruction to buffer if there is space and the instruction does not cause a WAR or WAW hazard. Any instruction in buffer whose RAW hazards are satisfied can be issued (for now at most one dispatch per cycle). On a write back (WB), new instructions may get enabled. 3/13/2008 CS152-Spring’08 CS252 S05

Issue Limitations: In-Order and Out-of-Order latency 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 DIVD F4, F2, F8 4 6 ADDD F10, F6, F4 1 1 2 3 4 5 6 WAR hazard delays issue of 5 In-order: 1 (2,1) . . . . . . 2 3 4 4 3 5 . . . 5 6 6 Out-of-order: 1 (2,1) 4 4 . . . . 2 3 . . 3 5 . . . 5 6 6 3/13/2008 CS152-Spring’08 CS252 S05

How many Instructions can be in the pipeline Which features of an ISA limit the number of instructions in the pipeline? Which features of a program limit the number of Ilustrates how one feature alone may not help – happens today when people study single new idea in a very detailed model. 3/13/2008 CS152-Spring’08 CS252 S05

Overcoming the Lack of Register Names Floating-point pipelines often cannot be kept filled with small number of registers. IBM 360 had only 4 floating-point registers Can a microarchitecture use more registers than specified by the ISA without loss of ISA compatibility ? Robert Tomasulo of IBM suggested an ingenious solution in 1967 based on on-the-fly register renaming 3/13/2008 CS152-Spring’08 CS252 S05

Throughput (T) = Number in Flight (N) / Latency (L) Little’s Law Throughput (T) = Number in Flight (N) / Latency (L) WB Execution Issue Example: --- 4 floating-point registers --- 8 cycles per floating-point operation ½ issue per cycle! 3/13/2008 CS152-Spring’08 CS252 S05

Instruction-level Parallelism via Renaming latency 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 DIVD F4’, F2, F8 4 6 ADDD F10, F6, F4’ 1 1 2 3 4 5 6 X In-order: 1 (2,1) . . . . . . 2 3 4 4 3 5 . . . 5 6 6 Out-of-order: 1 (2,1) 4 4 5 . . . 2 (3,5) 3 6 6 Any antidependence can be eliminated by renaming. (renaming  additional storage) Can it be done in hardware? yes! 3/13/2008 CS152-Spring’08 CS252 S05

Register Renaming IF ID WB ALU Mem Fadd Fmul Issue Decode does register renaming and adds instructions to the issue stage reorder buffer (ROB)  renaming makes WAR or WAW hazards impossible Any instruction in ROB whose RAW hazards have been satisfied can be dispatched.  Out-of-order or dataflow execution 3/13/2008 CS152-Spring’08 CS252 S05

Dataflow execution Reorder buffer ptr2 next to deallocate prt1 next available Ins# use exec op p1 src1 p2 src2 t1 t2 . tn Reorder buffer Instruction slot is candidate for execution when: It holds a valid instruction (“use” bit is set) It has not already started execution (“exec” bit is clear) Both operands are available (p1 and p2 are set) 3/13/2008 CS152-Spring’08 CS252 S05

Renaming & Out-of-order Issue An example Renaming table Reorder buffer Ins# use exec op p1 src1 p2 src2 t1 t2 t3 t4 t5 . data / ti p data F1 F2 F3 F4 F5 F6 F7 F8 1 1 1 LD 1 1 0 LD t1 v1 2 0 2 1 0 LD 2 1 1 LD v1 3 1 0 MUL 1 v2 1 v1 3 1 0 MUL 0 t2 1 v1 t2 t5 4 0 4 1 1 SUB 1 v1 1 v1 4 1 0 SUB 1 v1 1 v1 5 1 0 DIV 1 v1 1 v4 5 1 0 DIV 1 v1 0 t4 t3 t4 v4 1 LD F2, 34(R2) 2 LD F4, 45(R3) 3 MULTD F6, F4, F2 4 SUBD F8, F2, F2 5 DIVD F4, F2, F8 6 ADDD F10, F6, F4 When are names in sources replaced by data? When can a name be reused? 3/13/2008 CS152-Spring’08 CS252 S05

Data-Driven Execution Renaming table & reg file Ins# use exec op p1 src1 p2 src2 t1 t2 . tn Reorder buffer Replacing the tag by its value is an expensive operation Load Unit Store Unit FU FU < t, result > Instruction template (i.e., tag t) is allocated by the Decode stage, which also stores the tag in the reg file When an instruction completes, its tag is deallocated 3/13/2008 CS152-Spring’08 CS252 S05

Simplifying Allocation/Deallocation ptr2 next to deallocate prt1 next available Ins# use exec op p1 src1 p2 src2 t1 t2 . tn Reorder buffer Instruction buffer is managed circularly “exec” bit is set when instruction begins execution When an instruction completes its “use” bit is marked free ptr2 is incremented only if the “use” bit is marked free 3/13/2008 CS152-Spring’08 CS252 S05

IBM 360/91 Floating Point Unit R. M. Tomasulo, 1967 2 3 4 5 6 data load buffers (from memory) ... instructions 1 2 3 4 p data Floating Point Reg distribute instruction templates by functional units Adder p data 1 2 3 Mult p data 1 2 < t, result > Common bus ensures that data is made available immediately to all the instructions waiting for it p data store buffers (to memory) 3/13/2008 CS152-Spring’08 CS252 S05

Effectiveness? Renaming and Out-of-order execution was first implemented in 1969 in IBM 360/91 but did not show up in the subsequent models until mid-Nineties. Why ? Reasons 1. Effective on a very small class of programs 2. Memory latency a much bigger problem 3. Exceptions not precise! One more problem needed to be solved 3/13/2008 CS152-Spring’08 CS252 S05

Precise Interrupts It must appear as if an interrupt is taken between two instructions (say Ii and Ii+1) the effect of all instructions up to and including Ii is totally complete no effect of any instruction after Ii has taken place The interrupt handler either aborts the program or restarts it at Ii+1 . 3/13/2008 CS152-Spring’08 CS252 S05

Effect on Interrupts Out-of-order Completion I1 DIVD f6, f6, f4 I2 LD f2, 45(r3) I3 MULTD f0, f2, f4 I4 DIVD f8, f6, f2 I5 SUBD f10, f0, f6 I6 ADDD f6, f8, f2 out-of-order comp 1 2 2 3 1 4 3 5 5 4 6 6 restore f2 restore f10 Consider interrupts Precise interrupts are difficult to implement at high speed - want to start execution of later instructions before exception checks finished on earlier instructions 3/13/2008 CS152-Spring’08 CS252 S05

Exception Handling (In-Order Five-Stage Pipeline) Asynchronous Interrupts Exc D PC Inst. Mem Decode E M Data Mem W + Cause EPC Kill D Stage Kill F Stage Kill E Stage Illegal Opcode Overflow Data Addr Except PC Address Exceptions Kill Writeback Select Handler PC Commit Point Hold exception flags in pipeline until commit point (M stage) Exceptions in earlier pipe stages override later exceptions Inject external interrupts at commit point (override others) If exception at commit: update Cause and EPC registers, kill all stages, inject handler PC into fetch stage 3/13/2008 CS152-Spring’08 CS252 S05

Phases of Instruction Execution PC Fetch: Instruction bits retrieved from cache. I-cache Fetch Buffer Decode: Instructions placed in appropriate issue (aka “dispatch”) stage buffer Issue Buffer Execute: Instructions and operands sent to execution units . When execution completes, all results and exception flags are available. Func. Units Result Buffer Commit: Instruction irrevocably updates architectural state (aka “graduation” or “completion”). Arch. State 3/13/2008 CS152-Spring’08 CS252 S05

In-Order Commit for Precise Exceptions Out-of-order In-order Commit Fetch Decode Reorder Buffer Kill Kill Kill Exception? Execute Inject handler PC Instructions fetched and decoded into instruction reorder buffer in-order Execution is out-of-order (  out-of-order completion) Commit, write-back to architectural state, i.e., regfile & memory, is in-order Temporary storage needed to hold results before commit (shadow registers and store buffers) 3/13/2008 CS152-Spring’08 CS252 S05

Extensions for Precise Exceptions Inst# use exec op p1 src1 p2 src2 pd dest data cause ptr2 next to commit ptr1 next available Reorder buffer add <pd, dest, data, cause> fields in the instruction template commit instructions to reg file and memory in program order  buffers can be maintained circularly on exception, clear reorder buffer by resetting ptr1=ptr2 (stores must wait for commit before updating memory) 3/13/2008 CS152-Spring’08 CS252 S05

Rollback and Renaming Register File Reorder buffer < t, result > (now holds only committed state) Reorder buffer Load Unit FU Store < t, result > t1 t2 . tn Ins# use exec op p1 src1 p2 src2 pd dest data Commit Register file does not contain renaming tags any more. How does the decode stage find the tag of a source register? 3/13/2008 CS152-Spring’08 CS252 S05

Renaming Table tag Register File Rename valid bit Table Reorder buffer Load Unit FU Store < t, result > t1 t2 . tn Ins# use exec op p1 src1 p2 src2 pd dest data Commit Renaming table is a cache to speed up register name look up. It needs to be cleared after each exception taken. When else are valid bits cleared? 3/13/2008 CS152-Spring’08 CS252 S05

Acknowledgements These slides contain material developed and copyright by: Arvind (MIT) Krste Asanovic (MIT/UCB) Joel Emer (Intel/MIT) James Hoe (CMU) John Kubiatowicz (UCB) David Patterson (UCB) MIT material derived from course 6.823 UCB material derived from course CS252 3/13/2008 CS152-Spring’08 CS252 S05