1. ENEE350 Ankur Srivastava University of Maryland, College Park Based on Slides from Mary Jane Irwin ( www.cse.psu.edu/~mji )www.cse.psu.edu/~mji www.cse.psu.edu/~cg431 [Adapted from Computer Organization and Design, Patterson & Hennessy, © 2005, UCB] Computer Systems Organization: Lecture 6

2. ENEE350 Single Cycle Datapath with Control Unit Read Address Instr[31-0] Instruction Memory Add PC 4 Write Data Read Addr 1 Read Addr 2 Write Addr Register File Read Data 1 Read Data 2 ALU ovf zero RegWrite Data Memory Address Write Data Read Data MemWrite MemRead Sign Extend 1632 MemtoReg ALUSrc Shift left 2 Add PCSrc RegDst ALU control 1 1 1 0 0 0 0 1 ALUOp Instr[5-0] Instr[15-0] Instr[25-21] Instr[20-16] Instr[15 -11] Control Unit Instr[31-26] Branch

3. ENEE350 What is Pipelining ALU Instr Mem Reg Data Mem Reg Instruc Fetch Decode Exec Mem WB 5 Basic Steps in MIPS 1.Instruction Fetch from Memory 2.Instruction Decode by the control and Read the Rs and Rt Registers, Perform Sign Extension 3.Execute the Computation in the ALU, Calculate the target Branch Address 4.Memory Access: Read of Write data from/into memory 5.Write Back the result from the ALU or MemeorY Read into the register file

4. ENEE350 What is Pipelining ALU Instr Mem Reg Data Mem Reg Instruc Fetch Decode Exec Mem WB Pipelining: Start the next instruction before the current one has completed. Hence when the first instruction is being decoded, start fetching the next instructions and so on.

5. ENEE350 A Pipelined MIPS Processor  Start the next instruction before the current one has completed l improves throughput - total amount of work done in a given time l instruction latency (execution time, delay time, response time - time from the start of an instruction to its completion) is not reduced Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5 IFetchDecExecMemWB lw Cycle 7Cycle 6Cycle 8 sw IFetchDecExecMemWB R-type IFetchDecExecMemWB -clock cycle (pipeline stage time) is limited by the slowest stage -for some instructions, some stages are wasted cycles

6. ENEE350 Single Cycle, Multiple Cycle, vs. Pipeline lw IFetchDecExecMemWB Pipeline Implementation: IFetchDecExecMemWB sw IFetchDecExecMemWB R-type Clk Single Cycle Implementation: lwsw Waste Cycle 1Cycle 2

7. ENEE350 What is Pipelining ALU Instr Mem Reg Data Mem Reg Instruc Fetch Decode Exec Mem WB In Order to Implement Pipelining, we will need to redesign the datapath so that we can start executing the next instruction when the first one is still being processed. The first thing to do is to design the data-path such that each of the 5 steps could work independently

8. ENEE350 What is Pipelining ALU Instr Mem Reg Data Mem Reg This is achieved by having what are called as pipeline registers in between the 5 stages. The clock delay is long enough to finish any of these steps.

9. ENEE350 What is Pipelining ALU Instr Mem Reg Data Mem Reg Let us suppose Instruction Fetch : 200ps, Instruction Decode: 100ps, ALU Operation: 200ps, Mem Access: 200ps, Register Write Back: 100ps. Single Cycle Architecture: Clock Delay: 800ps Pipeline Clock Delay: 200ps

10. ENEE350 What is Pipelining IFetchDecExecMemWB lw sw IFetchDecExecMemWB R-type IFetchDecExecMemWB Execution time on a single cycle processor: 800*3 = 2400 Execution time on a pipelined processor: 200*5 + 200 + 200 = 1400

11. ENEE350 Single Cycle Datapath with Control Unit Read Address Instr[31-0] Instruction Memory Add PC 4 Write Data Read Addr 1 Read Addr 2 Write Addr Register File Read Data 1 Read Data 2 ALU ovf zero RegWrite Data Memory Address Write Data Read Data MemWrite MemRead Sign Extend 1632 MemtoReg ALUSrc Shift left 2 Add PCSrc RegDst ALU control 1 1 1 0 0 0 0 1 ALUOp Instr[5-0] Instr[15-0] Instr[25-21] Instr[20-16] Instr[15 -11] Control Unit Instr[31-26] Branch

12. ENEE350 MIPS Pipeline Datapath Modifications  What do we need to add/modify in our MIPS datapath? l State registers between each pipeline stage to isolate them l For simplicity the entire datapath is not shown

13. ENEE350 Corrected Datapath to Save RegWrite Addr  Need to preserve the destination register address in the pipeline state registers

14. ENEE350 Corrected Datapath to Save RegWrite Addr  Need to preserve the destination register address in the pipeline state registers

15. ENEE350 MIPS Pipeline Control Path Modifications  All control signals can be determined during Decode l and held in the state registers between pipeline stages Control

16. ENEE350 Pipelining the MIPS ISA  What makes it easy l all instructions are the same length (32 bits) -can fetch in the 1 st stage and decode in the 2 nd stage l few instruction formats (three) with symmetry across formats -can begin reading register file in 2 nd stage l memory operations can occur only in loads and stores -can use the execute stage to calculate memory addresses l each MIPS instruction writes at most one result (i.e., changes the machine state) and does so near the end of the pipeline (MEM and WB)  What makes it hard l structural hazards: what if we had only one memory? l control hazards: what about branches? l data hazards: what if an instruction’s input operands depend on the output of a previous instruction?

17. ENEE350 Why Pipeline? For Performance! I n s t r. O r d e r Time (clock cycles) Inst 0 Inst 1 Inst 2 Inst 4 Inst 3 ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg Once the pipeline is full, one instruction is completed every cycle, so CPI = 1 Time to fill the pipeline

18. ENEE350 Can Pipelining Get Us Into Trouble?  Yes: Pipeline Hazards l structural hazards: attempt to use the same resource by two different instructions at the same time l data hazards: attempt to use data before it is ready -An instruction’s source operand(s) are produced by a prior instruction still in the pipeline l control hazards: attempt to make a decision about program control flow before the condition has been evaluated and the new PC target address calculated -branch instructions  Can always resolve hazards by waiting l pipeline control must detect the hazard l and take action to resolve hazards

19. ENEE350 I n s t r. O r d e r Time (clock cycles) lw Inst 1 Inst 2 Inst 4 Inst 3 ALU Mem Reg MemReg ALU Mem Reg MemReg ALU Mem Reg MemReg ALU Mem Reg MemReg ALU Mem Reg MemReg A Single Memory Would Be a Structural Hazard Reading data from memory Reading instruction from memory  Fix with separate instr and data memories (I$ and D$)

20. ENEE350 How About Register File Access? I n s t r. O r d e r Time (clock cycles) add $1, Inst 1 Inst 2 add $2,$1, ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg

21. ENEE350 How About Register File Access? I n s t r. O r d e r Time (clock cycles) Inst 1 Inst 2 ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg Fix register file access hazard by doing reads in the second half of the cycle and writes in the first half add $1, add $2,$1, clock edge that controls register writing clock edge that controls loading of pipeline state registers

22. ENEE350 Register Usage Can Cause Data Hazards I n s t r. O r d e r add $1, sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg  Dependencies backward in time cause hazards  Read before write data hazard

23. ENEE350 Register Usage Can Cause Data Hazards ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg  Dependencies backward in time cause hazards add $1,$x,$y sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9  Read before write data hazard

24. ENEE350 Loads Can Cause Data Hazards I n s t r. O r d e r lw $1,4($2) sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg  Dependencies backward in time cause hazards  Load-use data hazard

25. ENEE350 stall One Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, ALU IM Reg DMReg sub $4,$1,$5 and $6,$1,$7 ALU IM Reg DMReg ALU IM Reg DMReg Can fix data hazard by waiting – stall – but impacts CPI

26. ENEE350 Another Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, ALU IM Reg DMReg sub $4,$1,$5 and $6,$1,$7 ALU IM Reg DMReg ALU IM Reg DMReg Fix data hazards by forwarding results as soon as they are available to where they are needed xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DMReg ALU IM Reg DMReg

27. ENEE350 Another Way to “Fix” a Data Hazard ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg Fix data hazards by forwarding results as soon as they are available to where they are needed ALU IM Reg DMReg ALU IM Reg DMReg I n s t r. O r d e r add $1, sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9

28. ENEE350 Forwarding with Load-use Data Hazards I n s t r. O r d e r lw $1,4($2) sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg

29. ENEE350 Forwarding with Load-use Data Hazards ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg  Will still need one stall cycle even with forwarding I n s t r. O r d e r lw $1,4($2) sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9

30. ENEE350 Branch Instructions Cause Control Hazards I n s t r. O r d e r lw Inst 4 Inst 3 beq ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg ALU IM Reg DMReg  Dependencies backward in time cause hazards

31. ENEE350 stall One Way to “Fix” a Control Hazard I n s t r. O r d e r beq ALU IM Reg DMReg lw ALU IM Reg DMReg ALU Inst 3 IM Reg DM Fix branch hazard by waiting – stall – but affects CPI

32. ENEE350 Summary  All modern day processors use pipelining  Pipelining doesn’t help latency of single task, it helps throughput of entire workload  Potential speedup: a CPI of 1 and fast a CC  Pipeline rate limited by slowest pipeline stage l Unbalanced pipe stages makes for inefficiencies l The time to “fill” pipeline and time to “drain” it can impact speedup for deep pipelines and short code runs  Must detect and resolve hazards l Stalling negatively affects CPI (makes CPI less than the ideal of 1)