1. CPE 442 pipeline.1 Intro to Computer Architecture CpE 242 Computer Architecture and Engineering Designing a Pipeline Processor

2. CPE 442 pipeline.2 Intro to Computer Architecture A Single Cycle Processor 32 ALUctr Clk busW RegWr 32 busA 32 busB 555 RwRaRb 32 32-bit Registers Rs Rt Rd RegDst Extender Mux 32 16 imm16 ALUSrc ExtOp Mux MemtoReg Clk Data In WrEn 32 Adr Data Memory 32 MemWr ALU Zero 0 1 0 1 01 Instruction Fetch Unit Clk Instruction Jump Branch Imm16 Rd Main Control op ALU Control func ALUop 3 RegDst ALUSrc : Instr Zero 3

3. CPE 442 pipeline.3 Intro to Computer Architecture Drawbacks of this Single Cycle Processor °Long cycle time: Cycle time must be long enough for the load instruction: -PC’s Clock -to-Q + -Instruction Memory Access Time + -Register File Access Time + -ALU Delay (address calculation) + -Data Memory Access Time + -Register File Setup Time + -Clock Skew °Cycle time is much longer than needed for all other instructions. Examples: R-type instructions do not require data memory access Jump does not require ALU operation nor data memory access

4. CPE 442 pipeline.4 Intro to Computer Architecture Overview of a Multiple Cycle Implementation °The root of the single cycle processor’s problems: The cycle time has to be long enough for the slowest instruction °Solution: Break the instruction into smaller steps Execute each step (instead of the entire instruction) in one cycle -Cycle time: time it takes to execute the longest step -Keep all the steps to have similar length This is the essence of the multiple cycle processor °The advantages of the multiple cycle processor: Cycle time is much shorter Different instructions take different number of cycles to complete -Load takes five cycles -Jump only takes three cycles Allows a functional unit to be used more than once per instruction

5. CPE 442 pipeline.5 Intro to Computer Architecture Multiple Cycle Processor °MCP: A functional unit to be used more than once per instruction Ideal Memory WrAdr Din RAdr 32 Dout MemWr 32 ALU 32 ALUOp ALU Control Instruction Reg 32 IRWr 32 Reg File Ra Rw busW Rb 5 5 32 busA 32busB RegWr Rs Rt Mux 0 1 Rt Rd PCWr ALUSelA Mux 01 RegDst Mux 0 1 32 PC MemtoReg Extend ExtOp Mux 0 1 32 0 1 2 3 4 16 Imm 32 << 2 ALUSelB Mux 1 0 Target 32 Zero PCWrCondPCSrcBrWr 32 IorD

6. CPE 442 pipeline.6 Intro to Computer Architecture Outline of Today’s Lecture °Recap and Introduction (5 minutes) °Introduction to the Concept of Pipelined Processor (15 minutes) °Pipelined Datapath and Pipelined Control (25 minutes) °How to Avoid Race Condition in a Pipeline Design? (5 minutes) °Pipeline Example: Instructions Interaction (15 minutes) °Summary (5 minutes)

7. CPE 442 pipeline.7 Intro to Computer Architecture Timing Diagram of a Load Instruction Clk PC Rs, Rt, Rd, Op, Func Clk-to-Q ALUctr Instruction Memory Access Time Old ValueNew Value RegWrOld ValueNew Value Delay through Control Logic busA Register File Access Time Old ValueNew Value busB ALU Delay Old ValueNew Value Old ValueNew Value Old Value ExtOpOld ValueNew Value ALUSrcOld ValueNew Value AddressOld ValueNew Value busWOld ValueNew Delay through Extender & Mux Data Memory Access Time Instruction FetchInstr Decode / Reg. Fetch AddressReg WrData Memory Register File Write Time

8. CPE 442 pipeline.8 Intro to Computer Architecture The Five Stages of Load °Ifetch: Instruction Fetch Fetch the instruction from the Instruction Memory °Reg/Dec: Registers Fetch and Instruction Decode °Exec: Calculate the memory address °Mem: Read the data from the Data Memory °Wr: Write the data back to the register file Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5 IfetchReg/DecExecMemWrLoad

9. CPE 442 pipeline.9 Intro to Computer Architecture Key Ideas Behind Pipelining °Grading the Final exam for a class of 100 students: 5 problems, five people grading the exam Each person ONLY grade one problem Pass the exam to the next person as soon as one finishes his part Assume each problem takes 12 min to grade -Each individual exam still takes 1 hour to grade -But with 5 people, all exams can be graded five times quicker °The load instruction has 5 stages: Five independent functional units to work on each stage -Each functional unit is used only once The 2nd load can start as soon as the 1st finishes its Ifetch stage Each load still takes five cycles to complete The throughput, however, is much higher

10. CPE 442 pipeline.10 Intro to Computer Architecture Key Ideas Behind Pipelining °Let n be number of tasks or exams (or instructions) °Let k be number of stages for each task °Let T be the time per stage °Time per task = T. k °Total Time per n tasks for non-pipelined solution = T. k. n °Total Time per n tasks for pipelined solution = T. k + T. (n-1) °Speedup = pipelined perform/ non-pipelined performance = Total Time non-pipelined/ Total Time for pipelined = k. n / k + n-1 = k approx. when n >> k Input Tasks K – stage pipeline buffer Stage 1Stage 2Stage k

11. CPE 442 pipeline.11 Intro to Computer Architecture Pipelining the Load Instruction °The five independent functional units in the pipeline datapath are: Instruction Memory for the Ifetch stage Register File’s Read ports (bus A and busB) for the Reg/Dec stage ALU for the Exec stage Data Memory for the Mem stage Register File’s Write port (bus W) for the Wr stage °One instruction enters the pipeline every cycle One instruction comes out of the pipeline (complete) every cycle The “Effective” Cycles per Instruction (CPI) is 1 Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7 IfetchReg/DecExecMemWr1st lw IfetchReg/DecExecMemWr2nd lw IfetchReg/DecExecMemWr3rd lw

12. CPE 442 pipeline.12 Intro to Computer Architecture The Four Stages of R-type °Ifetch: Instruction Fetch Fetch the instruction from the Instruction Memory °Reg/Dec: Registers Fetch and Instruction Decode °Exec: ALU operates on the two register operands °Wr: Write the ALU output back to the register file Cycle 1Cycle 2Cycle 3Cycle 4 IfetchReg/DecExecWrR-type

13. CPE 442 pipeline.13 Intro to Computer Architecture Pipelining the R-type and Load Instruction °We have a problem: Two instructions try to write to the register file at the same time! Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9 IfetchReg/DecExecWrR-type IfetchReg/DecExecWrR-type IfetchReg/DecExecMemWrLoad IfetchReg/DecExecWrR-type IfetchReg/DecExecWrR-type Ops! We have a problem!

14. CPE 442 pipeline.14 Intro to Computer Architecture Important Observation °Each functional unit can only be used once per instruction °Each functional unit must be used at the same stage for all instructions: Load uses Register File’s Write Port during its 5th stage R-type uses Register File’s Write Port during its 4th stage IfetchReg/DecExecMemWrLoad 12345 IfetchReg/DecExecWrR-type 1234

15. CPE 442 pipeline.15 Intro to Computer Architecture Solution 1: Insert “Bubble” into the Pipeline °Insert a “bubble” into the pipeline to prevent 2 writes at the same cycle The control logic can be complex °No instruction is completed during Cycle 5: The “Effective” CPI for load is 2 Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9 IfetchReg/DecExecWrR-type IfetchReg/DecExec IfetchReg/DecExecMemWrLoad IfetchReg/DecExecWr R-type IfetchReg/DecExecWr R-type Pipeline Bubble IfetchReg/DecExecWr

16. CPE 442 pipeline.16 Intro to Computer Architecture Solution 2: Delay R-type’s Write by One Cycle °Delay R-type’s register write by one cycle: Now R-type instructions also use Reg File’s write port at Stage 5 Mem stage is a NOOP stage: nothing is being done Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9 IfetchReg/DecMemWrR-type IfetchReg/DecMemWrR-type IfetchReg/DecExecMemWrLoad IfetchReg/DecMemWrR-type IfetchReg/DecMemWrR-type IfetchReg/DecExecWrR-typeMem Exec 12345

17. CPE 442 pipeline.17 Intro to Computer Architecture The Four Stages of Store °Ifetch: Instruction Fetch Fetch the instruction from the Instruction Memory °Reg/Dec: Registers Fetch and Instruction Decode °Exec: Calculate the memory address °Mem: Write the data into the Data Memory Cycle 1Cycle 2Cycle 3Cycle 4 IfetchReg/DecExecMemStoreWr

18. CPE 442 pipeline.18 Intro to Computer Architecture The Four Stages of Beq °Ifetch: Instruction Fetch Fetch the instruction from the Instruction Memory °Reg/Dec: Registers Fetch and Instruction Decode °Exec: ALU compares the two register operands Adder calculates the branch target address °Mem: If the registers we compared in the Exec stage are the same, Write the branch target address into the PC Cycle 1Cycle 2Cycle 3Cycle 4 IfetchReg/DecExecMemBeqWr

19. CPE 442 pipeline.19 Intro to Computer Architecture A Pipelined Datapath Clk IfetchReg/DecExecMemWr

20. CPE 442 pipeline.20 Intro to Computer Architecture The Instruction Fetch Stage IF/ID: lw $1, 100 ($2) ID/Ex Register Ex/Mem Register Mem/Wr Register PC = 14 Data Mem WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr RegWr ExtOp Exec Unit busA busB Imm16 ALUOp ALUSrc Mux 1 0 MemtoReg 1 0 RegDst Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 Clk IfetchReg/DecExecMem You are here! °Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100]

21. CPE 442 pipeline.21 Intro to Computer Architecture A Detail View of the Instruction Unit °Location 10: lw $1, 0x100($2) IF/ID: lw $1, 100 ($2) PC = 14 1 0 10 Adder Instruction Memory “4” Instruction Address Clk Ifetch You are here! Reg/Dec

22. CPE 442 pipeline.22 Intro to Computer Architecture The Decode / Register Fetch Stage IF/ID: ID/Ex: Reg. 2 & 0x100 Ex/Mem Register Mem/Wr Register PC Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr RegWr ExtOp Exec Unit busA busB Imm16 ALUOp ALUSrc Mux 1 0 MemtoReg 1 0 RegDst Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 Clk IfetchReg/DecExecMem You are here! °Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100]

23. CPE 442 pipeline.23 Intro to Computer Architecture Load’s Address Calculation Stage IF/ID: ID/Ex Register Ex/Mem: Load’s Address Mem/Wr Register PC Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr RegWr ExtOp=1 Exec Unit busA busB Imm16 ALUOp=Add ALUSrc=1 Mux 1 0 MemtoReg 1 0 RegDst=0 Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 Clk IfetchReg/DecExecMem You are here! °Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100]

24. CPE 442 pipeline.24 Intro to Computer Architecture A Detail View of the Execution Unit ID/Ex Register Ex/Mem: Load’s Memory Address ALU Control ALUctr 32 busA 32 busB Extender Mux 16 imm16 ALUSrc=1 ExtOp=1 3 ALU Zero 0 1 32 ALUout 32 Adder 3 ALUOp=Add << 2 32 PC+4 Target 32 Clk Exec You are here! Mem

25. CPE 442 pipeline.25 Intro to Computer Architecture Load’s Memory Access Stage IF/ID: ID/Ex Register Ex/Mem Register Mem/Wr: Load’s Data PC Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr=0 RegWr ExtOp Exec Unit busA busB Imm16 ALUOp ALUSrc Mux 1 0 MemtoReg 1 0 RegDst Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch=0 1 0 Clk IfetchReg/DecExecMem You are here! °Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100]

26. CPE 442 pipeline.26 Intro to Computer Architecture Load’s Write Back Stage IF/ID: ID/Ex Register Ex/Mem Register Mem/Wr Register PC Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr RegWr=1 ExtOp Exec Unit busA busB Imm16 ALUOp ALUSrc Mux 1 0 MemtoReg=1 1 0 RegDst Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 Clk IfetchReg/DecExecMem You are somewhere out there! °Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100] Wr

27. CPE 442 pipeline.27 Intro to Computer Architecture How About Control Signals? IF/ID: ID/Ex Register Ex/Mem: Load’s Address Mem/Wr Register PC Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr RegWr ExtOp=1 Exec Unit busA busB Imm16 ALUOp=Add ALUSrc=1 Mux 1 0 MemtoReg 1 0 RegDst=0 Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 IfetchReg/DecExecMem °Key Observation: Control Signals at Stage N = Func (Instr. at Stage N) N = Exec, Mem, or Wr °Example: Controls Signals at Exec Stage = Func(Load’s Exec) Wr

28. CPE 442 pipeline.28 Intro to Computer Architecture Pipeline Control °The Main Control generates the control signals during Reg/Dec Control signals for Exec (ExtOp, ALUSrc,...) are used 1 cycle later Control signals for Mem (MemWr Branch) are used 2 cycles later Control signals for Wr (MemtoReg MemWr) are used 3 cycles later IF/ID Register ID/Ex Register Ex/Mem Register Mem/Wr Register Reg/DecExecMem ExtOp ALUOp RegDst ALUSrc Branch MemWr MemtoReg RegWr Main Control ExtOp ALUOp RegDst ALUSrc MemtoReg RegWr MemtoReg RegWr MemtoReg RegWr Branch MemWr Branch MemWr Wr

29. CPE 442 pipeline.29 Intro to Computer Architecture Beginning of the Wr’s Stage: A Real World Problem °At the beginning of the Wr stage, we have a problem if: RegAdr’s (Rd or Rt) Clk-to-Q > RegWr’s Clk-to-Q °Similarly, at the beginning of the Mem stage, we have a problem if: WrAdr’s Clk-to-Q > MemWr’s Clk-to-Q °We have a race condition between Address and Write Enable! Ex/Mem Mem/Wr RegAdr RegWrMemWr Data WrAdr Data Reg File Data Memory Clk RegAdr RegWr RegWr’s Clk-to-Q RegAdr’s Clk-to-Q Clk WrAdr MemWr MemWr’s Clk-to-Q WrAdr’s Clk-to-Q

30. CPE 442 pipeline.30 Intro to Computer Architecture The Pipeline Problem °Multiple Cycle design prevents race condition between Addr and WrEn: Make sure Address is stable by the end of Cycle N Asserts WrEn during Cycle N + 1 °This approach can NOT be used in the pipeline design because: Must be able to write the register file every cycle Must be able write the data memory every cycle Clock IfetchReg/DecExecMemWrStore IfetchReg/DecExecMemWrStore IfetchReg/DecExecMemWrR-type IfetchReg/DecExecMemWrR-type

31. CPE 442 pipeline.31 Intro to Computer Architecture Synchronize Register File & Synchronize Memory °Solution: And the Write Enable signal with the Clock This is the ONLY place where gating the clock is used MUST consult circuit expert to ensure no timing violation: -Example: Clock High Time > Write Access Delay WrEn I_Addr I_Data Reg File or Memory Clk I_Addr I_WrEn Address Data I_WrEn C_WrEn Clk Address Data WrEn Reg File or Memory Synchronize Memory and Register File Address, Data, and WrEn must be stable at least 1 set-up time before the Clk edge Write occurs at the cycle following the clock edge that captures the signals

32. CPE 442 pipeline.32 Intro to Computer Architecture A More Extensive Pipelining Example Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8 IfetchReg/DecExecMemWr0: Load IfetchReg/DecExecMemWr4: R-type IfetchReg/DecExecMemWr8: Store IfetchReg/DecExecMemWr12: Beq (target is 1000) End of Cycle 4 End of Cycle 5 End of Cycle 6 End of Cycle 7 °End of Cycle 4: Load’s Mem, R-type’s Exec, Store’s Reg, Beq’s Ifetch °End of Cycle 5: Load’s Wr, R-type’s Mem, Store’s Exec, Beq’s Reg °End of Cycle 6: R-type’s Wr, Store’s Mem, Beq’s Exec °End of Cycle 7: Store’s Wr, Beq’s Mem

33. CPE 442 pipeline.33 Intro to Computer Architecture Pipelining Example: End of Cycle 4 °0: Load’s Mem 4: R-type’s Exec 8: Store’s Reg 12: Beq’s Ifetch IF/ID: Beq Instruction ID/Ex: Store’s busA & B Ex/Mem: R-type’s Result Mem/Wr: Load’s Dout PC = 16 Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw RegWr=0 ExtOp=x Exec Unit busA busB Imm16 ALUOp=R-type ALUSrc=0 Mux 1 0 MemtoReg=x 1 0 RegDst=1 Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch=0 1 0 12: Beq’s Ifet 8: Store’s Reg4: R-type’s Exec0: Load’s Mem Clk MemWr=0 Clk

34. CPE 442 pipeline.34 Intro to Computer Architecture Pipelining Example: End of Cycle 5 °0: Lw’s Wr 4: R’s Mem 8: Store’s Exec 12: Beq’s Reg 16: R’s Ifetch IF/ID: Instruction @ 16 ID/Ex: Beq’s busA & B Ex/Mem: Store’s Address Mem/Wr: R-type’s Result PC = 20 Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw RegWr=1 ExtOp=1 Exec Unit busA busB Imm16 ALUOp=Add ALUSrc=1 Mux 1 0 MemtoReg=1 1 0 RegDst=x Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch=0 1 0 16: R’s Ifet 12: Beq’s Reg8: Store’s Exec4: R-type’s Mem 0: Load’s Wr Clk MemWr=0 Clk

35. CPE 442 pipeline.35 Intro to Computer Architecture Pipelining Example: End of Cycle 6 °4: R’s Wr 8: Store’s Mem 12: Beq’s Exec 16: R’s Reg 20: R’s Ifet IF/ID: Instruction @ 20 ID/Ex:R-type’s busA & B Ex/Mem: Beq’s Results Mem/Wr: Nothing for St PC = 24 Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw RegWr=1 ExtOp=1 Exec Unit busA busB Imm16 ALUOp=Sub ALUSrc=0 Mux 1 0 MemtoReg=0 1 0 RegDst=x Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch=0 1 0 20: R-type’s Ifet 16: R-type’s Reg12: Beq’s Exec8: Store’s Mem 4: R-type’s Wr Clk MemWr=1 Clk

36. CPE 442 pipeline.36 Intro to Computer Architecture Pipelining Example: End of Cycle 7 °8: Store’s Wr 12: Beq’s Mem 16: R’s Exec 20: R’s Reg 24: R’s Ifet IF/ID: Instruction @ 24 ID/Ex:R-type’s busA & B Ex/Mem: Rtype’s Results Mem/Wr:Nothing for Beq PC = 1000 Data Me m WA Di RADo IUnit A I RFile Di Ra Rb Rw RegWr=0 ExtOp=x Exec Unit busA busB Imm16 ALUOp=R-type ALUSrc=0 Mux 1 0 MemtoReg=x 1 0 RegDst=1 Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch=1 1 0 24: R-type’s Ifet 20: R-type’s Reg16: R-type’s Exec12: Beq’s Mem 8: Store’s Wr Clk MemWr=0 Clk

37. CPE 442 pipeline.37 Intro to Computer Architecture The Delay Branch Phenomenon °Although Beq is fetched during Cycle 4: Target address is NOT written into the PC until the end of Cycle 7 Branch’s target is NOT fetched until Cycle 8 3-instruction delay before the branch take effect °This is referred to as Branch Hazard: Clever design techniques can reduce the delay to ONE instruction Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9Cycle 10Cycle 11 IfetchReg/DecExecMemWr IfetchReg/DecExecMemWr 16: R-type IfetchReg/DecExecMemWr IfetchReg/DecExecMemWr24: R-type 12: Beq (target is 1000) 20: R-type Clk IfetchReg/DecExecMemWr1000: Target of Br

38. CPE 442 pipeline.38 Intro to Computer Architecture The Delay Load Phenomenon °Although Load is fetched during Cycle 1: The data is NOT written into the Reg File until the end of Cycle 5 We cannot read this value from the Reg File until Cycle 6 3-instruction delay before the load take effect °This is referred to as Data Hazard: Clever design techniques can reduce the delay to ONE instruction Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8 IfetchReg/DecExecMemWrI0: Load IfetchReg/DecExecMemWrPlus 1 IfetchReg/DecExecMemWrPlus 2 IfetchReg/DecExecMemWrPlus 3 IfetchReg/DecExecMemWrPlus 4

39. CPE 442 pipeline.39 Intro to Computer Architecture Summary °Disadvantages of the Single Cycle Processor Long cycle time Cycle time is too long for all instructions except the Load °Multiple Clock Cycle Processor: Divide the instructions into smaller steps Execute each step (instead of the entire instruction) in one cycle °Pipeline Processor: Natural enhancement of the multiple clock cycle processor Each functional unit can only be used once per instruction If a instruction is going to use a functional unit: -it must use it at the same stage as all other instructions Pipeline Control: -Each stage’s control signal depends ONLY on the instruction that is currently in that stage

40. CPE 442 pipeline.40 Intro to Computer Architecture Single Cycle, Multiple Cycle, vs. Pipeline Clk Cycle 1 Multiple Cycle Implementation: IfetchRegExecMemWr Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9Cycle 10 LoadIfetchRegExecMemWr IfetchRegExecMem LoadStore Pipeline Implementation: IfetchRegExecMemWrStore Clk Single Cycle Implementation: LoadStoreWaste Ifetch R-type IfetchRegExecMemWrR-type Cycle 1Cycle 2

41. CPE 442 pipeline.41 Intro to Computer Architecture Where to get more information? °Everything You Need to know about Pipeline Computer: Peter Kogge, “The Architecture of Pipeline Computers,” McGraw Hill Book Company, 1981 °Some Classic References on RISC Pipelines: Manolis Katevenis, “Reduced Instruction Set Computer Architectures for VLSI,” PhD Thesis, UC Berkeley, 1984. °Other references: David. A Patterson, “Reduced Instruction Set Computers,” Communications of the ACM, January 1985. Shing Kong, “Performance, Resources, and Complexity,” PhD Thesis, UC Berkeley, 1989.