1. CML CML CS 230: Computer Organization and Assembly Language Aviral Shrivastava Department of Computer Science and Engineering School of Computing and Informatics Arizona State University Slides courtesy: Prof. Yann Hang Lee, ASU, Prof. Mary Jane Irwin, PSU, Ande Carle, UCB

2. CML CMLAnnouncements Project 4 –MIPS Simulator –Due Nov 10, 2009 Alternate Project Quiz 5 –Thursday, Nov 19, 2009 –Pipelining Finals –Tuesday, Dec 08, 2009 –Please come on time (You’ll need all the time) –Open book, notes, and internet –No communication with any other human

3. CML CML Benefits of Pipelining Pipeline latches: pass the status and result of the current instruction to next stage Comparison: Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9Cycle 10 Ifetch lw sw Dec/Reg Exec Mem Wr Dec/Reg Exec Mem Ifetch Single- cycle inst. IfetchDec/Reg Exec Mem Wr IfetchDec/Reg Exec Mem Wr IfetchDec/Reg Exec Mem Wr pipelined

4. CML CML Passing control w/pipe registers Analogy: send instruction with car on assembly line –“Install Corinthian leather interior on car 6 @ stage 3” WB M EX WB M Control IF/IDID/EXEX/MEMMEM/WB I n s t r u c t i o n RegDst ALUOp ALUSrc Branch MemRead MemWrite MemtoReg RegWrite strip off signals for execution phase strip off signals for write-back phase strip off signals for memory phase Genera- tion

5. CML CML The hazards of pipelining Pipeline hazards prevent next instruction from executing during designated clock cycle There are 3 classes of hazards: –Structural Hazards: Arise from resource conflicts HW cannot support all possible combinations of instructions –Data Hazards: Occur when given instruction depends on data from an instruction ahead of it in pipeline –Control Hazards: Result from branch, other instructions that change flow of program (i.e. change PC)

6. CML Structural Hazard ALU RegMemDMReg ALU RegMemDMReg ALU RegMemDMReg ALU RegMemDMReg Time ALU RegMemDMReg Load Instruction 1 Instruction 2 Instruction 3 Instruction 4 What’s the problem here? 4/14/20156CSE 420: Computer Architecture

7. CML CML Structural hazards A way to avoid structural hazards is to duplicate resources –i.e.: An ALU to perform an arithmetic operation and an adder to increment PC If not all possible combinations of instructions can be executed, structural hazards occur Most common instances of structural hazards: –When a functional unit not fully pipelined –When some resource not duplicated enough 4/14/20157CSE 420: Computer Architecture

8. CML How is it resolved? ALU RegMemDMReg ALU RegMemDMReg ALU RegMemDMReg Time ALU RegMemDMReg Load Instruction 1 Instruction 2 Stall Instruction 3 Bubble Pipeline generally stalled by inserting a “bubble” or NOP 4/14/20158CSE 420: Computer Architecture

9. CML Or alternatively… Inst. #12345678910 LOADIFIDEXMEMWB Inst. i+1IFIDEXMEMWB Inst. i+2IFIDEXMEMWB Inst. i+3stallIFIDEXMEMWB Inst. i+4IFIDEXMEMWB Inst. i+5IFIDEXMEM Inst. i+6IFIDEX Clock Number LOAD instruction “steals” an instruction fetch cycle which will cause the pipeline to stall. Thus, no instruction completes on clock cycle 8 4/14/20159CSE 420: Computer Architecture

10. CML CML 4/14/2015CSE 420: Computer Architecture 10 Example: Dual-port vs. Single-port Machine A: Dual ported memory (“Harvard Architecture”) Machine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate Ideal CPI = 1 for both Loads are 40% of instructions executed SpeedUp A = Pipeline Depth/(1 + 0) x (clock unpipe /clock pipe ) = Pipeline Depth SpeedUp B = Pipeline Depth/(1 + 0.4 x 1) x (clock unpipe /(clock unpipe / 1.05) = (Pipeline Depth/1.4) x 1.05 = 0.75 x Pipeline Depth SpeedUp A / SpeedUp B = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33 Machine A is 1.33 times faster

11. CML Illustrating a data hazard ALU RegMemDMReg ALU RegMemDMReg ALU RegMemDM RegMem Time ADD R1, R2, R3 SUB R4, R1, R5 AND R6, R1, R7 OR R8, R1, R9 XOR R10, R1, R11 ALU RegMem ADD instruction causes a hazard in next 3 instructions b/c register not written until after those 3 read it.

12. CML CML stall One Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, ALU IM Reg DMReg sub $4,$1,$5 and $6,$7,$1 ALU IM Reg DMReg ALU IM Reg DMReg Fix data hazard by waiting – stall – but impacts CPI

13. CML CML stall One Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, ALU IM Reg DMReg sub $4,$1,$5 and $6,$7,$1 ALU IM Reg DMReg ALU IM Reg DMReg Fix data hazard by waiting – stall – but impacts CPI

14. CML CML Another Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, ALU IM Reg DMReg sub $4,$1,$5 and $6,$7,$1 ALU IM Reg DMReg ALU IM Reg DMReg Fix data hazards by forwarding results as soon as they are available to where they are needed sw $4,4($1) or $8,$1,$1 ALU IM Reg DMReg ALU IM Reg DMReg

15. CML CML Data Forwarding (aka Bypassing) Take the result from the earliest point that it exists in any of the pipeline state registers and forward it to the functional units (e.g., the ALU) that need it that cycle For ALU functional unit: the inputs can come from any pipeline register rather than just from ID/EX by –adding multiplexors to the inputs of the ALU –connecting the Rd write data in EX/MEM or MEM/WB to either (or both) of the EX’s stage Rs and Rt ALU mux inputs –adding the proper control hardware to control the new muxes Other functional units may need similar forwarding logic (e.g., the DM) With forwarding can achieve a CPI of 1 even in the presence of data dependencies

16. CML When can we forward? ALU RegMemDMReg ALU RegMemDMReg ALU RegMemDM RegMem Time ADD R1, R2, R3 SUB R4, R1, R5 AND R6, R1, R7 OR R8, R1, R9 XOR R10, R1, R11 ALU RegMem SUB gets info. from EX/MEM pipe register AND gets info. from MEM/WB pipe register OR gets info. by forwarding from register file Rule of thumb:If line goes “forward” you can do forwarding. If its drawn backward, it’s physically impossible.

17. CML CML Datapath with Forwarding Hardware PCSrc ID/EX.RegisterRt ID/EX.RegisterRs EX/MEM.RegisterRd MEM/WB.RegisterRd ForwardA ForwardB

18. CML CML Data Forwarding Control Conditions 1.EX/MEM hazard: if (EX/MEM.RegWrite and (EX/MEM.RegisterRd != 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) ForwardA = 10 if (EX/MEM.RegWrite and (EX/MEM.RegisterRd != 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRt)) ForwardB = 10 Forwards the result from the previous instr. to either input of the ALU Forwards the result from the second previous instr. to either input of the ALU 2. MEM/WB hazard: if (MEM/WB.RegWrite and (MEM/WB.RegisterRd != 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA = 01 if (MEM/WB.RegWrite and (MEM/WB.RegisterRd != 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB = 01

19. CML CML Yet Another Complication! I n s t r. O r d e r add $1,$6,$2 ALU IM Reg DMReg add $1,$1,$3 add $5,$1,$4 ALU IM Reg DMReg ALU IM Reg DMReg Another potential data hazard can occur when there is a conflict between the result of the WB stage instruction and the MEM stage instruction – which should be forwarded?

20. CML CML Corrected Data Forwarding Control Conditions 2.MEM/WB hazard: if (MEM/WB.RegWrite and (MEM/WB.RegisterRd != 0) and (EX/MEM.RegisterRd != ID/EX.RegisterRs) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA = 01 if (MEM/WB.RegWrite and (MEM/WB.RegisterRd != 0) and (EX/MEM.RegisterRd != ID/EX.RegisterRt) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB = 01

21. CML CML Inserting bubbles PCIF/ID EX/MEM ID/EX MEM/WB WB M EX WB M M u x 0 Hazard Detection Unit Control Insert Bubble

22. CML CML Datapath with Forwarding Hardware PCSrc ID/EX.RegisterRt ID/EX.RegisterRs EX/MEM.RegisterRd MEM/WB.RegisterRd

23. CML CML Yoda says… Death is a natural part of life. Rejoice for those around you who transform into the Force. Mourn them do not. Miss them do not