1. CS152 / Kubiatowicz Lec13.1 10/17/01©UCB Fall 2001 CS152 Computer Architecture and Engineering Lecture 13 Introduction to Pipelining: Datapath and Control October 17 th, 2001 John Kubiatowicz (http.cs.berkeley.edu/~kubitron) lecture slides: http://www-inst.eecs.berkeley.edu/~cs152/

2. CS152 / Kubiatowicz Lec13.2 10/17/01©UCB Fall 2001 Recap: Sequential Laundry °Sequential laundry takes 6 hours for 4 loads °If they learned pipelining, how long would laundry take? ABCD 304020304020304020304020 6 PM 789 10 11 Midnight TaskOrderTaskOrder Time

3. CS152 / Kubiatowicz Lec13.3 10/17/01©UCB Fall 2001 Recap: Pipelining Lessons °Pipelining doesn’t help latency of single task, it helps throughput of entire workload °Pipeline rate limited by slowest pipeline stage °Multiple tasks operating simultaneously using different resources °Potential speedup = Number pipe stages °Unbalanced lengths of pipe stages reduces speedup °Time to “fill” pipeline and time to “drain” it reduces speedup °Stall for Dependences ABCD 6 PM 789 TaskOrderTaskOrder Time 3040 20

4. CS152 / Kubiatowicz Lec13.4 10/17/01©UCB Fall 2001 Recap: Ideal Pipelining IFDCDEXMEMWB IFDCDEXMEMWB IFDCDEXMEMWB IFDCDEXMEMWB IFDCDEXMEMWB Maximum Speedup  Number of stages Speedup  Time for unpipelined operation Time for longest stage Example: 40ns data path, 5 stages, Longest stage is 10 ns, Speedup  4 Assume instructions are completely independent!

5. CS152 / Kubiatowicz Lec13.5 10/17/01©UCB Fall 2001 °The Five Classic Components of a Computer °Today’s Topics: Recap last lecture/finish datapath Pipelined Control/ Do it yourself Pipelined Control Administrivia Hazards/Forwarding Exceptions Review MIPS R3000 pipeline The Big Picture: Where are We Now? Control Datapath Memory Processor Input Output

6. CS152 / Kubiatowicz Lec13.6 10/17/01©UCB Fall 2001 Can pipelining get us into trouble? °Yes: Pipeline Hazards structural hazards: attempt to use the same resource two different ways at the same time -E.g., combined washer/dryer would be a structural hazard or folder busy doing something else (watching TV) data hazards: attempt to use item before it is ready -E.g., one sock of pair in dryer and one in washer; can’t fold until get sock from washer through dryer -instruction depends on result of prior instruction still in the pipeline control hazards: attempt to make a decision before condition is evaulated -E.g., washing football uniforms and need to get proper detergent level; need to see after dryer before next load in -branch instructions °Can always resolve hazards by waiting pipeline control must detect the hazard take action (or delay action) to resolve hazards

7. CS152 / Kubiatowicz Lec13.7 10/17/01©UCB Fall 2001 Mem Single Memory is a Structural Hazard I n s t r. O r d e r Time (clock cycles) Load Instr 1 Instr 2 Instr 3 Instr 4 ALU Mem Reg MemReg ALU Mem Reg MemReg ALU Mem Reg MemReg ALU Reg MemReg ALU Mem Reg MemReg Detection is easy in this case! (right half highlight means read, left half write)

8. CS152 / Kubiatowicz Lec13.8 10/17/01©UCB Fall 2001 Structural Hazards limit performance °Example: if 1.3 memory accesses per instruction and only one memory access per cycle then average CPI  1.3 otherwise resource is more than 100% utilized

9. CS152 / Kubiatowicz Lec13.9 10/17/01©UCB Fall 2001 °Stall: wait until decision is clear °Impact: 2 lost cycles (i.e. 3 clock cycles per branch instruction) => slow °Move decision to end of decode save 1 cycle per branch Control Hazard Solution #1: Stall I n s t r. O r d e r Time (clock cycles) Add Beq Load ALU Mem Reg MemReg ALU Mem Reg MemReg ALU Reg MemReg Mem Lost potential

10. CS152 / Kubiatowicz Lec13.10 10/17/01©UCB Fall 2001 °Predict: guess one direction then back up if wrong °Impact: 0 lost cycles per branch instruction if right, 1 if wrong (right ­ 50% of time) Need to “Squash” and restart following instruction if wrong Produce CPI on branch of (1 *.5 + 2 *.5) = 1.5 Total CPI might then be: 1.5 *.2 + 1 *.8 = 1.1 (20% branch) °More dynamic scheme: history of 1 branch (­ 90%) Control Hazard Solution #2: Predict I n s t r. O r d e r Time (clock cycles) Add Beq Load ALU Mem Reg MemReg ALU Mem Reg MemReg Mem ALU Reg MemReg

11. CS152 / Kubiatowicz Lec13.11 10/17/01©UCB Fall 2001 °Delayed Branch: Redefine branch behavior (takes place after next instruction) °Impact: 0 clock cycles per branch instruction if can find instruction to put in “slot” (­ 50% of time) °As launch more instruction per clock cycle, less useful Control Hazard Solution #3: Delayed Branch I n s t r. O r d e r Time (clock cycles) Add Beq Misc ALU Mem Reg MemReg ALU Mem Reg MemReg Mem ALU Reg MemReg Load Mem ALU Reg MemReg

12. CS152 / Kubiatowicz Lec13.12 10/17/01©UCB Fall 2001 Data Hazard on r1: Read after write hazard (RAW) add r1,r2,r3 sub r4,r1,r3 and r6,r1,r7 or r8,r1,r9 xor r10,r1,r11

13. CS152 / Kubiatowicz Lec13.13 10/17/01©UCB Fall 2001 Dependencies backwards in time are hazards I n s t r. O r d e r Time (clock cycles) add r1,r2,r3 sub r4,r1,r3 and r6,r1,r7 or r8,r1,r9 xor r10,r1,r11 IFIF ID/R F EXEX ME M WBWB ALU Im Reg Dm Reg ALU Im Reg DmReg ALU Im Reg DmReg Im ALU Reg DmReg ALU Im Reg DmReg Data Hazard on r1: Read after write hazard (RAW)

14. CS152 / Kubiatowicz Lec13.14 10/17/01©UCB Fall 2001 “Forward” result from one stage to another “or” OK if define read/write properly Data Hazard Solution: Forwarding I n s t r. O r d e r Time (clock cycles) add r1,r2,r3 sub r4,r1,r3 and r6,r1,r7 or r8,r1,r9 xor r10,r1,r11 IFIF ID/R F EXEX ME M WBWB ALU Im Reg Dm Reg ALU Im Reg DmReg ALU Im Reg DmReg Im ALU Reg DmReg ALU Im Reg DmReg

15. CS152 / Kubiatowicz Lec13.15 10/17/01©UCB Fall 2001 Reg Dependencies backwards in time are hazards Can’t solve with forwarding: Must delay/stall instruction dependent on loads Forwarding (or Bypassing): What about Loads? Time (clock cycles) lw r1,0(r2) sub r4,r1,r3 IFIF ID/R F EXEX ME M WBWB ALU Im Reg Dm ALU Im Reg DmReg

16. CS152 / Kubiatowicz Lec13.16 10/17/01©UCB Fall 2001 Reg Dependencies backwards in time are hazards Can’t solve with forwarding: Must delay/stall instruction dependent on loads Forwarding (or Bypassing): What about Loads Time (clock cycles) lw r1,0(r2) sub r4,r1,r3 IFIF ID/R F EXEX ME M WBWB ALU Im Reg Dm ALU Im Reg DmReg Stall

17. CS152 / Kubiatowicz Lec13.17 10/17/01©UCB Fall 2001 Designing a Pipelined Processor °Go back and examine your datapath and control diagram °associated resources with states °ensure that flows do not conflict, or figure out how to resolve conflicts °assert control in appropriate stage

18. CS152 / Kubiatowicz Lec13.18 10/17/01©UCB Fall 2001 Control and Datapath: Split state diag into 5 pieces IR <- Mem[PC]; PC <– PC+4; A <- R[rs]; B<– R[rt] S <– A + B; R[rd] <– S; S <– A + SX; M <– Mem[S] R[rd] <– M; S <– A or ZX; R[rt] <– S; S <– A + SX; Mem[S] <- B If Cond PC < PC+SX; Exec Reg. File Mem Acces s Data Mem ABS Reg File Equal PC Next PC IR Inst. Mem DM

19. CS152 / Kubiatowicz Lec13.19 10/17/01©UCB Fall 2001 Pipelined Processor (almost) for slides °What happens if we start a new instruction every cycle? Exec Reg. File Mem Acces s Data Mem A B S M Reg File Equal PC Next PC IR Inst. Mem Valid IRex Dcd Ctrl IRmem Ex Ctrl IRwb Mem Ctrl WB Ctrl D

20. CS152 / Kubiatowicz Lec13.20 10/17/01©UCB Fall 2001 Pipelined Datapath (as in book); hard to read

21. CS152 / Kubiatowicz Lec13.21 10/17/01©UCB Fall 2001 Administrivia °Test was way too hard! Don’t go away! It is graded on a curve (only 15% of grade as well) Average: 53.8, Standard Dev: 14.4 Back in office hours tomorrow. °Monday: Sections in Cory lab (119) Since we are going to be testing things that you may not have tried, think again carefully about testing your design! Perhaps read the paper by Doug Clark on handouts page Everyone should go to section Get started on LAB 5! Out later today Problem 0 due tomorrow night at 12 Midnight via email -Evaluate your teammates. Organization on Lab due Thursday by Midnight via email

22. CS152 / Kubiatowicz Lec13.22 10/17/01©UCB Fall 2001 Computers in the News °Transmeta in trouble! Fired CEO yesterday Having trouble getting adoption of its new chips. °IBM Autonomic Computing Initiative "Civilization advances by extending the number of important operations which we can perform without thinking about them." - Alfred North Whitehead Within the past two decades the development of raw computing power coupled with the proliferation of computer devices has grown at exponential rates. This phenomenal growth along with the advent of the Internet have led to a new age of accessibility - to other people, other systems, and most importantly, to information. This boom has also led to unprecedented levels of complexity. Autonomic computing  self managing/healing systems -Flexible. The system will be able to sift data via a platform- and device-agnostic approach. -Accessible. The nature of the system is that it is always on. -Transparent. The system will perform its tasks and adapt to a user's needs without dragging the user into the intricacies of its workings.

23. CS152 / Kubiatowicz Lec13.23 10/17/01©UCB Fall 2001 Summary: Pipelining °What makes it easy all instructions are the same length just a few instruction formats memory operands appear only in loads and stores °What makes it hard? HAZARDS! structural hazards: suppose we had only one memory control hazards: need to worry about branch instructions data hazards: an instruction depends on a previous instruction