1. Computer Organization and Design Pipelining
Montek Singh
Wed, Dec 5, 2012
Lecture 18

2. Pipelining Read Chapter 4.5-4.8
Between 411 problems sets, I haven’t had a minute to do laundry
Now that’s what I call dirty laundry
Read Chapter

3. Laundry Example INPUT: dirty laundry Device: Washer
Function: Fill, Agitate, Spin
WasherPD = 30 mins
OUTPUT:
4 more weeks
Device: Dryer
Function: Heat, Spin
DryerPD = 60 mins

4. Laundry: One Load at a Time
Everyone knows that the real reason that UNC students put off doing laundry so long is not because they procrastinate, are lazy, or even have better things to do.
The fact is, doing laundry one load at a time is not smart.
Step 1:
Step 2:
Total = WasherPD + DryerPD
= _________ mins 90

5. Total = N*(WasherPD + DryerPD)
Laundry: Doing N Loads!
Here’s how they do laundry at Duke, the “unpipelined” way.
Step 1:
Step 2:
Step 3:
Step 4: …
Total = N*(WasherPD + DryerPD)
= ____________ mins
N*90

6. Laundry: Doing N Loads! … UNC students “pipeline” the laundry process.
That’s why we wait!
Step 1:
Step 2:
Step 3: …
Actually, it’s more like N* if we account for the startup time (i.e., filling up the pipeline) correctly. When doing pipeline analysis, we’re mostly interested in the “steady state” where we assume we have an infinite supply of inputs.
Total = N * Max(WasherPD, DryerPD)
= ____________ mins
N*60

7. Recall Our Performance Measures
Latency:
Delay from input to corresponding output
Duke Laundry = _________ mins
UNC Laundry = _________ mins
Throughput:
Rate at which inputs or outputs are processed
Duke Laundry = _________ outputs/min
UNC Laundry = _________ outputs/min 90
Assuming that the wash is started as soon as possible and waits (wet) in the washer until dryer is available.
120
1/90
1/60
Even though
we increase
latency, it takes less time per load

8. Okay, Back to Circuits… F G H X P(X) X F(X) G(X) P(X)
For combinational logic: latency = tPD, throughput = 1/tPD. We can’t get the answer faster, but are we making effective use of our hardware at all times? X
F(X)
G(X)
P(X)
F & G are “idle”, just holding their outputs stable while H performs its computation

9. Pipelined Circuits use registers to hold H’s input stable! F G H X
Now F & G can be working on input Xi+1 while H is performing its computation on Xi. We’ve created a 2-stage pipeline : if we have a valid input X during clock cycle j, P(X) is valid during clock j+2. F G H X
P(X) 15 20 25
Suppose F, G, H have propagation delays of 15, 20, 25 ns and we are using ideal zero-delay registers (ts = 0, tpd = 0):
unpipelined
2-stage pipeline
latency 45
______
throughput
1/45
______
Pipelining uses registers to improve the throughput of combinational circuits 50
worse
1/25
better

10. Pipeline Diagrams Clock cycle Pipeline stagesF G H X
P(X) 15 20 25
This is an example of parallelism. At
any instant we are
computing 2 results.
Clock cycle i
i+1
i+2
i+3
Input Xi
Xi+1
F(Xi)
G(Xi)
Xi+2
F(Xi+1)
G(Xi+1)
H(Xi)
Xi+3
F(Xi+2)
G(Xi+2)
H(Xi+1)
H(Xi+2) …
F Reg
Pipeline stages
G Reg
H Reg
The results associated with a particular set of input data moves diagonally through the diagram, progressing through one pipeline stage each clock cycle.

11. Pipelining Summary Advantages: Disadvantages:
Higher throughput than combinational system
Different parts of the logic work on different parts of the problem…
Disadvantages:
Generally, increases latency
Only as good as the *weakest* link (often called the pipeline’s BOTTLENECK)

12. Review of CPU Performance
MIPS = Millions of Instructions/Second
MIPS =
Freq
CPI
Freq = Clock Frequency, MHz
CPI = Clocks per Instruction
To Increase MIPS:
1. DECREASE CPI.
- RISC simplicity reduces CPI to 1.0.
- CPI below 1.0? State-of-the-art multiple instruction issue
2. INCREASE Freq.
- Freq limited by delay along longest combinational path; hence
- PIPELINING is the key to improving performance.

13. Where Are the Bottlenecks?
Pipelining goal:
Break LONG combinational paths
 memories, ALU in separate stages WA PC +4
Instruction
Memory A D
Register
File
RA1
RA2
RD1
RD2
ALU B
ALUFN
Control Logic
Data Memory RD WD
R/W
Adr Wr
WDSEL
BSEL
J:<25:0>
PCSEL
WERF 00
PC+4
Rt: <20:16>
Imm: <15:0>
ASEL
SEXT + x4 BT Z
WASEL
Rd:<15:11>
Rt:<20:16> 1 2 3
PC<31:29>:J<25:0>:00 JT N V C
Rs: <25:21>
shamt:<10:6> 4 5 6
“16”
IRQ 0x 0x 0x
RESET
“31”
“27” WE

14. Goal: 5-Stage Pipeline IF ID/RF ALU MEM WB
GOAL: Maintain (nearly) 1.0 CPI, but increase clock speed to barely include slowest components (mems, regfile, ALU)
APPROACH: structure processor as 5-stage pipeline: IF
Instruction Fetch stage: Maintains PC, fetches one instruction per cycle and passes it to
ID/RF
Instruction Decode/Register File stage: Decode control lines and select source operands
ALU
ALU stage: Performs specified operation, passes result to …
MEM
Memory stage: If it’s a lw, use ALU result as an address, pass mem data (or ALU result if not lw) to … WB
Write-Back stage: writes result back into register file.

15. PC<31:29>:J<25:0>:00
5-Stage miniMIPS 0x 0x
PC<31:29>:J<25:0>:00 0x JT BT
PCSEL 6 5 4 3 2 1
• Omits some details PC 00
Instruction
Memory A D
Instruction +4
Fetch
PCREG 00
IRREG
Rt: <20:16>
Rs: <25:21>
J:<25:0>
RA1
Register
RA2 WA
File
RD1
RD2 JT =
Imm: <15:0>
SEXT
SEXT BZ x4
shamt:<10:6> +
“16”
Register 1 2
ASEL 1
BSEL
File BT
PCALU 00
IRALU A B
WDALU
Address is available right after instruction enters Memory stage A B
ALU
ALUFN
ALU N V C Z
PCMEM 00
IRMEM
YMEM
WDMEM Wr
Adr WD
R/W
Memory
PC+4
PCWB 00
IRWB
YWB
Data Memory RD
Rt:<20:16>
Rd:<15:11>
“31”
“27”
Data is needed just before rising clock edge at end of Write Back stage
WASEL
WDSEL
Write WA
Register WD
Back
WERF WE WA
File

16. Pipelining Improve performance by increasing instruction throughput
 Ideal speedup is number of stages in the pipeline. Do we achieve this?

17. Pipelining What makes it easy What makes it hard? Net effect:
all instructions are the same length
just a few instruction formats
memory operands appear only in loads and stores
What makes it hard?
structural hazards: suppose we had only one memory
control hazards: need to worry about branch instructions
data hazards: an instruction depends on a previous instruction
Net effect:
Individual instructions still take the same number of cycles
But improved throughput by increasing the number of simultaneously executing instructions

18. Data Hazards
Problem with starting next instruction before first is finished
dependencies that “go backward in time” are data hazards

19. Software Solution Have compiler guarantee no hazards
Where do we insert the “nops” ?
Between “producing” and “consuming” instructions! sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)
Problem: this really slows us down!

20. Forwarding
Bypass/forward results as soon as they are produced/needed. Don’t wait for them to be written back into registers!

21. Can't always forward Load word can still cause a hazard:
an instruction tries to read a register following a load instruction that writes to the same register. STALL!

22. Stalling
When needed, stall the pipeline by keeping an instruction in the same stage fpr an extra clock cycle.

23. Branch Hazards When branching, other instructions are in the pipeline!
need to add hardware for flushing instructions if we are wrong

24. Pipeline Summary
A very common technique to improve throughput of any circuit
used in all modern processors!
Fallacies:
“Pipelining is easy.” No, smart people get it wrong all of the time!
“Pipelining is independent of ISA.” No, many ISA decisions impact how easy/costly it is to implement pipelining (i.e. branch semantics, addressing modes).
“Increasing pipeline stages improves performance.” No, returns diminish because of increasing complexity.

25. … to improve parallelism?
What else can we do?
… to improve parallelism?

26. Multicore/multiprocessor
Use more than one processor = multiprocessor
called multicore when they are all on the same chip
read all about it in Chapter 7 of textbook
FIGURE 7.2 Classic organization of a shared memory multiprocessor. Copyright © 2009 Elsevier, Inc.

27. So, what did we learn this semester?
That’s it folks!
So, what did we learn this semester?

28. What we learnt this semester
You now have a pretty good idea about how computers are designed and how they work:
How data and instructions are represented
How arithmetic and logic operations are performed
How ALU and control circuits are implemented
How registers and the memory hierarchy are implemented
How performance is measured
(Self Study) How performance is increased via pipelining
Lots of low-level programming experience:
C and MIPS
This is how programs are actually executed!
This is how OS/networking code is actually written!
Java and other higher-level languages are convenient high-level abstractions. You probably have new appreciation for them!

29. Grades? We are frantically trying to wrap up all grading!
Your final grades will be on ConnectCarolina by the end of this week.
Also, don’t forget to submit your course evaluation!