1. CMPE 421 Parallel Computer Architecture
Part 1
Pipeline: HAZARD

2. Pipelining MIPS
Lets us examine why the pipeline can not run at full speed
There are some cases, though, where the next instruction can not begin executing immediately
This limits to pipeline are known as hazards
What makes it hard?
structural hazards: different instructions, at different stages, in the pipeline want to use the same hardware resource (resource conflict)
control hazards:
succeeding instruction, to put into pipeline, depends on the outcome of a previous branch instruction, already in pipeline
Control decision determines execution path, such as when the instruction changes the PC
data hazards: an instruction in the pipeline requires data to be computed by a previous instruction still in the pipeline
Before actually building the pipelined datapath and control we first briefly examine these potential hazards individually…

3. Structural Hazards
Structural hazard: inadequate hardware to simultaneously support all instructions in the pipeline in the same clock cycle
E.g., suppose single – not separate – instruction and data memory in pipeline below with one read port
then a structural hazard between first and fourth lw instructions
MIPS was designed to be pipelined: structural hazards are easy to avoid! 2 4 6 8 1 I n s t r u c i o f e h R g A L U D a T m l w $ , ( ) 3 P x d
Pipelined
Structural Hazards
Hazard if single memory

4. Structural Hazard Ex 1: Suppose we have one memory unit instead of separate instruction and data memory
Inst
Fetch
Reg
Read
ALU
Data
Access
Reg Write
When a load or store word instruction is used the MEM stage tries to access the memory and because of single data memory a conflict occurs

5. Structural Hazard Consider a load followed immediately by a store
Processor only has a single write port
Clock
Cycle 1
Cycle 2
Cycle 3
Cycle 4
Cycle 5
Cycle 6
Cycle 7
Cycle 8
Cycle 9 IF
RF/ID EX WB
R-type
WBB
MEMM
Load
bubble

6. Structural Hazard Solutions
Delay instruction until functional unit is ready
Hardware inserts a pipeline stall or a bubble that delays execution of all instructions that follow (previous instructions continue)
Increases CPI from the ideal value of 1
Build more sophisticated functional units so that all combinations of instructions can be accommodated
Example: Allow two simultaneous writes to the register file

7. Structural Hazard Solution
Write Back Stall Solution:
Delay R-type register write by one cycle IF
RF/ID EX WB
R-type
MEM 1 2 3 4
Clock
Cycle 1
Cycle 2
Cycle 3
Cycle 4
Cycle 5
Cycle 6
Cycle 7
Cycle 8
Cycle 9 IF
RF/ID
MEM WB
R-type EX
Load

8. Control Hazards
Control hazard: need to make a decision based on the result of a previous instruction still executing in pipeline
Solution 1 Stall the pipeline P r o g r a m e x e c u t i o n 2 4 6 8 1 1 2 1 4 1 6 o r d e r T i m e ( i n i n s t r u c t i o n s ) I n s t r u c t i o n D a t a
Note that branch outcome is
computed in ID stage with
added hardware (later…) a d d $ 4 , $ 5 , $ 6 R e g A L U R e g f e t c h a c c e s s I n s t r u c t i o n D a t a b e q $ 1 , $ 2 , 4 R e g A L U R e g 2 n s f e t c h a c c e s s I n s t r u c t i o n D a t a l w $ 3 , 3 ( $ ) b u l e R e g A L U R e g f e t c h a c c e s s 4 n s 2 n s
Pipeline stall

9. Control Hazards Solution 2 Predict branch outcome
e.g., predict branch-not-taken :
Prediction success
Prediction failure: undo (=flush) lw

10. Control Hazards
Solution 3 Delayed branch: always execute the sequentially next statement with the branch executing after one instruction delay – compiler’s job to find a statement that can be put in the slot that is independent of branch outcome
MIPS does this – but it is an option in SPIM (Simulator -> Settings) P r o g r a m e x e c u t i o n 2 4 6 8 1 1 2 1 4 o r d e r T i m e ( i n i n s t r u c t i o n s ) b e q $ 1 , $ 2 , 4 I n s t r u c t i o n D a t a R e g A L U R e g f e t c h a c c e s s a d d $ 4 , $ 5 , $ 6 I n s t r u c t i o n D a t a R e g A L U R e g ( d e l a y e d b r a n c h s l o t ) 2 n s f e t c h a c c e s s I n s t r u c t i o n D a t a l w $ 3 , 3 ( $ ) R e g A L U R e g 2 n s f e t c h a c c e s s 2 n s
Delayed branch beq is followed by add that is
independent of branch outcome

11. Review: Pipelining Multiple Instructions
The Instructions in Figures 6-19, 6-20 and 6-21 were independent
None of them used the results calculated by any of the others (register numbers are different)

12. Review: Pipelining Multiple Instructions

13. Review: Pipelining Multiple Instructions

14. Data Hazards
Problem with starting next instruction before first is finished
dependencies that “go backward in time” are data hazards

15. Solution to Data Hazards
Data hazard: instruction needs data from the result of a previous instruction still executing in pipeline
Occur when pipeline changes the order of read/write access to operands so that the order differs from the order seen by sequentially executing instructions
Solution1 Forward data if possible…
Solution 2 Or change the relative timing of instructions (insert stalls)
Instruction pipeline diagram:
shade indicates use –
left=write, right=read P r o g r a m e x e c u t i o n 2 4 6 8 1 o r d e r T i m e ( i n i n s t r u c t i o n s )
Without forwarding – blue line –
data has to go back in time;
with forwarding – red line –
data is available in time a d d $ s , $ t , $ t 1 I F I D E X M E M W B s u b $ t 2 , $ s , $ t 3 I F I D E X M M E E M M W B
Caused by several different types of dependencies

16. Data Hazards
SOLUTION 1
Don’t wait for the instruction to complete before trying to resolve the data hazard
As soon as ALU creates the sum for “add”, we can supply it as an input for the add
Adding extra H/W to retrieve the missing item early from the internal resources is called forwarding or bypassing
Invalid
Remark: Forwarding path from the output of the memory access stage in the first instruction to the input of the execution stage is invalid (backward in time)

17. Data Dependency Types
-Three classifications of data dependencies for instruction j following instruction I
Read after Write (RAW)
Instr. j tries to read before instr. i tries to write it
Write after Write (WAW)
Instr. j tries to write an operand before i writes its value
Since register writes only occur in WB, the pipeline we have been discussing does not have this type of dependency
Write after Read (WAR)
Instr. j tries to write a destination before it is read by i
This also does not occur in this pipeline we have been discussing since all reads happen early in the ID/RF stage and all writes are late in the WB stage
-WAW and WAR are in later more complicated pipes

18. Data Hazards
Forwarding may not be enough (Hybrid solution is required)
e.g., if an R-type instruction following a load uses the result of the load – called load-use data hazard 2 4 6 8 1 1 2 1 4 P r o g r a m T i m e e x e c u t i o n o r d e r ( i n i n s t r u c t i o n s )
Without a stall it is impossible
to provide input to the sub
instruction in time l w $ s , 2 ( $ t 1 ) I F I D E X M E M W B I F D W B M E X s u b $ t 2 , $ s , $ t 3
-With a one-stage stall (solution 2)
-Forwarding can get the data to the sub instruction in time (solution 1)

19. Reordering Code to Avoid Pipeline Stall (Alternative Software Solution)
Example:
lw $t0, 0($t1)
lw $t2, 4($t1)
sw $t2, 0($t1)
sw $t0, 4($t1)
Reordered code:
Data hazard
Interchanged

20. Revisiting Hazards
So far our datapath and control have ignored hazards
We shall revisit data hazards and control hazards and enhance our datapath and control to handle them in hardware…

21. Data Hazards and Forwarding
Problem with starting an instruction before previous are finished:
data dependencies that go backward in time – called data hazards
$2 = 10 before sub;
$2 = -20 after sub
sub $2, $1, $3
and $12, $2, $5
or $13, $6, $2
add $14, $2, $2
sw $15, 100($2)

22. Software Solution Have compiler guarantee never any data hazards! nop
by rearranging instructions to insert independent instructions between instructions that would otherwise have a data hazard between them,
or, if such rearrangement is not possible, insert nops
Such compiler solutions may not always be possible, and nops slow the machine down
sub $2, $1, $3
nop
nop and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)
sub $2, $1, $3
lw $10, 40($3)
slt $5, $6, $7 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2) or
MIPS: nop = “no operation” = 00…0 (32bits) = sll $0, $0, 0

23. REVIEW: Solution to HAZARDS

24. How About Register File Access?
Time (clock cycles)
Fix register file access hazard by doing reads in the second half of the cycle and writes in the first half
ALU IM
Reg DM
add $1, I n s t r. O r d e
ALU IM
Reg DM
Inst 1
ALU IM
Reg DM
Inst 2
Define register reads to occur in the second half of the cycle and register writes in the first half
ALU IM
Reg DM
add $2,$1,
For lecture
clock edge that controls loading of pipeline state registers
clock edge that controls register writing

25. Register Usage Can Cause Data Hazards
Dependencies backward in time cause hazards
ALU IM
Reg DM
add $1, I n s t r. O r d e
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
For class handout
ALU IM
Reg DM
xor $4,$1,$5
Read before write data hazard

26. Register Usage Can Cause Data Hazards
Dependencies backward in time cause hazards
ALU IM
Reg DM
add $1,
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
For lecture
ALU IM
Reg DM
xor $4,$1,$5
Read before write data hazard

27. Loads Can Cause Data Hazards
Dependencies backward in time cause hazards
ALU IM
Reg DM
lw $1,4($2) I n s t r. O r d e
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
ALU IM
Reg DM
xor $4,$1,$5
Load-use data hazard

28. One Way to “Fix” a Data Hazard
Can fix data hazard by waiting – stall – but impacts CPI
ALU IM
Reg DM
add $1, I n s t r. O r d e
stall
stall
sub $4,$1,$5
and $6,$1,$7
ALU IM
Reg DM

29. Another Way to “Fix” a Data Hazard
Fix data hazards by forwarding results as soon as they are available to where they are needed
ALU IM
Reg DM
add $1, I n s t r. O r d e
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
For class handout
ALU IM
Reg DM
xor $4,$1,$5

30. Another Way to “Fix” a Data Hazard
Fix data hazards by forwarding results as soon as they are available to where they are needed
ALU IM
Reg DM
add $1, I n s t r. O r d e
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
ALU IM
Reg DM
xor $4,$1,$5
Forwarding paths are valid only if the destination stage is later in time than the source stage.
Forwarding is harder if there are multiple results to forward per instruction or if they need to write a result early in the pipeline

31. Forwarding with Load-use Data Hazards
ALU IM
Reg DM
lw $1,4($2) I n s t r. O r d e
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
ALU IM
Reg DM
xor $4,$1,$5

32. Forwarding with Load-use Data Hazards
ALU IM
Reg DM
lw $1,4($2) I n s t r. O r d e
ALU IM
Reg DM
sub $4,$1,$5
ALU IM
Reg DM
and $6,$1,$7
ALU IM
Reg DM
or $8,$1,$9
ALU IM
Reg DM
xor $4,$1,$5
Will still need one stall cycle even with forwarding

33. Branch Instructions Cause Control Hazards
Dependencies backward in time cause hazards
beq
ALU IM
Reg DM I n s t r. O r d e
ALU IM
Reg DM lw
ALU IM
Reg DM
Inst 3
ALU IM
Reg DM
Inst 4

34. One Way to “Fix” a Control Hazard
Another “solution” is to put in enough extra hardware so that we can test registers, calculate the branch address, and update the PC during the second stage of the pipeline. That would reduce the number of stalls to only one.
A third approach is to prediction to handle branches, e.g., always predict that branches will be untaken. When right, the pipeline proceeds at full speed. When wrong, have to stall (and make sure nothing completes – changes machine state – that shouldn’t have).

35. One Way to “Fix” a Control Hazard
Fix branch hazard by waiting – stall – but affects CPI
ALU IM
Reg DM
beq I n s t r. O r d e
stall
stall
stall lw
ALU IM
Reg DM
Inst 3