1. Intro to Computer Org. Pipelining, Part 2 – Data hazards + Stalls

2. Core Idea of Pipelining If we can’t make the “useful work” faster… what if we could do more of it at the same time?

3. Core Idea of Pipelining In technical terms, if you cannot improve the latency, try improving throughput. Latency – time required to complete a single task. Throughput – amount of tasks which can be handled at once.

4. Pipelined Instructions Blue =being actively used.

5. Pipelined Datapath

7. Impact On Performance If we examine the impact this will have on performance, under ideal circumstances… Each instruction now takes 5 cycles. We can run 5 instructions in parallel. CPI = 1.

8. Impact On Performance If we examine the impact this will have on performance, under ideal circumstances… Why might we not have ideal circumstances? Can we always run 5 instructions in parallel? No!

9. Pipelining – Hazards Hazard ­A situation where the next instruction cannot execute in the following cycle for a pipelined datapath There are three different types of hazards that we will examine.

10. Pipelining – Hazards Structural Hazard ­A hazard resulting from particular sets of instructions that we wish to execute simultaneously ­Happens when we try to use the same structural unit simultaneously for more than one task

11. Pipelining – Hazards Structural Hazard This is why we must have two different memory units in our datapath – one for instruction fetches and the other for data access.

12. Pipelining – Hazards Structural Hazard A similar problem could happen with the register file. (Used in IF and WB.) However, we noted that the register file is so fast that WB can happen in a different part of the cycle than IF. This fact is why the register file isn’t involved in a structural hazard.

13. Pipelining – Hazards Data Hazard ­A hazard caused by dependencies in data across instructions being pipelined Such a dependency requires that we force an instruction to wait for the data to be available for use, wasting a cycle of pipelining.

14. Pipelining – Hazards Consider the following instruction sequence: add $s0, $t0, $t1 sub $t2, $s0, $t3

15. Pipelining – Hazards Consider the following instruction sequence: add $s0, $t0, $t1 sub $t2, $s0, $t3 The value of $s0 won’t be available until written back!

16. add $s0, $t0, $t1 Pipelining – Hazards As a result, we must stall the sub instruction until the value of $s0 is ready. add $s0, $t0, $t1 sub $t2, $s0, $t3

17. add $s0, $t0, $t1 Pipelining – Hazards As a result, we must stall the sub instruction until the value of $s0 is ready. Really? sub $t2, $s0, $t3 Disclaimer: We’ll see how stalls are actually handled on a later slide in this lecture.

18. Pipelining – Hazards If we really think about it, isn’t the value of $s0 already calculated by the time we need it? We just haven’t written it back yet! add $s0, $t0, $t1 sub $t2, $s0, $t3

19. Overcoming Data Hazards Much of the time, it is possible to remove data hazards! The technique: forwarding. ­Adding extra hardware to retrieve data from early from already evaluated sources in the pipeline. ­Also called bypassing.

20. Pipelining – Forwarding Since the value of $s0 is already calculated, let’s forward it! add $s0, $t0, $t1 sub $t2, $s0, $t3

21. Pipelining – Forwarding Since the value of $s0 is already calculated, let’s forward it! Remember, we only need the value once we reach the EXE stage. add $s0, $t0, $t1 sub $t2, $s0, $t3

22. Pipelining – Forwarding In this case, we can still keep our throughput at its maximum. This isn’t always possible, though… add $s0, $t0, $t1 sub $t2, $s0, $t3

23. Pipelining – Forwarding Consider the case below. lw $s0, 20($t1) sub $t2, $s0, $t3 Can we avoid stalling an instruction? lw $s0, 20($t1) sub $t2, $s0, $t3

24. Pipelining – Forwarding When will the value of $s0 be available? When do we need the value of $s0? We have to stall! lw $s0, 20($t1) sub $t2, $s0, $t3

25. Pipelining – Forwarding When will the value of $s0 be available? When do we need the value of $s0? We have to stall! lw $s0, 20($t1) Stalled (nop) sub $t2, $s0, $t3

26. Pipelining – Forwarding When will the value of $s0 be available? When do we need the value of $s0? We have to stall! lw $s0, 20($t1) sub $t2, $s0, $t3 Stalled (nop) That is how stalls are actually implemented.

27. Pipelining – Forwarding Really? With forwarding, yes. Other techniques exist. lw $s0, 20($t1) sub $t2, $s0, $t3 Stalled (nop)

28. Overcoming Data Hazards If you must have a data hazard between two instructions, try to squeeze useful work inside. The technique: code reordering. ­Changing the order of instructions so that useful work may be done in what would otherwise be a stall.

29. Pipelining – Code Reordering Consider the case below. lw $s0, 20($t1) addi $t1, $t1, 4 sub $t2, $s0, $t3 lw $s0, 20($t1) sub $t2, $s0, $t3 addi $t1, $t1, 4

30. Pipelining – Code Reordering Do we still need to stall? Check for when data will be available and when it will be needed. lw $s0, 20($t1) sub $t2, $s0, $t3 addi $t1, $t1, 4

31. Pipelining – Code Reordering No stall required! The second instruction may execute because $t1 isn’t modified by the previous instruction. lw $s0, 20($t1) sub $t2, $s0, $t3 addi $t1, $t1, 4

32. Overcoming Data Hazards For the purposes of this class, code reordering must be done by compilers. Knowing how the pipelining system works allows us to optimize for these scenarios!

33. Overcoming Data Hazards In production CPUs, the processor itself can perform code reordering. This is referred to as out-of-order execution.

34. Overcoming Data Hazards The next question – how do we implement forwarding and stalls in hardware?

35. Overcoming Data Hazards 1. Note the type of operation at each stage. 2. Save the register fields used by each instruction – especially the destination register. 3. If a source register in one stage matches the destination register in another, forward the data!

36. Overcoming Data Hazards

38. Wait! Weren’t there supposed to be three different types of hazards?