1. Pipelining What is it? How does it work? What are the benefits? What could go wrong? By Derek Closson

2. What is pipelining? Pipelining is a design feature that allows individual common processor tasks to run simultaneously, such as: Fetch Decode Execute

3. What is pipelining? Or, more specifically in modern computers, Instruction read Decode Operand read Execute Operand write

4. What is pipelining? No more “one instruction at a time” processing Processor works simultaneously on multiple instructions Each instruction is at a different stage (or is being delayed until its next stage is open)

5. How does pipelining work? First instruction is fetched from memory

6. How does pipelining work? First instruction is fetched from memory First instruction is decoded; second instruction is fetched

7. How does pipelining work? First instruction is fetched from memory First instruction is decoded; second instruction is fetched First instruction’s operands are fetched; second instruction is decoded; third instruction is fetched

8. How does pipelining work? First instruction is fetched from memory First instruction is decoded; second instruction is fetched First instruction’s operands are fetched; second instruction is decoded; third instruction is fetched And on, and on, and on…

9. How does pipelining work? Because each instruction demands one stage of the processor, the maximum number of simultaneous instructions is the number of stages in the processor

10. How does pipelining work? A short animation of simple processor routines

48. How does pipelining work? A short(er) animation of pipelined processor routines

71. What are the benefits of pipelining? Uses most (if not all) of a processor’s ability at all times Efficiency goes up Relatively minor hardware changes give more completed cycles per second Speed goes up

72. What can go wrong in pipelines? Data hazards Control hazards Structural hazards

73. What can go wrong…? Data hazards Two instructions at different stages One instruction needs the results of the other instruction’s operation … add%r1,5, %r2 add%r2, 7, %r3 …

74. What can go wrong…? Data hazards Well-designed programs can anticipate this, and spread the connected commands out with other operations (or nop in a pinch) … add%r1, 5, %r2(i1 fetch) sub%r4, 7, %r8(i1 decode) mov%r5, %r6(i1 read) add%r2, 7, %r3 (i1 execute, i2 fetch) nop(i1 write, i2 decode) …

75. What can go wrong…? Data hazards Hardware can detect some data hazards and “forward” the information to the instructions that need it, even as the data goes to the registers Otherwise, programs may simply have bad data - testing and good programming are the best cures!

76. What can go wrong…? Control hazards Instructions are read sequentially from memory - even though loops may return to previous statements Instructions following the loop are already in the pipeline! Or, simple if/else blocks of code. Which block do you get instructions from?

77. What can go wrong…? Control hazards At worst, the entire pipeline must be flushed: hardware can stall the influx of new instructions until the conditional is evaluated

78. What can go wrong…? Control hazards Branch prediction inserts a couple instructions of its own to guess the outcome of the conditional If prediction is usually correct, fewer cycles lost on average If prediction is wrong, flush the pipeline and undo any improper commands

79. What can go wrong…? Structure hazards Two different instructions try to use the same hardware resource One instruction will obviously get access first, second instruction must be stalled (maybe along with all other instructions behind it) Again, careful programming will prevent this