Chapter 4 The Processor Part 2 박능수

Performance Issues Longest delay determines clock period Critical path: load instruction Instruction memory  register file  ALU  data memory  register file Not feasible to vary period for different instructions Violates design principle Making the common case fast We will improve performance by pipelining

Overview of Pipeline Pipelining provides a method for executing multiple instructions at the same time. Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes “Folder” takes 20 minutes A B C D

Sequential Laundry 6 PM 7 8 9 10 11 Midnight 30 40 20 30 40 20 30 40 Time 30 40 20 30 40 20 30 40 20 30 40 20 T a s k O r d e A B C D Sequential laundry takes 6 hours for 4 loads If they learned pipelining, how long would laundry take?

Pipelined Laundry: Start work ASAP 6 PM 7 8 9 10 11 Midnight Time 30 40 20 T a s k O r d e A B C D Pipelined laundry takes 3.5 hours for 4 loads

Pipelining Lessons 6 PM 7 8 9 30 40 20 A B C D Pipelining doesn’t help latency of single task, it helps throughput of entire workload Pipeline rate limited by slowest pipeline stage Multiple tasks operating simultaneously using different resources Potential speedup = Number pipe stages Unbalanced lengths of pipe stages reduces speedup Time to “fill” pipeline and time to “drain” it reduces speedup Stall for Dependences 6 PM 7 8 9 Time 30 40 20 T a s k O r d e A B C D

Pipelining Lessons Five stages, one step per stage IF: Instruction fetch from memory ID: Instruction decode & register read EX: Execute operation or calculate address MEM: Access memory operand WB: Write result back to register

One Way to Break up the Datapath Operations Ifetch: Instruction Fetch Fetch the instruction from the Instruction Memory ID(ID/Reg): Instruction Decode and Registers Fetch Exec: Calculate the memory address Mem: Read the data from the Data Memory Wr: Write the data back to the register file Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Load Ifetch ID/Reg Exec Mem Wr

Pipelined Execution On a processor multiple instructions are in various stages at the same time. Assume each instruction takes five cycles Time IFetch ID Exec Mem WB IFetch ID Exec Mem WB IFetch ID Exec Mem WB IFetch ID Exec Mem WB IFetch ID Exec Mem WB Program Flow IFetch ID Exec Mem WB

Single Cycle, Multiple Cycle, vs. Pipeline Clk Single Cycle Implementation: Load Store Cycle 1 Cycle 2 Waste Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Clk Multiple Cycle Implementation: Ifetch ID Exec Mem Wr Load Store R-type Load Ifetch ID Exec Mem Wr Pipeline Implementation: Store R-type

Pipeline Performance Assume time for stages is 100ps for register read or write 200ps for other stages Compare pipelined datapath with single-cycle datapath Instr Instr fetch Register read ALU op Memory access Register write Total time lw 200ps 100 ps 800ps sw 700ps R-format 600ps beq 500ps

Ideal speedup is number of stages in the pipeline. Do we achieve this? Pipelining Improve performance by increasing instruction throughput Ideal speedup is number of stages in the pipeline. Do we achieve this?

If all stages are balanced i.e., all take the same time 𝑇𝑖𝑚𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑖𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛𝑠 𝑝𝑖𝑝𝑒𝑙𝑖𝑛𝑒𝑑 = 𝑇𝑖𝑚𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑖𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛𝑠 𝑛𝑜𝑛𝑝𝑖𝑝𝑒𝑙𝑖𝑛𝑒𝑑 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑡𝑎𝑔𝑒𝑠 If not balanced, speedup is less Speedup due to increased throughput Latency (time for each instruction) does not decrease

Why Pipeline? Suppose Single Cycle Machine Multicycle Machine 100 instructions are executed The single cycle machine has a cycle time of 45 ns The multicycle and pipeline machines have cycle times of 10 ns The multicycle machine has a CPI of 3.6 Single Cycle Machine 45 ns/cycle x 1 CPI x 100 inst = 4500 ns Multicycle Machine 10 ns/cycle x 3.6 CPI x 100 inst = 3600 ns Ideal pipelined machine 10 ns/cycle x (1 CPI x 100 inst + 4 cycle drain) = 1040 ns Ideal pipelined vs. single cycle speedup 4500 ns / 1040 ns = 4.33 What has not yet been considered?

Pipelining and ISA Design MIPS ISA designed for pipelining All instructions are 32-bits Easier to fetch and decode in one cycle c.f. x86: 1- to 17-byte instructions Few and regular instruction formats Can decode and read registers in one step Load/store addressing Can calculate address in 3rd stage, access memory in 4th stage Alignment of memory operands Memory access takes only one cycle

MIPS Pipelined Datapath MEM Right-to-left flow leads to hazards WB

Pipeline registers Need registers between stages To hold information produced in previous cycle

Morgan Kaufmann Publishers 10 November, 2018 Pipeline Operation Cycle-by-cycle flow of instructions through the pipelined datapath “Single-clock-cycle” pipeline diagram Shows pipeline usage in a single cycle Highlight resources used c.f. “multi-clock-cycle” diagram Graph of operation over time We’ll look at “single-clock-cycle” diagrams for load & store Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 IF for Load, Store, … Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 ID for Load, Store, … Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 EX for Load Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 MEM for Load Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 WB for Load Wrong register number Chapter 4 — The Processor

Corrected Datapath for Load Morgan Kaufmann Publishers 10 November, 2018 Corrected Datapath for Load Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 EX for Store Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 MEM for Store Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 WB for Store Chapter 4 — The Processor

Single-Cycle Pipeline Diagram Morgan Kaufmann Publishers 10 November, 2018 Single-Cycle Pipeline Diagram State of pipeline in a given cycle Chapter 4 — The Processor

Multi-Cycle Pipeline Diagram Morgan Kaufmann Publishers 10 November, 2018 Multi-Cycle Pipeline Diagram Form showing resource usage Chapter 4 — The Processor

Multi-Cycle Pipeline Diagram Morgan Kaufmann Publishers 10 November, 2018 Multi-Cycle Pipeline Diagram Traditional form Chapter 4 — The Processor

Graphically Representing Pipelines Can help with answering questions like: how many cycles does it take to execute this code? what is the ALU doing during cycle 4? use this representation to help understand datapaths

Why Pipeline? For Performance! Time (clock cycles) Once the pipeline is full, one instruction is completed every cycle, so CPI = 1 ALU IM Reg DM Inst 0 I n s t r. O r d e ALU IM Reg DM Inst 1 ALU IM Reg DM Inst 2 ALU IM Reg DM Inst 3 ALU IM Reg DM Inst 4 Time to fill the pipeline

MIPS Pipeline Datapath Additions/Mods State registers between each pipeline stage to isolate them IF:IFetch ID:Dec EX:Execute MEM: MemAccess WB: WriteBack Read Address Instruction Memory Add PC 4 Write Data Read Addr 1 Read Addr 2 Write Addr Register File Data 1 Data 2 16 32 ALU Shift left 2 Data IF/ID Sign Extend ID/EX EX/MEM MEM/WB Any info that is needed in a later pipe stage must be passed to that stage via a pipeline register (e.g., need to preserve the destination register address in the pipeline state registers) Note two exceptions to right-to-left flow WB that writes the result back into the register file in the middle of the datapath Selection of the next value of the PC, one input comes from the calculated branch address from the MEM stage Only later instructions in the pipeline can be influenced by these two REVERSE data movements. The first one (WB to ID) leads to data hazards. The second one (MEM to IF) leads to control hazards. All instructions must update some state in the processor – the register file, the memory, or the PC – so separate pipeline registers are redundant to the state that is updated (not needed). PC can be thought of as a pipeline register: the one that feeds the IF stage of the pipeline. Unlike all of the other pipeline registers, the PC is part of the visible architecture state – its content must be saved when an exception occurs (the contents of the other pipe registers are discarded). System Clock

Pipelined Control (Simplified) Morgan Kaufmann Publishers 10 November, 2018 Pipelined Control (Simplified) Chapter 4 — The Processor

Pipeline control We have 5 stages. What needs to be controlled in each stage? Instruction Fetch and PC Increment Instruction Decode / Register Fetch Execution Memory Stage Write Back How would control be handled in an automobile plant? a fancy control center telling everyone what to do? should we use a finite state machine?

Morgan Kaufmann Publishers 10 November, 2018 Pipelined Control Control signals derived from instruction Pass control signals along just like the data implementation Chapter 4 — The Processor

Morgan Kaufmann Publishers 10 November, 2018 Pipelined Control Chapter 4 — The Processor

MIPS Pipeline Data and Control Paths 1 PCSrc ID/EX EX/MEM Control IF/ID Add MEM/WB Branch Add 4 RegWrite Shift left 2 Instruction Memory Read Addr 1 Data Memory Register File Read Data 1 Read Addr 2 MemtoReg Read Address ALUSrc PC Read Data Address 1 Write Addr ALU Read Data 2 Write Data Write Data 1 ALU cntrl How many bits wide is each pipeline register? PC = 32 bits IF/ID = 64 bits ID/EX = 9 + 32x4 + 10 = 147 EX/MEM = 5 + 1 + 32x3 + 5 = 107 MEM/WB = 2 + 32x2 + 5 = 71 MemWrite MemRead Sign Extend 16 32 ALUOp 1 RegDst