1. Morgan Kaufmann Publishers The Processor
13 November, 2018
Chapter 4
The Processor
Chapter 4 — The Processor

2. Morgan Kaufmann Publishers
Multiple Exceptions
13 November, 2018
Pipelining overlaps multiple instructions
Could have multiple exceptions at once
Simple approach: deal with exception from earliest instruction
Flush subsequent instructions
“Precise” exceptions - always associating the proper exception with the correct instruction
Imprecise exceptions - Interrupts or exceptions in pipelined computers that are not associated with the exact instruction that was the cause of the interrupt or exception.
In complex pipelines
Multiple instructions issued per cycle
Out-of-order completion
Maintaining precise exceptions is difficult!
Chapter 4 — The Processor

3. Morgan Kaufmann Publishers
13 November, 2018
Imprecise Exceptions
Just stop pipeline and save state
Including exception cause(s)
Let the handler work out
Which instruction(s) had exceptions
Which to complete or flush
May require “manual” completion
Simplifies hardware, but more complex handler software
Not feasible for complex multiple-issue out-of-order pipelines
Chapter 4 — The Processor

4. Instruction-Level Parallelism (ILP)
Morgan Kaufmann Publishers
13 November, 2018
Instruction-Level Parallelism (ILP)
§4.10 Parallelism via Instructions
Pipelining: executing multiple instructions in parallel
To increase ILP
Deeper pipeline
Less work per stage  shorter clock cycle
Multiple issue
Replicate pipeline stages  multiple pipelines
Start multiple instructions per clock cycle
CPI < 1, so use Instructions Per Cycle (IPC)
E.g., 4GHz 4-way multiple-issue
16 BIPS, peak CPI = 0.25, peak IPC = 4
But dependencies reduce this in practice
Chapter 4 — The Processor

5. Morgan Kaufmann Publishers
13 November, 2018
Multiple Issue
Static multiple issue
Compiler groups instructions to be issued together
Packages them into “issue slots”
Compiler detects and avoids hazards
Dynamic multiple issue
CPU examines instruction stream and chooses instructions to issue each cycle
Compiler can help by reordering instructions
CPU resolves hazards using advanced techniques at runtime
Chapter 4 — The Processor

6. Morgan Kaufmann Publishers
Speculation
13 November, 2018
“Guess” what to do with an instruction
Start operation as soon as possible
Check whether guess was right
If so, complete the operation
If not, roll-back and do the right thing
Common to static and dynamic multiple issue
Examples
Speculate on branch outcome
instructions after the branch could be executed earlier
Roll back if path taken is different
Speculate on load
a store that precedes a load does not refer to the same address, which would allow the load to be executed before the store
Roll back if location is updated
Chapter 4 — The Processor

7. Compiler/Hardware Speculation
Morgan Kaufmann Publishers
13 November, 2018
Compiler/Hardware Speculation
Compiler can reorder instructions
e.g., move an instruction across a branch or a store across a load
insert additional instructions that check the accuracy of the speculation, and include “fix-up” instructions to recover from incorrect guess
Hardware can look ahead for instructions to execute
Buffer results until it determines they are actually needed
Flush buffers on incorrect speculation
Buffers written to registers or memory if speculation is correct
Chapter 4 — The Processor

8. Speculation and Exceptions
Morgan Kaufmann Publishers
13 November, 2018
Speculation and Exceptions
What if exception occurs on a speculatively executed instruction?
e.g., speculative load before null-pointer check, i.e. address it uses is not within bounds when the speculation is incorrect
Compiler will ignore such exceptions until they really should occur
Hardware buffers the exceptions until it is known the instruction causing it is no longer speculative
Static speculation
Can add ISA support for deferring exceptions
Dynamic speculation
Can buffer exceptions until instruction completion (which may not occur)
Chapter 4 — The Processor

9. Morgan Kaufmann Publishers
13 November, 2018
Static Multiple Issue
Compiler groups instructions into “issue packets”
Group of instructions that can be issued on a single cycle
Determined by pipeline resources required
Most static issue processors also rely on the compiler to take on some responsibility for handling data and control hazards.
The compiler’s responsibilities may include static branch prediction and code scheduling to reduce or prevent all hazards.
Think of an issue packet as a very long instruction
Specifies multiple concurrent operations
 Very Long Instruction Word (VLIW)
Chapter 4 — The Processor

10. Scheduling Static Multiple Issue
Morgan Kaufmann Publishers
13 November, 2018
Scheduling Static Multiple Issue
Compiler must remove some/all hazards
Reorder instructions into issue packets
No dependencies with a packet
Possibly some dependencies between packets
Varies between ISAs; compiler must know!
Pad with nop if necessary
Chapter 4 — The Processor

11. LEGv8 with Static Dual Issue
Morgan Kaufmann Publishers
LEGv8 with Static Dual Issue
13 November, 2018
Two-issue packets
One ALU/branch instruction
One load/store instruction
Instructions are paired and aligned on a 64-bit boundary
ALU/branch, then load/store
Pad an unused instruction with nop
Address
Instruction type
Pipeline Stages n
ALU/branch IF ID EX
MEM WB
n + 4
Load/store
n + 8
n + 12
n + 16
n + 20
Chapter 4 — The Processor

12. LEGv8 with Static Dual Issue
Morgan Kaufmann Publishers
LEGv8 with Static Dual Issue
13 November, 2018
Chapter 4 — The Processor

13. Hazards in the Dual-Issue LEGv8
Morgan Kaufmann Publishers
13 November, 2018
Hazards in the Dual-Issue LEGv8
More instructions executing in parallel
EX data hazard
Forwarding avoided stalls with single-issue
Now can’t use ALU result in load/store in same packet
ADD X0, X0, X1 LDUR X2, [X0,#0]
Split into two packets, effectively a stall
Load-use hazard
Still one cycle use latency, but now two instructions
More aggressive scheduling required
Chapter 4 — The Processor

14. Morgan Kaufmann Publishers
Scheduling Example
13 November, 2018
Schedule this for dual-issue LEGv8
Loop: LDUR X0, [X20,#0] // X0=array element ADD X0, X0,X // add scalar in X STUR X0, [X20,#0] // store result SUBI X20, X20,# // decrement pointer CMP X20, X // branch $s1!=0
BGT Loop
ALU/branch
Load/store
cycle
Loop:
nop
LDUR X0, [X20,#0] 1
SUBI X20, X20,#8 2
ADD X0, X0,X21 3
CMP X20, X22 4
BGT Loop
STUR X0, [X20,#0] 5
IPC = 6/5 = 1.2 (c.f. peak IPC = 2)
CPI = 0.83 Vs best case of 0.5
Chapter 4 — The Processor

15. Morgan Kaufmann Publishers
13 November, 2018
Loop Unrolling
Replicate loop body to expose more parallelism
Reduces loop-control overhead
Use different registers per replication
Called “register renaming”
Avoid loop-carried “anti-dependencies”
Load followed by a store of the same register
Aka “name dependence”
Reuse of a register name
Chapter 4 — The Processor

16. Loop Unrolling Example
Morgan Kaufmann Publishers
13 November, 2018
Loop Unrolling Example
ALU/branch
Load/store
cycle
Loop:
SUBI X20, X20,#32
LDUR X0, [X20,#0] 1
nop
LDUR X1, [X20,#24] 2
ADD X0, X0, X21
LDUR X2, [X20,#16] 3
ADD X1, X1, X21
LDUR X3, [X20,#8] 4
ADD X2, X2, X21
STUR X0, [X20,#32] 5
ADD X3, X3, X21
STUR X1, [X20,#24] 6
CMP X20,X22
STUR X2, [X20,#16] 7
BGT Loop
STUR X3, [X20,#8] 8
IPC = 15/8 = 1.875
Closer to 2, partly from reducing the loop control instructions and partly from dual issue execution
Cost of performance improvement is 4 temporary registers and more than double the code size
Chapter 4 — The Processor

17. Dynamic Multiple Issue
Morgan Kaufmann Publishers
Dynamic Multiple Issue
13 November, 2018
“Superscalar” processors
CPU decides whether to issue 0, 1, 2, … each cycle
Avoiding structural and data hazards
Avoids the need for compiler scheduling
Though it may still help to achieve good performance by moving the dependences apart and thereby increasing issue rate
Code semantics ensured by the CPU
Differences between superscalar and a VLIW processor
the code, whether scheduled or not, is guaranteed by the hardware to execute correctly
compiled code will always run correctly independent of the issue rate or pipeline structure of the processor
Not the case in some VLIW
Either recompilation required across different processor models, or
Performance poor although runs correctly
Chapter 4 — The Processor

18. Dynamic Pipeline Scheduling
Morgan Kaufmann Publishers
13 November, 2018
Dynamic Pipeline Scheduling
Allow the CPU to execute instructions out of order to avoid stalls
But commit result to registers in order
Example
LDUR X0, [X21,#20] ADD X1, X0, X2 SUB X23,X23,X3 ANDI X5, X23,#20
Can start sub while ADD is waiting for LDUR
Chapter 4 — The Processor

19. Dynamically Scheduled CPU
Morgan Kaufmann Publishers
Dynamically Scheduled CPU
13 November, 2018
Preserves dependencies
Hold pending operands and operations
Results also sent to any waiting reservation stations
Reorder buffer for register writes
Can supply operands for issued instructions
Chapter 4 — The Processor

20. Morgan Kaufmann Publishers
Register Renaming
13 November, 2018
Reservation stations and reorder buffer effectively provide register renaming
On instruction issue, it is copied to a reservation station
If operand is available in register file or reorder buffer
Copied to reservation station
No longer required in the register; can be overwritten
If operand is not in the register file or reorder buffer
It will be provided to the reservation station by a functional unit
Register update may not be required
Out-of-order execution
instructions can be executed in a different order than they were fetched
The processor executes the instructions in some order that preserves the data flow order of the program
Chapter 4 — The Processor

21. In-order commit
The instruction fetch and decode unit issues instructions in order
The commit unit writes the results to registers and memory in program fetch order
The functional units are free to initiate execution whenever the data they need are available
Today, all dynamically scheduled pipelines use in-order commit.

22. Morgan Kaufmann Publishers
13 November, 2018
Speculation
Predict branch and continue fetch and issue on the predicted path
Don’t commit until branch outcome determined
Load speculation
Avoid load and cache miss delay
Predict the effective address
Predict loaded value
Load before completing outstanding stores
Bypass stored values to load unit
Don’t commit load until speculation cleared
Chapter 4 — The Processor