1. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Optimizing Compilers CISC 673 Spring 2009 Instruction Scheduling John Cavazos University of Delaware

2. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Instruction Scheduling Reordering instructions to improve performance Takes into account anticipated latencies Machine-specific Performed late in optimization pass Instruction-Level Parallelism (ILP)

3. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 3 Modern Architectures Features Superscalar Multiple logic units Multiple issue 2 or more instructions issued per cycle Speculative execution Branch predictors Speculative loads Deep pipelines

4. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 4 Types of Instruction Scheduling Local Scheduling Basic Block Scheduling Global Scheduling Trace Scheduling Superblock Scheduling Software Pipelining

5. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 5 Scheduling for different Computer Architectures Out-of-order Issue Scheduling is useful In-order issue Scheduling is very important VLIW Scheduling is essential!

6. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 6 Challenges to ILP Structural hazards: Insufficient resources to exploit parallelism Data hazards Instruction depends on result of previous instruction still in pipeline Control hazards Branches & jumps modify PC affect which instructions should be in pipeline

7. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Recall from Architecture… IF – Instruction Fetch ID – Instruction Decode EX – Execute MA – Memory access WB – Write back IF ID EX MA WB

8. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Structural Hazards IF ID EX MA WB addf R3,R1,R2 addf R3,R3,R4 stall EX Assumes floating point ops take 2 execute cycles Instruction latency: execute takes > 1 cycle

9. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Data Hazards IF ID EX MA WB lw R1,0(R2) add R3,R1,R4 stall Memory latency: data not ready

10. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Control Hazards IF ID --- EX --- MA --- WB IFIDEXMAWB IFIDEXMAWB Taken Branch Instr + 1 Branch Target Branch Target + 1

11. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 11 Basic Block Scheduling For each basic block: Construct directed acyclic graph (DAG) using dependences between statements Node = statement / instruction Edge (a,b) = statement a must execute before b Schedule instructions using the DAG

12. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Data Dependences If two operations access the same register and one access is a write, they are dependent Types of data dependences RAW=Read after WriteWAWWAR r1 = r2 + r3 r4 = r1 * 6 r1 = r2 + r3 r1 = r4 * 6 r1 = r2 + r3 r2 = r5 * 6 Cannot reorder two dependent instructions

13. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Basic Block Scheduling Example a) lw R2, (R1) b) lw R3, (R1) 4 c) R4  R2 + R3 d) R5  R2 - 1 ab dc 2 2 2 a) lw R2, (R1) b)lw R3, (R1) 4 --- nop ----- c) R4  R2 + R3 d) R5  R2 - 1 a) lw R2, (R1) b) lw R3, (R1) 4 d)R5  R2 - 1 c) R4  R2 + R3 Original Schedule Dependence DAG Schedule 1 (5 cycles) Schedule 2 (4 cycles)

14. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 14 Scheduling Algorithm Construct dependence dag on basic block Put roots in candidate set Use scheduling heuristics (in order) to select instruction While candidate set not empty Evaluate all candidates and select best one Delete scheduled instruction from candidate set Add newly-exposed candidates

15. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 15 Instruction Scheduling Heuristics NP-complete = we need heuristics Bias scheduler to prefer instructions: Earliest execution time Have many successors More flexibility in scheduling Progress along critical path Free registers Reduce register pressure Can be a combination of heuristics

16. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Computing Priorities Height(n) = exec(n) if n is a leaf max(height(m)) + exec(n) for m, where m is a successor of n Critical path(s) = path through the dependence DAG with longest latency

17. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 17 Example – Determine Height and CP Code alw r1, w badd r1,r1,r1 clw r2,x dmult r1,r1,r2 elw r2,y fmult r1,r1,r2 glw r2,z hmult r1,r1,r2 isw r1, a Assume: memory instrs = 3 mult = 2 = (to have result in register) rest = 1 cycle Critical path: _______ a b d f h i c e g 2 32 23 3 1 3

18. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 18 Example star t Schedule ___ cycles a b d f h i c e g 2 32 23 3 1 3 3 5 87 10 9 12 10 13 Code alw r1, w badd r1,r1,r1 clw r2,x dmult r1,r1,r2 elw r2,y fmult r1,r1,r2 glw r2,z hmult r1,r1,r2 isw r1, a

19. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Global Scheduling: Superblock Definition: single trace of contiguous, frequently executed blocks a single entry and multiple exits Formation algorithm: pick a trace of frequently executed basic block eliminate side entrance (tail duplication) Scheduling and optimization: speculate operations in the superblock apply optimization to scope defined by superblock

20. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Superblock Formation A 100 B 90 E 90 C 10 D0D0 F 100 A 100 B 90 E 90 C 10 D0D0 F 90 F’ 10 Select a trace Tail duplicate

21. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Optimizations within Superblock By limiting the scope of optimization to superblock: optimize for the frequent path may enable optimizations that are not feasible otherwise (CSE, loop invariant code motion,...) For example: CSE r1 = r2*3 r2 = r2 +1 r3 = r2*3 trace selection r1 = r2*3 r2 = r2 +1 r3 = r2*3 tail duplication r1 = r2*3 r2 = r2 +1 r3 = r1 r3 = r2*3 CSE within superblock (no merge since single entry)

22. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 22 Scheduling Algorithm Complexity Time complexity: O(n 2 ) n = max number of instructions in basic block Building dependence dag: worst-case O(n 2 ) Each instruction must be compared to every other instruction Scheduling then requires each instruction be inspected at each step = O(n 2 ) Average-case: small constant (e.g., 3)

23. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Very Long Instruction Word (VLIW) Compiler determines exactly what is issued every cycle (before the program is run) Schedules also account for latencies All hardware changes result in a compiler change Usually embedded systems (hence simple HW) Itanium is actually an EPIC-style machine (accounts for most parallelism, not latencies)

24. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Sample VLIW code c = a + bd = a - be = a * bld j = [x]nop g = c + dh = c - dnopld k = [y]nop i = j * cld f = [z]br g Add/Sub Mul/DivLd/StBranch VLIW processor: 5 issue 2 Add/Sub units (1 cycle) 1 Mul/Div unit (2 cycle, unpipelined) 1 LD/ST unit (2 cycle, pipelined) 1 Branch unit (no delay slots)

25. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT 25 Next Time Phase-ordering