1. Run Time Optimization 15-745: Optimizing Compilers Pedro Artigas

2. 2 Motivation A good reason Compiling a language that contains run- time constructs Java dynamic class loading Perl or Matlab eval(“statement”) Faster than interpreting A better reason May use program information only available at run time

3. 3 Example of run-time information The processor that will be used to run the program inc ax is faster on a Pentium III add ax,1 is faster on a Pentium 4 No need to recompile if generating code at run time The actual program input/run-time behavior Is my profile information accurate for the current program input? YES!

4. 4 The life cycle of a program CompileLink Load/Run One Object File Global Analysis One Binary Whole Program Analysis One Process Analysis? No observation! Larger scope, better information about program behavior

5. 5 New strategies are possible Pessimistic x Optimistic approaches Ex: Does int *a points to the same location as int *b ? Compile time/Pessimistic: Prove that in ANY execution those pointers point to different addresses Run Time/Optimistic: Up to now in the current execution a and b point to different locations Assume this holds If the assumption breaks, invalidate generated code and generate new code

6. 6 A sanity check Using run time information does not require run time code generation Example: Versioning ISA may allow cheaper tests IA-64 Transmeta if (a!=b) { } else { }

7. 7 Drawbacks Code generation has to be FAST Rule of thumb: almost linear on program size Code quality: Compromise on quality to achieve fast code generation shoot for good, not great Also this usually means: No time for classical Iterative Data Flow Analysis at run time

8. 8 No classical IDFA: Solutions Quasi-Static and/or Staged Compilation Perform IDFA at compile time Specialize the dynamic code generator for the obtained information That is, encode the obtained data flow information in the “binary” Do not rely on classical IDFA Use algorithms that do not require it Ex: Dominator based value numbering (coming up!) Generate code in a style that does not require it Ex: One entry multiple exits traces as in deco and dynamo

9. 9 Code generation Strategies Compiling a language that requires run-time code generation: Compile adaptively: Use a very simple and fast code generation scheme Re-compile frequently used regions using more advanced techniques

10. 10 Adaptive Compilation: Motivation Very simple code generation Higher execution cost Elaborate code generation Higher compilation cost Problem: We may not know in advance how frequently a region will execute Measure frequencies and re-compile dynamically Fast compiler Optimizing compiler Cost threshold

11. 11 Code generation Strategies Compiling selected regions that benefit from run-time code generation: Pick only the regions that should benefit the most Which regions? Select them statically Use profile information Re-compile (that is select then dynamically) Usually all of the above

12. 12 Code Optimization Unit What is the run-time unit of optimization? Option: Procedures/static code regions Similar to static compilers Option: Traces Start at the target of a backward branch Include all the instructions in a path May include procedure calls and returns Branches Fall through = remain in the trace Target = exit the trace 1 23 4 1 2 4 3 4

13. 13 Current strategies Static regionTrace JIT compilersJava JITs Matlab JITs ? Run-time performance engines Dyc Fabius Dynamo Deco

14. 14 Run-Time code generation: Case studies Two examples of algorithms that are suitable for run-time code generation Run time CSE/PRE replacement: Dominator based value numbering Run time Register Allocation: Linear scan register allocation

15. 15 Sidebar With traces CSE/PRE become almost trivial No need for register allocation if optimizing a binary (ex: dynamo) PRECSE A+B

16. 16 Review: Local value numbering Store expressions already computed (in a hash table) Store variable name  VN mapping in the VN array Store VN  variable name mapping in the Name array Same value number  same value for each basic block Table.empty() for each computed expression (“x=y op z”) if V=Table.lookup(“y op z”) VN[“x”]=V if VN[Name[V]]==V //expression is still there replace “x = y op z” with “x = Name[V]” else Name[V]=“x” else VN[“x”]=new_value_number() Table.insert(“y op z”,VN[“x”]) Name[VN[“x”]]=“x” Expression was computed in the past, check if result is available New expression, add to the table

17. 17 Local value numbering Works in linear time on program size Assuming accesses to the array and the hash table occur in constant time Can we make it work in a scope larger than a basic block? (Hint: Yes) What are the potential problems?

18. 18 Problems How to propagate the hash table contents across basic blocks? How to make sure that is safe to access the location containing the expression in other basic blocks? How do we make sure if the location containing the expression is fresh? Remember: no IDFA

19. 19 Control flow issues On split points things are simple Just keep the content of the hash table from the predecessor What about merge points? We do not know if the same expression was computed in all incoming paths We do not want to check the fact anyway (why?) Reset the state of the hash table to a safe state it had in the past Which program point in the past? The immediate dominator of the merge block

20. 20 Data flow issues Making sure the def of an expression is fresh and reaches the blocks of interest How? By construction! SSA All names are fresh (Single Assignment) All defs dominate its’ uses (regular uses not  functions) As, by construction, we introduce new defs using  functions at every point this would not hold

21. 21 Dominator/SSA based value numbering DVN(Block B) Table.PushScope() for each exp “n=  (…)” if (exp is redundant or meaningless) //meaningless:  (x 0,x 0 ) VN[“n”]= Table.lookup(“  (…)” or “x 0 ”) remove(“n=  (…)”) else VN[“n”]=“n” Table.insert(“  (…)”,VN[n]) for each exp “x=y op z” if (“v”=Table.lookup(“y op z”)) VN[“x”]=“v” remove(“x=y op z”) else VN[“x”]=“x” Table.insert(“x=y op z”,VN[“x”]) for each successor s of B Adjust the  inputs for each dominator tree child c in CFG reverse post-order DVN(c) Table.PopScope() First process the  expressions Them the regular ones Propagate info about  inputs and call DVN recursively

22. 22 Example NameVN u0u0 v0v0 w0w0 x0x0 y0y0 u1u1 x1x1 y1y1 u2u2 x2x2 y2y2 u3u3 u 0 =a 0 +b 0 v 0 =c 0 +d 0 w 0 =e 0 +f 0 x 0 =c 0 +d 0 y 0 =c 0 +d 0 u 1 =a 0 +b 0 x 1 =e 0 +f 0 y 1 =e 0 +f 0 u 2 =  (u 0,u 1 ) x 2 =  (x 0,x 1 ) y 2 =  (y 0,y 1 ) u 3 =a 0 +b 0 1 2 3 4

23. 23 Does not catch But it performs almost as well as CSE And runs much faster linear time ? (YES? NO?) Problems x 1 =a 0 +b 0 x 0 =a 0 +b 0 x 2 =  (x 0,x 1 ) x 0 =a 0 +b 0 x 1 =  (x 0,x 2 ) x 2 =a 0 +b 0

24. 24 Homework #4 The DVN algorithm scans the CFG in a similar way as the second phase of SSA translation SSA translation phase #1 Placing  functions SSA translation phase #2 assigning unique numbers to variables Combine both and save one pass Gives us a smaller constant But, at run time, it pays of!

25. 25 Run time register allocation Graph Coloring? Not an option Even the simple stack based heuristic shown in class is O(n 2 ) Not even counting: Building the graph Move coalescing optimization But register allocation is VERY important in terms of performance Remember, memory is REALLY slow We need a simple but effective (almost) linear time algorithm

26. 26 Let’s start simple Start with a local (basic block) linear time algorithm Assuming only one def and one use per variable (More constrained than SSA) Assuming that if a variable is spilled it must remain spilled (Why?) Can we find an optimum linear time algorithm? (Hint: Yes) Ideas? Think about liveness first …

27. 27 Simple Algorithm: Computing Liveness One def and one use per variable, only one block A live range is merely the interval between the def and the use Live Interval: Interval between the first def and the last use OBS: Live Range = Live Interval if there is no control flow, only one def and use We could compute live intervals using a linear scan if we store the def instructions (beginning of the interval) in a hash table

28. 28 Example S1: A=1 S2: B=2 S3: C=3 S4: D=A S5: E=B S6: use(E) S7: use(D) S8: use(C)

29. 29 Now Register Allocation Another linear scan Keep the active intervals in an list ( active ) Assumption: an interval, when spilled, will remain spilled Two scenarios #1: No problem #2: Must spill Which interval?

30. 30 Spilling heuristic Since there is no second chance: That is a spilled variable will always remain spilled Spill the interval that ends last Intuition: As one spill must occur … Pick the one that makes the remaining allocation least constrained That is, the interval that ends last This is the provably optimum solution (given all the constraints)

31. 31 Linear Scan Register Allocation active = {} freeregs = {all_registers} for each interval I (in order of increasing start point) for each interval J in active if J.end>I.start continue active.remove(J) freeregs.insert(J.register) end for each interval J if active.length()==R spill_candidade=active.last(); if (spill_candidate.end>I.end) I.register = spill_candidate.register spill(spill_candidate) active.remove(spill_candidate) active.insert_sorted(I) //sorted by end point else spill(I) else I.register = freeregs.pop() //get any register from the free list active.insert_sorted(I) //sorted by end point end for each interval I Expire old intervals Must spill, pick either the last interval in active or the new interval No constraints

32. 32 Example (R=2) S1: A=1 S2: B=2 S3: C=3 S4: D=A S5: E=B S6: use(E) S7: use(D) S8: use(C) A B C D E ABCDE S1 S2 S3 S4 S5 S6 S7 S8

33. 33 Is the second pass really linear? Invariant: active.length()<=R Complexity O(R*n) R is usually a small constant (128 at most) Therefore: O(n)

34. 34 And we are done! Right? YES and NO Use the same algorithm as before for register assignment Program representation: Linear list of instructions Live intervals are not precise anymore given control flow and multiple def/uses Not optimum, but still FAST Code quality: within 10% of graph coloring for spec95 benchmarks (One problem with this claim)

35. 35 The Worst problem: Obtaining precise live intervals How to obtain precise live interval information FAST? Claim of 10% relies on live interval information obtained using liveness analysis (IDFA) IDFA is SLOW, O(n 3 ) Most recent solutions: Use the local interval algorithm for variables that only live inside one basic block Use liveness analysis for more global variables Alleviates the problem, does not fully solve it

36. 36 More problems: Live intervals may not be precise OBS: The idea of lifetime holes leads to allocators that also try to use this holes to assign the same register to other live ranges (bin-packing) Such an allocator is used in the Alpha family of compilers (GEM compilers)

37. 37 Other problems: Linearization order Register allocation quality depends on chosen block linearization order Choose a good order in practice layout order depth first traversal of the CFG Both only 10% slower than graph coloring

38. 38 Graph coloring versus Linear scan Compilation cost scaling

39. 39 Conclusion Run time code generation provides new optimization opportunities Challenges Identify new optimization opportunities Design new compilation strategies example: optimistic versus conservative Design algorithms and implementations that: minimize run time overhead Do not compromise much on code quality Recent examples indicate: extending fast local methods is a promising way to obtain fast run-time code generation