U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Optimizing Compilers CISC 673 Spring 2009 Dynamic Compilation II John Cavazos University of Delaware

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT What is in a Dynamic Compiler? Interpretation Popular approach for high-level languages Ex, Python, APL, SNOBOL, BCPL, Perl, MATLAB Useful for memory-challenged environments Low startup time & space overhead, but much slower than native code execution MMI (Mixed Mode Interpreter) [Suganauma’01] Fast interpreter implemented in assembler

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT What is in a Dynamic Compiler? Quick compilation Reduced set of optimizations for fast compilation, little inlining Full compilation Full optimizations only for selected hot methods Classic just-in-time compilation Compile methods to native code on first invocation Ex, ParcPlace Smalltalk-80, Self-91 Initial high (time & space) overhead for each compilation Precludes use of sophisticated optimizations (eg. SSA) Responsible for many of today’s myths

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Interpretation vs JIT Execution: 20 time units Execution: 2000 time units

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Selective Optimization Hypothesis: most execution is spent in a small percentage of methods Idea: use two execution strategies 1. Interpreter or non-optimizing compiler 2. Full-fledged optimizing compiler Strategy: Use option 1 for initial execution of all methods Profile to find “hot” subset of methods Use option 2 on this subset

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Selective Optimization Selective opt: compiles 20% of methods, representing 99% of execution time Execution: 20 time units Execution: 2000 time units

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Designing an Adaptive Optimization System What is the system architecture? What are the profiling mechanisms and policies for driving recompilation? How effective are these systems?

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Basic Structure of a Dynamic Compiler Program Machine code Structural inlining unrolling loop perm Scalar cse constants expressions Memory scalar repl ptrs Reg. Alloc Scheduling peephole Still needs good core compiler - but more

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Raw Profile Data Instrumented code Basic Structure of a Dynamic Compiler Compiler subsystem Optimizations Interpreter or Simple Translation Program Executing Program Profile Processor History prior decisions compile time Controller Compilation decisions Processed Profile

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling Counters Call Stack Sampling Combinations

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling: Counters Insert method-specific counter on method entry and loop back edges Counts how often a method is called and approximates how much time is spent in a method Very popular approach: Self, HotSpot Issues: overhead for incrementing counter can be significant Not present in optimized code

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling: Counters foo ( … ) { fooCounter++; if (fooCounter > Threshold) { recompile( … ); }... }

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling: Call Stack Sampling Periodically record which method(s) are on call stack Approximates amount of time spent in each method Can be compiled into the code Jikes RVM, JRocket or use hardware sampling Issues: timer-based sampling is not deterministic

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling: Call Stack Sampling A B C A B A A B A B C A B C... Sample

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling Mixed Combinations Use counters initially and sampling later on IBM DK for Java foo ( … ) { fooCounter++; if (fooCounter > Threshold) { recompile( … ); }... } A B C

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling Mixed Software Hardware Combination Use interupts & sampling foo ( … ) { if (flag is set) { sample( … ); }... } A B C

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Recompilation Policies: Which Candidates to Optimize? Problem: given optimization candidates, which should be optimized? Counters: 1. Optimize method that surpasses threshold Simple, but hard to tune, doesn’t consider context 2. Optimize method on the call stack based on inlining policies Addresses context issue Call Stack Sampling: 1. Optimize all methods that are sampled − Simple, but doesn’t consider frequency of sampled methods 2. Use Cost/benefit model Seemingly complicated, but easy to engineer Maintenance free Naturally supports multiple optimization levels

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Jikes RVM: Recompilation Policy – Cost/Benefit Model Define cur, current opt level for method m Exe(j), expected future execution time at level j Comp(j), compilation cost at opt level j Choose j > cur that minimizes Exe(j) + Comp(j) If Exe(j) + Comp(j) < Exe(cur) recompile at level j Assumptions Sample data determines how long a method has executed Method will execute as much in the future as it has in the past Compilation cost and speedup are offline averages

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Startup Programs: Jikes RVM [Hind et al.’04] No FDO, Mar’04, AIX/PPC

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Startup Programs: Jikes RVM No FDO, Mar’04, AIX/PPC

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Steady State: Jikes RVM No FDO, Mar’04, AIX/PPC

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Steady State: Jikes RVM

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Feedback-Directed Optimization (FDO) Exploit information gathered at run-time to optimize execution “selective optimization”: what to optimize “FDO” : how to optimize

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Advantages of FDO Can exploit dynamic information that cannot be inferred statically System can change and revert decisions when conditions change Runtime binding allows more flexible systems

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Challenges for automatic online FDO Compensate for profiling overhead Compensate for runtime transformation overhead Account for partial profile available and changing conditions

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Profiling for What to Do Clients Inlining, unrolling, method dispatch Dispatch tables, synchronization services, GC Pretching Misses, Hardware performance monitors [Adl- Tabatabai et al.’04] Code layout values - loop counts edges & paths

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Profiling for What to Do Myth: Sophisticated profiling is too expensive to perform online Reality: Well-known technology can collect sophisticated profiles with sampling and minimal overhead

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Method Profiling Timer Based class Thread scheduler (...) {... flag = 1; } void handler(...) { // sample stack, perform GC, swap threads, etc..... flag = 0; } foo ( … ) { // on method entry, exit, & all loop backedges if (flag) { handler( … ); }... } A B C Useful for more than profiling Jikes RVM Schedule garbage collection Thread scheduling policies, etc. if (flag) handler(); if (flag) handler(); if (flag) handler();

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Arnold-Ryder [PLDI 01]: Full Duplication Profiling Generate two copies of a method Execute “fast path” most of the time Execute “slow path” with detailed profiling occassionally Adapted by J9 due to proven accuracy and low overhead

U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Suggested Reading Dynamic Compilation Adaptive optimization in the Jalapeno JVM, M. Arnold, S. Fink, D. Grove, M. Hind, and P. Sweeney, Proceedings of the 2000 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages & Applications (OOPSLA '00), pages , Oct