U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Intelligent Compilation John Cavazos Computer & Information Sciences Department University of Delaware

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 2 The Optimization Problem

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 3 The Optimization Problem

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 4 The Optimization Problem

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 5 The Optimization Problem

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 6 The Optimization Problem

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 7 The Optimization Problem Intelligent Compilation

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 8 Architectural Characteristics Can use as machine learning features Example Mnemonic Description Avg Values FPU_IDL (Floating Unit Idle) VEC_INS (Vector Instructions) BR_INS (Branch Instructions) L1_ICH (L1 Icache Hits)

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 9 Domain Knowledge Generate training data from different domains Berkeley “motifs” can serve as starting point

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 10 Application of Machine Learning

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 11 Case Study: Method-Specific Compilation Use static code properties to characterize Java methods to control the optimizations applied

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 12 Method-Specific Compilation Details Intelligent Java JIT compiler Used simple code properties Extracted from a linear pass of code 26 features used to describe method Model controlled up to 20 optimizations Outperformed hand-tuned heuristics

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 13 Code Properties Used (inputs) Meaning Number of bytecodes Is syncronized, has exceptions, is leaf method Method Features Size Words allocated for locals space Locals Space Characteristics Declaration Fraction of Bytecodes Has array loads and stores primitive and long computations compares, branches, jsrs, switches, put, get, invoke, new, arraylength athrow, checkcast, monitor Is it declared final, static, private

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 14 Optimizations (outputs) Optimization Level Opt Level O0 Opt Level O1 Opt Level O2 Optimizations Controlled Branch Opts Low Constant Prop / Local CSE Reorder Code Copy Prop / Tail Recursion Static Splitting / Branch Opt Med Simple Opts Low While into Untils / Loop Unroll Branch Opt High / Redundant BR Simple Opts Med / Load Elim Expression Fold / Coalesce Global Copy Prop / Global CSE SSA

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 15 Generate Training Data For each method Evaluate many optimization settings Use fine-grained timers Record running time Record total time Levels 02 (20 optimizations) Evaluate 1000 random settings

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 16 Logistic Regression Variant of ordinary regression Inputs Continuous, discrete, or mix Outputs Restricted to between 0 and 1 Probability of optimization being beneficial

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 17 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor LR

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 18 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor LR

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 19 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor Feature Vector {108;25;0;0;0;0;1;0;0:2;0:0;0:0;0:0;0:0;0:0 0:12;0:0;0:08;0:0;0:0;0:0;0:2;0:32;0:08;0:0} LR

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 20 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor Feature Vector {108;25;0;0;0;0;1;0;0:2;0:0;0:0;0:0;0:0;0:0 0:12;0:0;0:08;0:0;0:0;0:0;0:2;0:32;0:08;0:0} LR

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 21 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor Feature Vector {108;25;0;0;0;0;1;0;0:2;0:0;0:0;0:0;0:0;0:0 0:12;0:0;0:08;0:0;0:0;0:0;0:2;0:32;0:08;0:0} {1;0;1;1;0;0;0;1;1;1;1;1;1;1;1;0;1;1;1;0} {1;0;1;1;0;0;0;1;1;1;1;1;1;1;1;0;1;1;1;0} Opt Flags LR

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 22 Experimental Results JikesRVM Highly-tuned Java JIT Compiler Benchmarks SPEC JVM 98 DaCapo benchmarks + 2 others Total Time (compilation + running time) Running Time (no compilation time)

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 23 SPEC JVM (Highest Opt Level) Total time improvements of 30%!

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 24 DaCapo (Highest Opt Level) Running time improvements over 50%!

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 25 % of O3 Applied to Hot Methods

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 26 % of O3 Applied to Hot Methods OPT compress jess raytrace db javac mpegaudio jack UNROLL Reorder BrchOpt

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 27 Single-Core Opts Still Important! Machine learning applied to auto-tuning Optimization Phase Ordering Speculative optimizations

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 28 Optimizations for Multi-Cores Auto-code partitioning and mapping Communication/computation overlap Task and Data placement/migration

U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 29 Conclusion Using machine learning successful Out-performs hand-tuning Simple code features can drive optimizations ML determines which features are important Optimizations applied only when beneficial