1. U NIVERSITY OF D ELAWARE C OMPUTER & I NFORMATION S CIENCES D EPARTMENT Intelligent Compilation John Cavazos Computer & Information Sciences Department University of Delaware

2. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 2 The Optimization Problem

3. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 3 The Optimization Problem

4. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 4 The Optimization Problem

5. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 5 The Optimization Problem

6. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 6 The Optimization Problem

7. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 7 The Optimization Problem Intelligent Compilation

8. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 8 Architectural Characteristics Can use as machine learning features Example Mnemonic Description Avg Values FPU_IDL (Floating Unit Idle) 0.473 VEC_INS (Vector Instructions) 0.017 BR_INS (Branch Instructions) 0.047 L1_ICH (L1 Icache Hits) 0.0006

9. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 9 Domain Knowledge Generate training data from different domains Berkeley “motifs” can serve as starting point

10. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 10 Application of Machine Learning

11. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 11 Case Study: Method-Specific Compilation Use static code properties to characterize Java methods to control the optimizations applied

12. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 12 Method-Specific Compilation Details Intelligent Java JIT compiler Used simple code properties Extracted from a linear pass of code 26 features used to describe method Model controlled up to 20 optimizations Outperformed hand-tuned heuristics

13. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 13 Code Properties Used (inputs) Meaning Number of bytecodes Is syncronized, has exceptions, is leaf method Method Features Size Words allocated for locals space Locals Space Characteristics Declaration Fraction of Bytecodes Has array loads and stores primitive and long computations compares, branches, jsrs, switches, put, get, invoke, new, arraylength athrow, checkcast, monitor Is it declared final, static, private

14. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 14 Optimizations (outputs) Optimization Level Opt Level O0 Opt Level O1 Opt Level O2 Optimizations Controlled Branch Opts Low Constant Prop / Local CSE Reorder Code Copy Prop / Tail Recursion Static Splitting / Branch Opt Med Simple Opts Low While into Untils / Loop Unroll Branch Opt High / Redundant BR Simple Opts Med / Load Elim Expression Fold / Coalesce Global Copy Prop / Global CSE SSA

15. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 15 Generate Training Data For each method Evaluate many optimization settings Use fine-grained timers Record running time Record total time Levels 02 (20 optimizations) Evaluate 1000 random settings

16. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 16 Logistic Regression Variant of ordinary regression Inputs Continuous, discrete, or mix Outputs Restricted to between 0 and 1 Probability of optimization being beneficial

17. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 17 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor LR

18. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 18 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor LR

19. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 19 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor Feature Vector {108;25;0;0;0;0;1;0;0:2;0:0;0:0;0:0;0:0;0:0 0:12;0:0;0:08;0:0;0:0;0:0;0:2;0:32;0:08;0:0} LR

20. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 20 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor Feature Vector {108;25;0;0;0;0;1;0;0:2;0:0;0:0;0:0;0:0;0:0 0:12;0:0;0:08;0:0;0:0;0:0;0:2;0:32;0:08;0:0} LR

21. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 21 Compiler Heuristic Tuned Method bytecodes Compiler Heuristic Optimizer Jikes RVM Optimized method Feature extractor Feature Vector {108;25;0;0;0;0;1;0;0:2;0:0;0:0;0:0;0:0;0:0 0:12;0:0;0:08;0:0;0:0;0:0;0:2;0:32;0:08;0:0} {1;0;1;1;0;0;0;1;1;1;1;1;1;1;1;0;1;1;1;0} {1;0;1;1;0;0;0;1;1;1;1;1;1;1;1;0;1;1;1;0} Opt Flags LR

22. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 22 Experimental Results JikesRVM Highly-tuned Java JIT Compiler Benchmarks SPEC JVM 98 DaCapo benchmarks + 2 others Total Time (compilation + running time) Running Time (no compilation time)

23. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 23 SPEC JVM (Highest Opt Level) Total time improvements of 30%!

24. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 24 DaCapo (Highest Opt Level) Running time improvements over 50%!

25. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 25 % of O3 Applied to Hot Methods

26. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 26 % of O3 Applied to Hot Methods OPT compress jess raytrace db javac mpegaudio jack UNROLL 50 0 5 0 0 0 14 Reorder 100 27 70 83 7 92 97 BrchOpt 100 78 88 100 96 98 91

27. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 27 Single-Core Opts Still Important! Machine learning applied to auto-tuning Optimization Phase Ordering Speculative optimizations

28. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 28 Optimizations for Multi-Cores Auto-code partitioning and mapping Communication/computation overlap Task and Data placement/migration

29. U NIVERSITY OF D ELAWARE Computer & Information Sciences Department 29 Conclusion Using machine learning successful Out-performs hand-tuning Simple code features can drive optimizations ML determines which features are important Optimizations applied only when beneficial