1. Online Subpath Profiling
Yossi Matias
David Oren
Mooly Sagiv
School of Computer Science
Tel-Aviv University
מבוסס על מסטר, עבודה עם המנחים
יופיע ב-Compiler construction
עותקים בסוף השבוע למי שרוצה

2. Motivation for Profiling
Feedback on dynamic program behavior
The rule
Can be used by:
Computer Architects
Compiler Writers
Programmers
Better program performance
אופטימיזציות בחומרה
להוסיף תמונה – משהו מורכב עם FLOW שרץ בפנים

3. Types of Profiling Vertex profiling Edge profiling Path profiling
No context, just count of instructions
Edge profiling
Branch-transition
Profile-directed optimization
Path profiling
Multiple branch-transition
Intra- or inter-procedural
Edge profiling:
Reorganizing basic blocks
Avoiding/Predicting branches
Path profiling:
Depends on the definition of path
Where can they start?
תמונות
סדרה של קשתות

4. Types of Profiling Offline Online
Results are collected and then displayed
User in the loop
Online
Results are collected and acted upon
JIT compilation
Display to user
Edge profiling:
Reorganizing basic blocks
Avoiding/Predicting branches
Path profiling:
Depends on the definition of path
Where can they start?
תמונות
סדרה של קשתות

5. Motivation for Subpath Profiling
Programs may have hot subpaths
which are part of cold paths
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
doMore ()
למה VERTEX לא מספיק טוב?
להוסיף שקף שמתאר את doCommon

6. Challenges Large number of subpaths
>4M distinct subpaths of length 2,4,...,64k in JLex
>35M total subpaths
Counting all subpaths is prohibitively expensive
Memory
Time
non linear

7. Online Subpath Profiler
Based on an adaptive sampling technique
Identifies arbitrary hot subpaths
Low memory overhead
Low runtime overhead
Online
Appropriate for JIT-like compilers
Can be adapted to different requirements

8. Outline Algorithm overview Adaptive sampling Issues The OSP algorithm
Reference implementation
Experimental results
Related work
Conclusion

9. Algorithm Overview Select on-the-fly a random sample of subpaths
Count the popularity of sampled subpaths and obtain estimation by scaling
Achieve high accuracy using limited memory

10. Adaptive Sampling
Based on a hot-list algorithm by Gibbons and Matias (SIGMOD 1998)
Sample elements from the input set
Frequently occurring elements will be sampled more often
Sampling probability determined at runtime, according to the allowed memory usage
Tradeoff between overhead and accuracy
Give an estimate of the sample’s accuracy

11. Concise Samples Uniform random sampling
Maintain an <id, count> pair for each element
The sample size can be much larger than the memory size
For skewed input sets the gain is much larger
Sampling is not applied at every block
Vitter’s reservoir sampling

12. Concise Samples

13. Issues Encoding Path length bias Path representation
Generating a unique ID for paths
Path length bias
Longer or shorter paths?
Path representation

14. The OSP Algorithm void enterBlock (BasicBlock b) { if (sampling)
sampleBlock (b);
else if (--skip == 0) {
length = choosePathLength ();
sampling = true; }
void sampleBlock (BasicBlock b) {
subpath.appendBlock (b);
if (--length == 0) {
updateHotList (subpath.id);
skip = chooseSkipValue ();
subpath = new subPath ();
sampling = false; }
Sampled path
Basic blocks skipped

15. OSP Algorithm Walkthrough
skip = 5
sampling = false
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
void enterBlock (BasicBlock b) {
if (sampling)
sampleBlock (b);
else if (--skip == 0) {
length = choosePathLength ();
sampling = true; }
להסביר מה הסתברות הדגימה, ואיך זה משפיע על R
Skipping
Sampling

16. OSP Algorithm Walkthrough
skip = 4
sampling = false
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

17. OSP Algorithm Walkthrough
skip = 3
sampling = false
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

18. OSP Algorithm Walkthrough
skip = 2
sampling = false
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

19. OSP Algorithm Walkthrough
skip = 1
sampling = false
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

20. OSP Algorithm Walkthrough
skip = 0
length = 2
sampling = true
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
void enterBlock (BasicBlock b) {
if (sampling)
sampleBlock (b);
else if (--skip == 0) {
length = choosePathLength ();
sampling = true; }
Skipping
Sampling

21. OSP Algorithm Walkthrough
skip = 0
length = 1
sampling = true
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

22. OSP Algorithm Walkthrough
skip = 4
length = 0
sampling = false
doA-doCommon: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
void sampleBlock (BasicBlock b) {
subpath.appendBlock (b);
if (--length == 0) {
updateHotList (subpath.id);
skip = chooseSkipValue ();
subpath = new subPath ();
sampling = false; }
Skipping
Sampling

23. OSP Algorithm Walkthrough
skip = 3
sampling = false
doA-doCommon: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

24. OSP Algorithm Walkthrough
skip = 2
sampling = false
doA-doCommon: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

25. OSP Algorithm Walkthrough
skip = 1
sampling = false
doA-doCommon: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

26. OSP Algorithm Walkthrough
skip = 0
length = 2
sampling = true
doA-doCommon: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

27. OSP Algorithm Walkthrough
skip = 0
length = 1
sampling = true
doA-doCommon: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

28. OSP Algorithm Walkthrough
skip = 8
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

29. OSP Algorithm Walkthrough
skip = 7
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

30. OSP Algorithm Walkthrough
skip = 6
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

31. OSP Algorithm Walkthrough
skip = 5
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

32. OSP Algorithm Walkthrough
skip = 4
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

33. OSP Algorithm Walkthrough
skip = 3
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

34. OSP Algorithm Walkthrough
skip = 2
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

35. OSP Algorithm Walkthrough
skip = 1
sampling = false
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

36. OSP Algorithm Walkthrough
skip = 0
length = 2
sampling = true
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

37. OSP Algorithm Walkthrough
skip = 0
length = 1
sampling = true
doA-doCommon: 1
doCommon-if2: 1
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

38. OSP Algorithm Walkthrough
skip = 6
sampling = false
doA-doCommon: 1
doCommon-if2: 2
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()
Skipping
Sampling

39. After 1000 Iterations doCommon-if2: 253 If1-doA: 130 If2-doD: 127
if1-doB: 122
if2-doC: 118
if1-doA-..-if2: 65 …
if (condition1)
doB ()
doA ()
doCommon ()
if (condition2)
doD ()
doC ()

40. Prototype Implementation
Written in Java, using the Soot Framework
Handles full Java
Low memory overhead (~50kB)
Low sampling overhead (5%-50%)
Sampling + Skipping overhead (current implementation): 30%-360%
High accuracy on tested benchmarks
אבל כרגע אנחנו עובדים בכל JVM
להגיד SOOT מגיע מ-MCGILL
JAVAGRANDE
מאיפה זה בא

41. Prototype Implementation
Limited to paths of length 2n
Favorable tradeoff
Simple encoding
Tested for practical performance
Gives more weight to shorter paths
Only implementation details!

42. Results – Runtime Overhead
Program
Full Overhead
Sampling Overhead
JLex
93%
47%
FFT
36%
16%
HeapSort
204%
56%
MolDyn
241%
32%
RayTrace
361%
41%
javac
79% 7%

43. Results – Memory Overhead
Program
Program memory
Profiler memory
JLex
169,728
43,304
FFT
107,416
39,742
HeapSort
107,400
48,960
MolDyn
111,800
40,864
RayTrace
108,106
65,800

44. Results – Accuracy (FFT)
Rank in sample
Accurate rank
Count error 1
0.94% 2
0.11% 3
1.00% 4
0.76% 5 6
29.27% 11
3.04% 7 12

45. Results – Incremental (FFT)
True Rank 6%
12%
18%
24%
30%
36% 1 2 6 3 4 8 5 7

46. Related Work Ball, Larus: Efficient path profiling (MICRO 1996)
Larus: Whole program paths (PLDI 1999)
Melski, Reps: Interprocedural path profiling (CC 1999)
Taub, Schechter, Smith: Ephemeral instrumentation for lightweight program profiling (2000)
Sastry, Bodik, Smith: Rapid profiling via stratified sampling (Computer Architecture 2001)
Bala, Duesterwald, Banerjia: Dynamo: a transparent dynamic optimization system (PLDI 2001)

47. Related Work Ball-Larus path profiler (MICRO 1996) and extensions
Only Acyclic paths
Whole Program Path (Larus, PLDI 1999)
Uses an alphabet representing acyclic paths
Compact image of a whole program trace
Not online BL
Give each path a unique integer identifier

48. Related Work Dynamo (PLDI 2000) A dynamic compiler for native code
Locates “hot traces” and optimizes them
Limits places where hot traces may start
It would be interesting to integrate OSP into Dynamo

49. Limitations Results are only an approximation
Other methods are approximations as well
Guaranteed confidence and accuracy as function of hotness
Context not taken into account
Robust, works for arbitrary subpaths
Stand alone tool
Integrate into existing tools

50. Conclusions We have presented a framework for online subpath profiling
We have a reference implementation
Simple
Efficient
Accurate