1. Automatically Characterizing Large Scale Program Behavior Timothy Sherwood Erez Perelman Greg Hamerly Brad Calder Used with permission of author

2. ASPLOS : Sherwood et. al.2 Characterizing Program Behavior Ideal: To understand the effects of cycle-level events on full program execution Challenge: To achieve this without doing complete detailed simulation How: Build a high-level model of program behavior that can be used in conjunction with limited detailed simulation

3. ASPLOS : Sherwood et. al.3 Goals The goals of this research are: –To create an automatic system that is capable of intelligently characterizing time- varying program behavior –To provide both analytic and software tools to help with program phase identification –To demonstrate the utility of these tools for finding places to simulate (SimPoints) – Without full program detailed simulation

4. ASPLOS : Sherwood et. al.4 Our Approach Programs are neither –Completely Homogenous –nor Totally Random Instead they are quite structured Discover this structure The key is the code that is executing –the code determines the program behavior

5. ASPLOS : Sherwood et. al.5 Large Scale Behavior (gzip) IPC Energy DL1 IL1 L2 bpred

6. ASPLOS : Sherwood et. al.6 Some Definitions Interval is –A set of instructions that execute one after the other in program order –100 Million Instructions Phase is –A set of intervals with very similar behavior –Regardless of temporal adjacency

7. ASPLOS : Sherwood et. al.7 Outline Examining the Programs Finding Phases Automatically Application to Efficient Simulation Conclusions

8. ASPLOS : Sherwood et. al.8 Fingerprinting Intervals Fingerprint each interval in program –Enabling us to build high level model Basic Block Vector [PACT’01] –Tracks the code that is executing –Long sparse vector –1 dimension per static basic block –Based on instruction execution frequency

9. ASPLOS : Sherwood et. al.9 Basic Block Vectors ID: 1 2 3 4 5. BB Exec Count: weigh by Block Size: = Normalize to 1 = For each interval: One BBV for each interval We can now compare vectors Start with simple manual analysis –Compare all N 2 pairs of intervals Enter the Similarity Matrix…

10. ASPLOS : Sherwood et. al.10 Distance Between 2 BBVs  a i -b i   i=1 D  a i -b i  i=1 D Euclidean Distance (a,b) = Manhattan Distance (a,b) =

11. ASPLOS : Sherwood et. al.11 Similarity Matrix Element (x,y) is distance between BBVs of 2 intervals Compare N 2 intervals Executed Instructions on Diagonal axis To compare 2 points go horizontal from one and vertically from the other Darker points indicate similar vectors Clearly shows the phase-behavior Gzip-graphic

12. ASPLOS : Sherwood et. al.12 Time Varying Behavior Gzip-graphic IPC and L1 data cache miss rate as a % of max value of that metric

13. ASPLOS : Sherwood et. al.13 A More Complex Matrix - gcc Still much structure Dark boxes show phase-behavior Boxes in interior show recurring phases Strong diagonal line indicates first half is similar to second half Manual inspection is not feasible or scalable

14. ASPLOS : Sherwood et. al.14 Outline Examining the Programs Finding Phases Automatically Application to Efficient Simulation Conclusions

15. ASPLOS : Sherwood et. al.15 Finding the Phases Basic Block Vector is a point in space The problem is to find groups of vectors/points that are all similar –Making sure that all points in a group are similar to one another –And ensuring all points that are different, are put into different groups This is a Clustering Problem A Phase is a Cluster of BBVectors

16. ASPLOS : Sherwood et. al.16 Phase-finding Algorithm I.Profile Program and track BB Vectors II.Use the K-means algorithm to find clusters in the data for many different values of K III.Score the likelihood of each clustering IV.Pick the best clustering

17. ASPLOS : Sherwood et. al.17 Improving Performance K-means requires many manipulations –Basic Block Vectors are very long > 100,000 for gcc; 800,000 for microsoft apps –Need to make the Vectors smaller Still preserve relative distances

18. ASPLOS : Sherwood et. al.18 Random Projection Multiply the vector by a random matrix Can safely reduce down to 15 dimensions Reduce run-time from days to minutes x 0,0 … x 0,nbb X int,0 … x int,nbb...... x M 0,0 …M 0,15 M nbb,0 …M nbb,15...... x 0,0 … x 0,15 X int,0 … x int,15...... = rand(-1, 1)

19. ASPLOS : Sherwood et. al.19 K-means and Scoring Initialize by randomly choosing k points as cluster centers Iterate until convergence (no membership changes): –For each data point, compare distance to each center and assign to “closest” cluster –For each cluster, make new center as centroid of its member points Do this for k = 1 to 10 Use BIC (Bayesian Information Criterion) to score each of the 10 resulting clusterings –Choose one that is at least 90% of spread in BICs –High BIC score predicted performance metric with low variance.

20. ASPLOS : Sherwood et. al.20 Example: gzip Revisited IPC Energy DL1 IL1 L2 bpred

21. ASPLOS : Sherwood et. al.21 gzip – Phases Discovered IPC Energy DL1 IL1 L2 bpred

22. ASPLOS : Sherwood et. al.22 gcc - A Complex Example IPC Energy DL1 IL1 L2 bpred

23. ASPLOS : Sherwood et. al.23 gcc – Phases Discovered IPC Energy DL1 IL1 L2 bpred

24. ASPLOS : Sherwood et. al.24 Outline Examining the Programs Finding Phases Automatically Application to Efficient Simulation Conclusions

25. ASPLOS : Sherwood et. al.25 Efficient Simulation Simulating to completion not feasible –Detailed simulation on SPEC takes months –Cycle level effects can’t be ignored To reduce simulation time –Simulate only a subset of the program at cycle-level accuracy –What subset you pick is very important For accuracy and efficiency

26. ASPLOS : Sherwood et. al.26 Simulation Options Simulate Blind: no estimate of accuracy Single Point: problem with complex programs that have many phases Random Sample: high accuracy, but many sections of similar code, you will be doing a lot of redundant work Choose Multiple Points: by examining the calculated phase information

27. ASPLOS : Sherwood et. al.27 Multiple SimPoints Perform phase analysis For each phase in the program –Pick the interval most representative of the phase (smallest Euclidean distance from centroid) –This is the SimPoint for that phase Perform detailed simulation for SimPoints Weigh results for each SimPoint –According to the size of the phase it represents

28. ASPLOS : Sherwood et. al.28 Sample of SPEC2000 SimPoints...... Intervals of 100M instr weight SimPoint

29. ASPLOS : Sherwood et. al.29 Results – Average Error w.r.t full SimpleScalar simulation, integer pgms

30. ASPLOS : Sherwood et. al.30 An Aside: High Ave. Error Rates ? For L2 and Instr Cache (esp. for integer benchmarks), multiple SimPoints errors were high Why? –Using arithmetic mean of errors –Some programs have small number of misses and errors for them serve as outliers

31. ASPLOS : Sherwood et. al.31 Results – Max Error

32. ASPLOS : Sherwood et. al.32 Outline Examining the Programs Finding Phases Automatically Application to Efficient Simulation Conclusions

33. ASPLOS : Sherwood et. al.33 Conclusions Gap between –Cycle level events –Full program effects Exploit large scale structure –Provide high level model –Find the model with no detail simulation –In conjunction with limited detail simulation

34. ASPLOS : Sherwood et. al.34 Conclusions Our Strategy –Take advantage of structure found in program –Summarize the structure in the form of phases –Find phases using techniques from clustering Use this for doing efficient simulation –High accuracy –With orders of magnitude less time http://www.cs.ucsd.edu/~sherwood

35. Workloads Experimental environment prototype real sys exec- driven sim trace- driven sim stochastic sim Live workload SPEC2000 Micro- benchmark programs SPEC2000 intervals Basic Block Vectors SIMPOINTS monitor analysis Fast forward Synthetic traces “Real” workloads Made-up © 2003, Carla Ellis Data sets

36. Vague idea “groping around” experiences Hypothesis Initial observations Discussion: Destination Initial Hypothesis Pre-proposal 1: Sketch out what information you would need to collect (or have already gathered) in a “groping around” phase to get from a vague idea to the hypothesis stage for your planned project © 2003, Carla Ellis