1. The Memory Behavior of Data Structures Kartik K. Agaram, Stephen W. Keckler, Calvin Lin, Kathryn McKinley Department of Computer Sciences The University of Texas at Austin

2. 2 Memory hierarchy trends Growing latency to main memory Growing cache complexity –More cache levels –New mechanisms, optimizations Growing application complexity –Lots of abstraction Hard to predict how an application will perform on a specific system

3. 3 Application understanding is hard Observations can generate Gigabytes of data –Aggregation is necessary Current metrics are too lossy –Different application behaviors →similar miss-rate New metrics needed, richer but still concise Our approach: data structure decomposition

4. 4 Why decompose by data structure? Irregular app = multiple regular data structures –while (tmp) tmp=tmp->next; Data structures are high-level –Results easy to visualize –Can be correlated back to application source code

5. 5 Outline Data structure decomposition using DTrack –Automatic instrumentation + timing simulation Methodology –Tools, configurations simulated, benchmarks studied Results –Data structures causing the most misses –Different types of access patterns –Case study: data structure criticality

6. 6 Conventional simulation methodology Simulated application shares resources with simulator –disk, file system, network..but is not aware of it Application Simulator Host Processor Resources

7. 7 A different perspective Application can communicate with simulator Leave core application oblivious; automatically add simulator-aware instrumentation Application Simulator Resources

8. 8 DTrack Application Sources Detailed Statistics Application Executable Instrumented Sources Source Translator CompilerSimulator - DTrack’s protocol for application-simulator communication

9. 9 DTrack’s protocol 1.Application stores mapping at a predetermined shared location –(start address, end address) → variable name 2.Application signals simulator somehow We enhance ISA with new opcode Other techniques possible 3.Simulator detects signal, reads shared location 1.Application stores mapping at a predetermined shared location –(start address, end address) → variable name 2.Application signals simulator somehow We enhance ISA with new opcode Other techniques possible 3.Simulator detects signal, reads shared location Simulator now knows variable names of address regions

10. 10 DTrack instrumentation Global variables: just after initialization int globalTime ; int main () { … } int Time ; int main () { print (FILE, “Time”, Time, sizeof(Time)); … asm (“mop”); } After: Before:

11. 11 DTrack instrumentation Heap variables: just after allocation x = malloc(4); DTRACK_PTR = x ; DTRACK_NAME = “x” ; DTRACK_SIZE = 4 ; asm(“mop”); After: Before:

12. 12 Design decisions Source-based rather than binary-based translation Local variables – no instrumentation –Instrumenting every call/return is too much overhead –Doesn’t cause many cache misses anyway –Dynamic allocation on the stack: handle alloca just like malloc Signalling opcode: overload an existing one –avoid modifying compiler, allow running natively

13. 13 Instrumentation can perturb app behavior Minimizing perturbance –Global variables are easy One-time cost –Heap variables are hard DTRACK_PTR, etc. always hit in the cache Measuring perturbance –Communicate specific start and end points in application to simulator –Compare instruction counts between them with and without instrumentation Minimizing perturbance –Global variables are easy One-time cost –Heap variables are hard DTRACK_PTR, etc. always hit in the cache Measuring perturbance –Communicate specific start and end points in application to simulator –Compare instruction counts between them with and without instrumentation Instruction count <4% even with frequent malloc

14. 14 Outline Data structure decomposition using DTrack –Automatic instrumentation + timing simulation Methodology –Tools, configurations simulated, benchmarks studied Results –Data structures causing the most misses –Different types of access patterns –Case study: data structure criticality

15. 15 Methodology Source translator: C-Breeze Compiler: Alpha GEM cc Simulator: sim-alpha –Validated model of 21264 pipeline Simulated machine: Alpha 21264 –4-way issue, 64KB 3-cycle DL1 Benchmarks: 12 C applications from SPEC CPU2000 suite

16. 16 Major data structures by DL1 misses % DL1 misses

17. 17 twolf mcf art f1[i] i=i+1 t1 = b[c[i]  cblock] t2 = t1  tile  term t3 = n[t2  net] … bu[i] i=i+1 node[i] i=i+1 node  child node  parent node  sibling node  siblingp i=rand() node = DFS(node) twolf mcf art f1[i] i=i+1 t1 = b[c[i]  cblock] t2 = t1  tile  term t3 = n[t2  net] … bu[i] i=i+1 node[i] i=i+1 node  child node  parent node  sibling node  siblingp i=rand() node = DFS(node) Large variety in access patterns Code + Data profile = Access pattern

18. 18 Most misses ≣ Most pipeline stalls? Process: –Detect stall cycles when no instructions were committed –Assign blame to data structure of oldest instruction in pipeline Result –Stall cycle ranks track miss count ranks –Exceptions: tds in 179.art search in 186.crafty

19. 19 Summary Toolchain for mapping addresses to high- level data structure –Communicating information to simulator Reveals new patterns about applications –Applications show wide variety of distributions –Within an application, data structures have a variety of access patterns –Misses not correlated to accesses or footprint –..but they correlate well with data structure criticality