1. Instrumentation and Profiling David Kaeli Department of Electrical and Computer Engineering Northeastern University Boston, MA kaeli@ece.neu.edu

2. Application Areas Trace generation Performance profiling Dynamic optimization Binary translation Power/energy management Architecture/hardware design studies Virtualization Software validation and testing Information security

3. Definitions Instrumentation – A hardware or software probe that provides access to either machine state or program state Profiling – The ability to capture information available through instrumentation Tracing – One form of profiling that captures execution information in the form of a sequential record Static – Techniques applied before a program runs (i.e., at compile time) Dynamic – Techniques applied as the program executes (i.e., at run time)

4. Definitions Trace-driven simulation –Using a previously collected trace as input to a simulator Execution-driven simulation –Tracing and simulating concurrently Tracing Level - –Source – high-level language program –Object – unlinked binary file –Executable – linked, runable, binary file

5. Definitions Binary translation – translating between two binary executable formats Binary optimization – modifying the binary to produce an optimized binary Emulation - One software program, executing software binary, potentially targeting a different ISA

6. Why are Instrumentation and Profiling important topics? Allow for pre-hardware evaluation Enable performance tuning after the hardware has been delivered Enable software to adapt to changes in workload usage Provide quantitative software testing quality Provide for power estimation Enable increased security Enable next-generation hardware design criteria

7. History of Instrumentation IBM Mainframe Instrumentation Systems (1981) –Instrumentation implemented in customized hardware (Logic Support Subsystem) –Provides performance data directly to the customer –Allowed for load balancing and capacity planning –Also provided a level of online diagnostic system Hang detection Recovery Intel In-Circuit-Emulators –Started with Intel 80386 in mid-80’s –Externalized signals to ICE pins –Allowed for external control of hardware functions (e.g., L1 caching) Hardware Counters –VAX 8800, Pentiums, PowerPCs, SPARCs, Alphas –On most CPUs today

8. History of Instrumentation SDS Sigma 7 UCLA 1967

9. History of Profiling Hardware tracing –Early memory studies (Belady, Denning) –VAX-Tracer (Emer and Clark ISCA 1984) –LLATT (IBM 1985) –ATUM (Stanford, Anant Agarwal 1989) –TRAPEDS (Stunkel, Univ. of Illinois 1989) –Trace/Trap (IBM - Sigmetrics 1989) Logic Analyzers –Used to debug early hardware –Used to tune programs (very difficult to do) –Used to obtain traces for architectural studies Counter readers –VAX8800 Histogram Monitors –DCPI – Digital Equipment (Alpha)

10. History of Profiling 1979 – Hewlett Packard Model 85 Used for control and data acquisition

11. Issues with Instrumentation and Profiling Overhead – what type of perturbation does the instrumentation and profiling introduce? Accuracy – does our profile reflect the actual execution? Completeness – do I capture OS, interrupts, libraries?? Portability – is my approach ISA and OS independent? Input independence – does my output change for different inputs? Capacity – how much can I capture? Speed – do I catch the fastest event of interest? Cost – is this worth the investment?

12. Today, Profiling, Simulation and Tracing have become mainstream research areas Cost of simulation is too high Need for increased performance accuracy Need for software security Need to model increasingly complex designs New workshops and symposia: –IEEE International Symposium on Workload Characterization – annual conference –Workshop on Binary Instrumentation and Analysis (held with PACT in 2005 and 2006, special issue of Dec 2005 ACM SIGARCH News) –ACM Symposium on Code Generation and Optimization (feedback directed optimization) – annual conference –ACM Transactions on Architecture and Code Optimization – high quality ACM journal

13. This class will: –Present a range of technologies that fall under the broad definition of profiling and instrumentation –Present some recent implementations that utilize these technologies to improve performance, scalability, reliability, power and security –Involve readings from the literature and a lab assignment

14. Your grade in this class will be based on: –Your course project – 40% –Your paper review – 40% –Your participation in class – 20%  Questions and discussion are encouraged!