1. 1 Low Overhead Program Monitoring and Profiling Department of Computer Science University of Pittsburgh Pittsburgh, Pennsylvania 15260 {naveen, childers}@cs.pitt.edu Department of Computer Science University of Virginia Charlottesville, Virginia 22904 soffa@virginia.edu Naveen Kumar, Bruce ChildersMary Lou Soffa

2. 2 Introduction Program instrumentation: Insertion of additional code into a program –Monitor program behavior or gather information –Can be inserted at source intermediate or binary level Applications –Detect program invariants [Ernst] –Dynamic slicing [Zhang] –Software testing [Misurda] –Software security checks [Scott]

3. 3 Running Example Consider a software security system that monitors the memory behavior of untrusted programs (e.g. Dynamo RIO) –Instrumentation at binary instruction level –Instrument all loads and stores –Program can be instrumented statically as well as dynamically

4. 4 Static instrumentation r[o1] = r[o1] << 10 r[o1] = r[o1] + 0x228 r[o0] = r[o2] << 0x14 r[l4] = r[o0] << 0x14 M[r[l0 ]+ 0x10 ] = r[o2] M[r[o1] + 0x228 ] = r[o0] r[i4] = r[o1] r[l1] = r[o0] jmp r[31] … M[r[l0] + 0x20 ] = r[o0] r[sp] = r[sp] -112 r[o0] = r[o0] << 10 r[o1] = M[r[o0] + 0x3d0 ] … … jmp probe1 jmp probe2 jmp probe3 jmp probe4 probe1: M[r[sp] + -20 ] = r[l0] save call save_gp_regs … r[o0] = M[r[sp] + 0x68 ] r[o0] = r[o0] +0x10 call secure r[o1] = r[g0] + 1 call restore_gp_regs restore r[sp] = r[sp] + 124 M[r[l0 ]+ 0x10 ] = r[o2] jmp probe1_ret probe1: call secure(…) probe2: call secure(…) probe3: call secure(…) probe4: call secure(…) Example from gzip. Instrumentation performed before execution starts

5. 5 Dynamic instrumentation r[o1] = r[o1] << 10 r[o1] = r[o1] + 0x228 r[o0] = r[o2] << 0x14 r[l4] = r[o0] << 0x14 M[r[l0 ]+ 0x10 ] = r[o2] M[r[o1] + 0x228 ] = r[o0] r[i4] = r[o1] r[l1] = r[o0] jmp r[31] … M[r[l0] + 0x20 ] = r[o0] r[sp] = r[sp] -112 r[o0] = r[o0] << 10 r[o1] = M[r[o0] + 0x3d0 ] … … jmp probe1 jmp probe2 jmp probe3 jmp probe4 probe1: call secure(…) probe2: call secure(…) probe3: call secure(…) probe4: call secure(…) Instrumentation performed at run-time on code that executes More powerful than static instrumentation, possibly less expensive

6. 6 Motivation Stumbling block: high overhead –Slowdown by an order of magnitude or more [Ernst] Existing solutions: user guided –Sampling [Arnold] –Smaller data sets analyzed (test data set of SPEC instead of Ref) [Mock] –Less aggressive uses, especially in dynamic settings [Deusterwald] –User has to decide how best to apply instrumentation What is needed are automatic techniques to mitigate the overheads systematically

7. 7 Goals Gather exact information Separate out the accuracy from efficiency –User should focus on what to gather, rather than how to efficiently gather Efficient –Comparable to hand-optimized instrumentation Automatic –No or little user guidance

8. 8 Instrumentation Optimization Costs associated with instrumentation –Dynamic probe count: Number of probes executed –Probe cost: Number of instructions in a probe –Payload cost: Frequency of invocation and cost of payload Optimize instrumentation code to reduce costs –Dynamic probe coalescing –Partial context switches –Partial payload inlining

9. 9 Base Instrumenter r[o1] = r[o1] << 10 r[o1] = r[o1] + 0x228 r[o0] = r[o2] << 0x14 r[l4] = r[o0] << 0x14 M[r[l0 ]+ 0x10 ] = r[o2] M[r[o1] + 0x228 ] = r[o0] r[i4] = r[o1] r[l1] = r[o0] jmp r[31] … M[r[l0] + 0x20 ] = r[o0] r[sp] = r[sp] -112 r[o0] = r[o0] << 10 r[o1] = M[r[o0] + 0x3d0 ] … … jmp probe1 jmp probe2 jmp probe3 jmp probe4 probe1: call secure(…) probe2: call secure(…) probe3: call secure(…) probe4: call secure(…) Base instrumenter generates a list of Instrumentation Points

10. 10 probe1: call secure(…) probe2: call secure(…) probe3: call secure(…) probe4: call secure(…) probe5: call secure(…) probe3: call secure(…) probe4: call secure(…) Dynamic Probe Coalescing r[o1] = r[o1] << 10 r[o1] = r[o1] + 0x228 r[o0] = r[o2] << 0x14 r[l4] = r[o0] << 0x14 M[r[l0 ]+ 0x10 ] = r[o2] M[r[o1] + 0x228 ] = r[o0] r[i4] = r[o1] r[l1] = r[o0] jmp r[31] … M[r[l0] + 0x20 ] = r[o0] r[sp] = r[sp] -112 r[o0] = r[o0] << 10 r[o1] = M[r[o0] + 0x3d0 ] … … jmp probe1 jmp probe2 jmp probe3 jmp probe4 jmp probe5 jmp probe6 probe6: call secure(…)

11. 11 jmp probe6 probe6: call secure(…) probe4: call secure(…) Partial Context Switch r[o1] = r[o1] << 10 r[o1] = r[o1] + 0x228 r[o0] = r[o2] << 0x14 r[l4] = r[o0] << 0x14 M[r[l0 ]+ 0x10 ] = r[o2] M[r[o1] + 0x228 ] = r[o0] r[i4] = r[o1] r[l1] = r[o0] jmp r[31] … M[r[l0] + 0x20 ] = r[o0] r[sp] = r[sp] -112 r[o0] = r[o0] << 10 r[o1] = M[r[o0] + 0x3d0 ] … jmp probe4 probe6: M[r[sp] -20 ] = r[l0] M[r[sp] -28 ] = r[o1] save call save_gp_regs … effective address … call secure … effective address … call secure … effective address … call secure call restore_gp_regs restore … … jmp probe6_ret Regs. used in payload: {…} Not used: {g0…g7} Analyze register usage in payloadRemove spill and reload of GP registers

12. 12 void secure(address) { if(address > REDZONE) return; redAlerts++; createReport(); if(critical(address)) assert(address); } r[o1] = M[r[g1]+0] r[o1] = r[o1] - r[o0] r[i0] = 1 jmp r[31] … r[o3] = M[r[g2] +0] r[o3] = r[o3] + 1 … !call createReport … !call assert call __full_secure probe6: M[r[sp] -20 ] = r[l0] M[r[sp] -28 ] = r[o1] r[sp] = r[sp] -140 … effective address … call secure … effective address … call secure … effective address … call secure r[sp] = r[sp] + 140 … jmp probe6_ret jmp probe6 Partial Payload Inlining r[o1] = r[o1] << 10 r[o1] = r[o1] + 0x228 r[o0] = r[o2] << 0x14 r[l4] = r[o0] << 0x14 M[r[l0 ]+ 0x10 ] = r[o2] M[r[o1] + 0x228 ] = r[o0] r[i4] = r[o1] r[l1] = r[o0] jmp r[31] … M[r[l0] + 0x20 ] = r[o0] r[sp] = r[sp] -112 r[o0] = r[o0] << 10 r[o1] = M[r[o0] + 0x3d0 ] … … jmp probe4 void __full_secure(address, tag) { __full_secure(address, tag); } void __inlined_secure(address) {

13. 13 Implementation Strata: dynamic translation system [Scott et. al.] –Generates code at run-time for an application –Suitable for dynamic instrumentation FIST: base instrumentation system [Kumar et. al.] –Flexible for diverse instrumentation needs –Generates a list of instrumentation points (IP’s) INS-OP: developed in this work –Constructs an IR for the list of IP’s obtained from FIST –Each optimization is a pass that modifies the IR

14. 14 Case Studies Case study 1: Program profiling –Lightweight instrumentation application –Lower initial overhead implies lesser benefits –Demonstrates efficacy of the optimizations in an unfavorable scenario Case study 2: Memory simulation –Relatively heavy-weight instrumentation application –Can compare with state-of-the-art systems to see the benefits of optimization

15. 15 Case study 1: Program profiling The benefit of optimization varies; depends upon the initial overhead The speedups range from 1.26 to 2.63

16. 16 Case study 2: Memory Simulation Strata-Embra is a SPARC implementation of cache simulator from SimOS Strata-Embra-Opt is optimized cache simulator using INS-OP INS-OP optimizes the fastest cache simulator we could find by 2 - 3.3 times

17. 17 Conclusions Introduced “instrumentation optimization” to reduce the cost of instrumented code –Reduced probe count –Reduce cost of an individual probe –Reduce the cost of payload –Speedups between 1.2 - 3.3 times More detailed information gathering –Accuracy need not be sacrificed for efficiency Feasibility of certain applications –Run-time monitoring more feasible –Example: applications that perform continuous testing

18. 18 Effectiveness of optimizations