1. Measuring Performance Chapter 12 CSE807

2. Performance Measurement To assist in guaranteeing Service Level Agreements For capacity planning For troubleshooting performance problems For Performance modeling

3. Step I What are the information sources of performance data What are the monitoring tools available for measuring response time and resource usage What techniques are used to calculate model input parameters

4. Sources of Performance Data Monitors –Provide raw data –Transformation and analysis Product specifications –Manufacturers –Benchmarks

5. Performance Measurement Framework Application oriented workload –Workload classes Should be tied to SLAs –Responsiveness –Availability –Reliability –Throughput

6. Capacity Planning Framework per application Measure and chart as a proactive analysis as a factor of time –Bandwidth usage –CPU and Memory utilization –End-to-end Response Time for frequently used transactions –Number of concurrent users

7. Troubleshooting Framework Monitor collect data for problem detection –Paging –I/O hot spots –Network collisions, error rates –CPU usage –Queue depths

8. Troubleshooting Framework SystemAccounting Data Example sar -u HP-UX FIADGMS1 B.11.00 B 9000/804 02/26/01 00:00:00 %usr %sys %wio %idle 01:00:00 37 8 19 35 02:00:00 22 5 34 39 03:00:00 9 2 16 73 04:00:00 7 2 14 77 05:00:00 18 4 27 51 06:00:00 6 2 14 78 07:00:00 2 2 5 91 08:00:00 1 1 4 94 08:20:00 36 2 2 60 08:40:00 51 1 1 46 09:00:00 43 2 9 46 09:20:01 35 4 15 45 Average 33 4 13 5088

9. Performance Tuning Framework Operating System Level –Establish kernel maximum and minimum values - resource allocation per process Application and Database Level –Oracle SGA –Priority values for classes of workload –Application code tuning

10. Performance Management of Network-based Environment Clients –Application Response Monitor (ARM) –Workstation Performance Monitors Networks –Sniffers, Network Monitoring software Openview, Tivoli, CiscoWorks Servers, Proxies –OS monitoring tools

11. Measuring C/S Performance Client Server LAN Network LAN

12. Measurement Techniques Specify Reference Points –e.g. where to start the measurements from Specify Measurements –e.g. Packets, collisions, error rates statistics Instrument and Collect Data –Install & configure measurement tools Analyze and Transform Data –Accounting data, web logs are too detailed –Transform into useful information

13. A Representation of Measurement Process Application Programs ------------------------------- Server Hardware ------------------------------ Operating System ------------------------------ Hardware Specify Reference Point Specify Measurements Instrument & collect data Analyze & transform data

14. Type of Measurements Event Mode –State of the system defined by a set of variables –e.g. I/O request, Completion of a transaction Tools –Special code inserted at specific point of OS –Records generated with date, time, event and other requested information –High overhead

15. Type of Measurements(cont.) Sampling Mode –At pre-defined time instants –Sampled at timer interrupts based on hardware clock –Overhead depends upon # of variables Size of sampling interval large sampling interval will reduce statistical significance of variable

16. Data Collection Tools Hardware Monitors –Collects data at pre-defined signals –Electronic probes to sense the state of hardware –Registers, memory locations, I/O channels –Do not create overhead –Do not collect any software related information –Portable

17. Data Collection Tools(cont.) Software Monitors –Event or interrupt driven –Can record information regarding software and operating system –Use resources –Easy to install and use –System Level - Global CPU, I/O, memory, queue etc. –Program Level - Application oriented, program identification, elapsed time, memory usage

18. Software Monitors Accounting Systems –Application level granularity –Most popular Program Analyzers –Used for measuring software related data –TP monitoring –Transaction counts, CPU/transaction

19. Software Monitors(cont.) Logs –NT Performance Monitor log can be used to graph the usage statistics –HP-UX Measureware log data can be extracted for historical trend analysis –Web Servers logs (Example 12.1) Access Logs Referrer Logs Agent Logs Error Logs

20. Performance Model Parameters Queues or Devices Workload Classes Workload Intensity Service Demands

21. Performance Model Parameters Queues –Determine what is the point of view –Determine the relevant queues –Queues in a System –Jobs waiting for components CPU, Memory, I/O

22. Performance Model Parameters Workload Classes –Logical grouping of workload –OPEN vs. CLOSED –Open useful for forecasting –Realistic for web servers modeling

23. Performance Model Parameters Workload Intensity - define the amount of work system needs to perform –Arrival Rate - # of cars arriving at the toll plaza per minute –Number of customers - # of customers in bank per hour –Pages requested per transaction –Average packet size

24. Performance Model Parameters Service Demand –For hardware resources –Service time for satisfying a request in the queue –Does not include waiting time –Non-linear scaling for additional CPU caused by OS overhead to manage CPU allocation –Example 12.2

25. Performance Model Parameters Parametric Estimation - combination of information provided by program analyzers –Identify the type of execution environment –Specify the measurement process –Monitor the system and collect the data –Estimate input parameters –Example 12.3

26. Collecting Performance Data Networks –SNMP - Simple Network Management Protocol –Manager and Agent –Management Information Database (MIB) –e.g. Openview, Tivoli –LAN analyzers - packets/sec, collision rate –RMON - Remote Monitoring MIB

27. Collecting Performance Data Servers –NT - Performance Monitor Counter by object types - CPU, Memory, disk etc. Averaging Counters Difference Counter - last and previous Chart, report, alert and logs

28. Collecting Performance Data UNIX –SAR - System Activity Report, a sampling tool –ps –vmstat –iostat –netstat –logfiles

29. Summary Guidelines for performance measurement Types of monitors Measurement techniques –Specify reference point –Specify measurements –Instrument and collect data –Analyze and transform