Download presentation

Presentation is loading. Please wait.

1
OS Fall ’ 02 Performance Evaluation Operating Systems Fall 2002

2
OS Fall ’ 02 Performance evaluation There are several approaches for implementing the same OS functionality Different scheduling algorithms Different memory management schemes Performance evaluation deals with the question how to compare wellness of different approaches Metrics, methods for evaluating metrics

3
OS Fall ’ 02 Performance Metrics Is something wrong with the following statement: The complexity of my OS is O(n)? This statement is inherently flawed The reason: OS is a reactive program What is the performance metric for the sorting algorithms?

4
OS Fall ’ 02 Performance metrics Response time Throughput Utilization Other metrics: Mean Time Between Failures (MTBF) Supportable load

5
OS Fall ’ 02 Response time The time interval between a user ’ s request and the system response Response time, reaction time, turnaround time, etc. Small response time is good: For the user: waiting less For the system: free to do other things

6
OS Fall ’ 02 Throughput Number of work units done per time unit Applications being run, files transferred, etc. High throughput is good For the system: was able to serve many clients For the user: might imply worse service

7
OS Fall ’ 02 Utilization Percentage of time the system is busy servicing clients Important for expensive shared system Less important (if at all) for single user systems, for real time systems Utilization and response time are interrelated Very high utilization may negatively affect response time

8
OS Fall ’ 02 Performance evaluation methods Mathematical analysis Based on a rigorous mathematical model Simulation Simulate the system operation (usually only small parts thereof) Measurement Implement the system in full and measure its performance directly

9
OS Fall ’ 02 Analysis: Pros and Cons + Provides the best insight into the effects of different parameters and their interaction Is it better to configure the system with one fast disk or with two slow disks? + Can be done before the system is built and takes a short time - Rarely accurate Depends on host of simplifying assumptions

10
OS Fall ’ 02 Simulation: Pros and Cons + Flexibility: full control of Simulation model, parameters, Level of detail Disk: average seek time vs. acceleration and stabilization of the head + Can be done before the system is built - Simulation of a full system is infeasible - Simulation of the system parts does not take everything into account

11
OS Fall ’ 02 Measurements: Pros and Cons + The most convincing - Effects of varying parameter values cannot (if at all) be easily isolated Often confused with random changes in the environment - High cost: Implement the system in full, buy hardware

12
OS Fall ’ 02 The bottom line Simulation is the most widely used technique Combination of techniques Never trust the results produced by the single method Validate with another one E.g., simulation + analysis, simulation + measurements, etc.

13
OS Fall ’ 02 Workload Workload is the sequence of things to do Sequence of jobs submitted to the system Arrival time, resources needed File system: Sequence of I/O operations Number of bytes to access Workload is the input of the reactive system The system performance depends on the workload

14
OS Fall ’ 02 Workload analysis Workload modeling Use past measurements to create a model E.g., fit them into a distribution Analysis, simulation, measurement Recorded workload Use past workload directly to drive evaluation Simulation, measurement

15
OS Fall ’ 02 Statistical characterization Every workload item is sampled at random from the distribution of some random variable Workload is characterized by a distribution E.g., take all possible job times and fit them to a distribution

16
OS Fall ’ 02 The Exponential Distribution A lot of low values and a few high values The distributions of salaries, lifetimes, and waiting times are often fit the exponential distribution The distribution of Job runtimes Job inter-arrival times File sizes

17
OS Fall ’ 02 Exponential Probability Density Function (pdf) If X has an exponential distribution with parameter, then its probability density function is given by: where

18
OS Fall ’ 02 The Exponential Distribution

19
OS Fall ’ 02 Mean and Variance If X~Exponential( ), then its mean and variance are given by:

20
OS Fall ’ 02 Exponential Probabilities a x f(x) a x 00

21
OS Fall ’ 02 Cumulative Distribution Function The cumulative distribution function (cdf), F(x), is defined as For the Exponential distribution:

22
OS Fall ’ 02 The Exponential CDF

23
OS Fall ’ 02 Memoryless Property The exponential is the only distribution with the property that For modeling runtimes: the probability to run for additional b time units is the same regardless of how much the process has been running already in average

24
OS Fall ’ 02 Fat-tailed distribution The real life workloads frequently do not fit the exponential distribution Fat-tailed distributions:

25
OS Fall ’ 02 Pareto Distribution The more you wait, the more additional time you should expect to wait The longer a job has been running, the longer additional time it is expected to run

26
OS Fall ’ 02 Pareto dist. CDF

27
OS Fall ’ 02 Exponential vs Pareto

28
OS Fall ’ 02 Queuing Systems Computing system can be viewed as a network of queues and servers Shared queues are also possible CPU Disk A Disk B queue new jobs finished jobs

29
OS Fall ’ 02 The role of randomness Arrival (departure) are random processes Deviations from the average are possible The deviation probabilities depend on the inter-arrival time distribution Randomness makes you wait in queue Each job takes exactly 100ms to complete If jobs arrive each 100ms exactly, utilization is 100% But what if both these values are on average?

30
OS Fall ’ 02 Queuing analysis arriving jobs queue server departing jobs

31
OS Fall ’ 02 Little ’ s Law

32
OS Fall ’ 02 M/M/1 queue arriving jobs queue server departing jobs Both interarrival time and service time are exponentially distributed M stands for memoryless

33
OS Fall ’ 02 How average response time depends on utilization? The job arrival and departure are approximated by Poisson processes The distribution of the number of jobs in the system in the steady state is unique Use queuing analysis to determine this distribution Once it is known, can be found Use the Little law to determine

34
OS Fall ’ 02 M/M/1 queue analysis 0132

35
OS Fall ’ 02

36
Response time (utilization)

37
OS Fall ’ 02 A Bank or a Supermarket? departing jobs departing jobs departing jobs departing jobs arriving jobs shared queue CPU1 CPU2 CPU3 CPU4 CPU1 CPU2 CPU3 CPU4 arriving jobs M/M/44 x M/M/1

38
OS Fall ’ 02 It is a Bank! M/M/4 4 x M/M/1

39
OS Fall ’ 02 Summary What are the three main performance evaluation metrics? What are the three main performance evaluation techniques? What is the most important thing for performance evaluation? Which workload models do you know? What does make you to wait in queue? How response time depends on utilization?

40
OS Fall ’ 02 To read more Notes Stallings, Appendix A Raj Jain, The Art of Computer Performance Analysis

41
OS Fall ’ 02 Next: Processes

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google