os spring ’ 03 performance evaluation operating systems spring 2003
Post on 21-Dec-2015
219 views
TRANSCRIPT
![Page 1: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/1.jpg)
OS Spring’03
Performance Evaluation
Operating Systems Spring 2003
![Page 2: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/2.jpg)
OS Spring’03
Performance evaluation There are several approaches for
implementing the same OS functionality
Different scheduling algorithmsDifferent memory management schemes
Performance evaluation deals with the question how to compare wellness of different approaches
Metrics, methods for evaluating metrics
![Page 3: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/3.jpg)
OS Spring’03
Performance Metrics
Is something wrong with the following statement:
The complexity of my OS is O(n)?
This statement is inherently flawedThe reason: OS is a reactive program
What is the performance metric for the sorting algorithms?
![Page 4: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/4.jpg)
OS Spring’03
Performance metrics Response time Throughput Utilization Other metrics:
Mean Time Between Failures (MTBF)Supportable load
![Page 5: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/5.jpg)
OS Spring’03
Response time The time interval between a user’s
request and the system responseResponse time, reaction time, turnaround time, etc.
Small response time is good:For the user: waiting lessFor the system: free to do other things
![Page 6: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/6.jpg)
OS Spring’03
Throughput Number of work units done per time
unitApplications being run, files transferred, etc.
High throughput is goodFor the system: was able to serve many clientsFor the user: might imply worse service
![Page 7: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/7.jpg)
OS Spring’03
Utilization Percentage of time the system is
busy servicing clientsImportant for expensive shared systemLess important (if at all) for single user systems, for real time
systems
Utilization and response time are interrelated
Very high utilization may negatively affect response time
![Page 8: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/8.jpg)
OS Spring’03
Performance evaluation methods
Mathematical analysisBased on a rigorous mathematical model
SimulationSimulate the system operation (usually only small parts thereof)
MeasurementImplement the system in full and measure its performance directly
![Page 9: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/9.jpg)
OS Spring’03
Analysis: Pros and Cons+Provides the best insight into the
effects of different parameters and their interaction• Is it better to configure the system with
one fast disk or with two slow disks?
+Can be done before the system is built and takes a short time
- Rarely accurateDepends on host of simplifying assumptions
![Page 10: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/10.jpg)
OS Spring’03
Simulation: Pros and Cons+Flexibility: full control of
Simulation model, parameters, Level of detail Disk: average seek time vs. acceleration and
stabilization of the head
+Can be done before the system is built- Simulation of a full system is infeasible- Simulation of the system parts does
not take everything into account
![Page 11: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/11.jpg)
OS Spring’03
Measurements: Pros and Cons
+The most convincing- Effects of varying parameter values
cannot (if at all) be easily isolatedOften confused with random changes in the environment
- High cost:Implement the system in full, buy hardware
![Page 12: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/12.jpg)
OS Spring’03
The bottom line Simulation is the most widely used
technique Combination of techniques
Never trust the results produced by the single method Validate with another one E.g., simulation + analysis, simulation +
measurements, etc.
![Page 13: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/13.jpg)
OS Spring’03
Workload Workload is the sequence of things to
doSequence of jobs submitted to the system Arrival time, resources needed
File system: Sequence of I/O operations Number of bytes to access
Workload is the input of the reactive system
The system performance depends on the workload
![Page 14: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/14.jpg)
OS Spring’03
Workload analysis Workload modeling
Use past measurements to create a model E.g., fit them into a distribution
Analysis, simulation, measurement
Recorded workloadUse past workload directly to drive evaluationSimulation, measurement
![Page 15: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/15.jpg)
OS Spring’03
Statistical characterization Every workload item is sampled at
random from the distribution of some random variable
Workload is characterized by a distributionE.g., take all possible job times and fit them to a distribution
![Page 16: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/16.jpg)
OS Spring’03
The Exponential Distribution A lot of low values and a few high
values The distributions of salaries, lifetimes, and
waiting times are often fit the exponential distribution
The distribution ofJob runtimes
Job inter-arrival times
File sizes
![Page 17: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/17.jpg)
OS Spring’03
Exponential Probability Density Function (pdf) If X has an exponential distribution with
parameter , then its probability density function is given by:
where...718.2e
0,)( xexf x
0
![Page 18: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/18.jpg)
OS Spring’03
The Exponential Distribution
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
![Page 19: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/19.jpg)
OS Spring’03
Mean and Variance
If X~Exponential( ), then its mean and variance are given by:
2
1)(and
1)(
XVXE
![Page 20: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/20.jpg)
OS Spring’03
Exponential Probabilities
a x
f(x)
)( aXP
a x
f(x)
)( aXP 0 0
![Page 21: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/21.jpg)
OS Spring’03
Memoryless Property The exponential is the only distribution
with the property that
For modeling runtimes: the probability to run for additional b time units is the same regardless of how much the process has been running already
in average
)()|( bXPaXbaXP
1
![Page 22: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/22.jpg)
OS Spring’03
Fat-tailed distribution The real life workloads frequently do not
fit the exponential distribution Fat-tailed distributions:
5
43
10]100Pr[ :tailed-Fat
10]100Pr[ :lExponentia
20 ,]Pr[
meanX
meanX
xxX
![Page 23: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/23.jpg)
OS Spring’03
Pareto Distribution
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
1 ifmean a havenot Does
tail theisheavier the
islower the:parameter shape a is
)( )1(
a
aa
axxf a
Mean is unboundedThe more you wait, the more additional time you should expect to waitThe longer a job has been running, the longer additional time it is expected to run
![Page 24: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/24.jpg)
OS Spring’03
Exponential vs. Pareto The mean additional time to wait is
determined by the shape of the tailThe fatter tail, the more additional time to wait
For exp.: the tail shape is the same regardless of how much we have waited already=>
The mean additional time stays the same For Pareto: The more we wait, the
fatter tail becomesThe more we wait, the more additional time we will wait
![Page 25: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/25.jpg)
OS Spring’03
Exp. vs. Pareto: Focus on tail
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 6 7 8
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
4 5 6 7 8 9 10 11
0
0.005
0.01
0.015
0.02
0.025
8 9 10 11 12 13 14 15
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
10 11 12 13 14 15 16 17
![Page 26: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/26.jpg)
OS Spring’03
Queuing Systems
Computing system can be viewed as a network of queues and servers
CPU
DiskA
DiskB
queue
queue
queue
newjobs
finishedjobs
![Page 27: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/27.jpg)
OS Spring’03
The role of randomness Arrival (departure) are random
processesDeviations from the average are possibleThe deviation probabilities depend on the inter-arrival time distribution
Randomness makes you wait in queueEach job takes exactly 100ms to completeIf jobs arrive each 100ms exactly, utilization is 100%But what if both these values are on average?
![Page 28: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/28.jpg)
OS Spring’03
Queuing analysis
arrivingjobs
queue serverdepartingjobs
nutilizatioor Load :/
:satisfies system stableA
nd)(jobs/seco rate serviceMean :
nd)(jobs/seco rate arrivalMean :
![Page 29: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/29.jpg)
OS Spring’03
Little’s Law
rn
timeresponse average theis:
system in the jobs ofnumber average theis :
r
n
![Page 30: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/30.jpg)
OS Spring’03
How response time depends on utilization?
Write the average number of jobs as a function of arrival and service rates
Queuing analysis
Substitute it to the Little’s law
![Page 31: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/31.jpg)
OS Spring’03
M/M/1 queue analysis
0 1 32
i
i i
tt
tt
state
got toyou how ofhistory on the dependnot does
1 state to state from moving ofy probabilit The
: intervalan in y probabilit Departure
: intervalan in y probabilit Arrival
![Page 32: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/32.jpg)
OS Spring’03
)())(()()(
:0
)())(()()()(
)()1)(()()(
: at time1 state in the being ofy probabilit The
at time statein being ofy probabilit the:)(
11
11
11
tptptpdt
tdp
t
tptptpt
tpttp
ttptttpttpttp
tt i
t itp
iiii
iiiii
iiii
i
![Page 33: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/33.jpg)
OS Spring’03
01
11
11
11
,2,1for ,)1(
)(
)(0
0)(
,)(lim :statesteady Under
i
dt
tdptp
iii
iii
iii
iii
t
![Page 34: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/34.jpg)
OS Spring’03
1
11
1
1
0
0
1
00
00
00
20
2
0
0
2
00012
i
i
i
i
i
i
i
![Page 35: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/35.jpg)
OS Spring’03
1
)1(1
:Law sLittle'By
1)1(
:system in the jobs ofnumber expected The
rrrn
iin i
iii
![Page 36: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/36.jpg)
OS Spring’03
Response time (utilization)
0
5
10
15
20
25
1 Assume
![Page 37: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/37.jpg)
OS Spring’03
Summary What are the three main performance
evaluation metrics? What are the three main performance
evaluation techniques? What is the most important thing for
performance evaluation? Which workload models do you know? What does make you to wait in
queue? How response time depends on
utilization?
![Page 38: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/38.jpg)
OS Spring’03
To read more Notes Stallings, Appendix A Raj Jain, The Art of Computer
Performance Analysis
![Page 39: OS Spring ’ 03 Performance Evaluation Operating Systems Spring 2003](https://reader035.vdocuments.us/reader035/viewer/2022062313/56649d585503460f94a37613/html5/thumbnails/39.jpg)
OS Spring’03
Next: Processes