Ó 1998 menascé & almeida. all rights reserved.1 part vi system-level performance models for the...

Part VISystem-level Performance Models for the Web(Book, Chapter 8)

Part VI: Learning ObjectivesCharacterize system-level modelsPresent State Transition Diagram (STD) techniqueShow solution for simple modelsShow general solution to STDsShow how to obtain performance metrics from the solution of STDs

System-level ModelsSystem is seen as a black box.Only its input-output characteristics are considered.Inputs: arrivals of requestsOutput: throughput.Throughput, X0(k)(requests/sec)requests in the system, k

System-level ExampleA Web server receives 10 requests/sec.The maximum number of requests in the server is 3. Requests that arrive and find three requests being processed are rejected.

System-level ExampleThe measured throughput as a function of the number of requests is:

System-level Example: a few questionsQ1: What is the probability that an incoming request is rejected?Q2: What is the average number of requests in execution?Q3: What is the average throughput of the Web server? Q4: What is the average time spent by an HTTP request in the Web server?

System-level ExampleCharacterize the Web server by its state, i.e., the number k of requests in the Web server.Assumptions made:homogeneous workload (or single class): all requests are statistically equivalent, only the number of req. countsmemoryless (markovian): how the system arrived at state k does not matter.operational equilibrium: no. requests at beginning of interval = no. request at the end.

System-level ExampleState Transition Diagram (STD)

System-level ExampleAssume we are able to find the values of:pk = probability that (or fraction of time where) there are k requests in the Web server. Question: can we answer all the questions posed before as a function of the pks?

System-level Example: a few questionsQ1: What is the probability that an incoming request is rejected?A: It is the probability that an arriving HTTP request finds 3 requests already being processed. The answer is then p3.

System-level Example: a few questionsQ2: What is the average number of requests in execution?A: using the definition of average:

Nreq = 0 x p0 + 1 x p1 + 2 x p2 + 3 x p3

System-level Example: a few questionsQ3: What is the average throughput of the Web server? A: again, using the definition of average:

X = 0 x p0 + 12 x p1 + 15 x p2 + 16 x p3throughput value at each state

System-level Example: a few questionsQ4: What is the average time spent by an HTTP request in the Web server?A: It will be a function of the average number of requests, Nreq, and the average throughput X. Can be computed by applying Littles law.

System-level Example: computing the pks012310 req/sec10 req/sec10 req/sec12 req/sec15 req/sec16 req/sec use the flow in = flow out principle: the flow into a set of states is equal to the flow out of this set of states in equilibrium.

System-level Example: computing the pks012310 req/sec10 req/sec10 req/sec12 req/sec15 req/sec16 req/secflow in = flow out12 x p1 = 10 x p0

System-level Example: computing the pks Putting it all together: 12 x p1 = 10 x p0 p1 = 10/12 p0 15 x p2 = 10 x p1 p2 = 10/15 p1 = 10x10 p0 15x12 16 x p3 = 10 x p2 p3 = 10/16 p2 = 10x10x10 p0 16x15x12

System-level Example: computing the pks Putting it all together: p1 = 10/12 p0 ; p2 = 10x10 p0; and 15x12 p3 = 10x10x10 p0 16x15x12 But, the Web server has to be in one of the four states at any time. So, p0 + p1+ p2 + p3 = 1.

System-level Example: computing the pksSolving for p0 and then for the other pks we get:

System-level Example: answering the questionsQ1: What is the probability that an incoming request is rejected?A: It is the probability that an arriving HTTP request finds 3 requests already being processed. The answer is then p3 = 0.127 = 12.7%.

System-level Example: answering the questionsQ2: What is the average number of requests in execution?A: using the definition of average:

Nreq = 0 x 0.365 + 1 x 0.305 + 2 x 0.203+ 3 x 0.127 = 1.091 requests

System-level Example: answering the questionsQ3: What is the average throughput of the Web server? A: again, using the definition of average: X = 0 x 0.365 + 12 x 0.305 + 15 x 0.203 + 16 x 0.127 = 8.731 requests/sec.

System-level Example: answering the questionsQ4: What is the average time spent by an HTTP request in the Web server?A: It is a function of the average number of requests, Nreq, and the average throughput X. We need Littles Law to answer this question.

Littles Lawavg. numberpeople in thepub=avg. departurerate from thepubXavg. timespent at thepub

Littles LawWebserveravg. numberrequests in theserver=avg. departurerate from theserverXavg. timespent at theserver8.731 req/sec1.091 req?

System-level Example: answering the questionsQ4: What is the average time spent by an HTTP request in the Web server?A: From Littles Law,

R = Nreq / X = 1.091 / 8.731 = 0.125 sec.

Practice DrillUsing Models for Decision MakingWhat happens if the maximum number of allowed TCP connections changes from 3 to 10?What if the load on the server doubles?What is the impact of a threefold increase in the servers capacity?

Types of System-level ModelsPopulation Size: infinitefiniteService Rate:fixedvariableMaximum Queue Size:unlimitedlimited

Types of System-level Models(population size)Infinite Population: the number of clients is very large. The rate at which requests arrive to the system does not depend on the number of requests being processed in the system.e.g., requests arriving from the Internet to a public Web server.

Types of System-level Models(infinite population)

...SERVERcompletedrequests X12Minfinite populationarrival rate l(requests/sec)internal requests NreqOpen Model

Practice DrillUsing Models for Decision MakingYour Web server is used for e-commerce. The company will announce a new product in the Fall and expects the number of current sales to increase by 50%. Will the server handle the load?What is the new response time? What is the server utilization?

Practice DrillUsing Models for Decision MakingWhat is the meaning of Will the server handle the load?What is the new response time? What is the server utilization?

Types of System-level Models (population size)Finite Population: the number of clients M is limited. The rate at which requests arrive to the system depends on how many have already arrived.e.g., requests arriving to an intranet Web server from a known number of clients within the organization (Intranet).

Types of System-level Models(finite population)SERVERcompletedrequests X12Mfinite population Mthinktime Zinternal requests Nreqarrival rate l(requests/sec)Closed Model

Practice DrillUsing Models for Decision MakingCompany Xs intranet collects sales data from 100 stores and processes reorder data on hundreds of products electronically. The company will merge with company Y that has 150 stores. Does the existing Web site have enough capacity to handle the merged companys demand?

Types of System-level Models(service rate)Fixed Service Rate: the throughput does not vary with the number of requests being processed.Throughput(requests/sec)requests in the system

Types of System-level Models(service rate)Variable Service Rate: the throughput depends on the number of requests being processed.Throughput(requests/sec)requests in the system

Types of System-level Models(maximum queue size)Unlimited Queue Size: all arriving requests are queued for service. No requests are rejected!Limited Queue Size: requests that find more than W requests waiting for service are rejected.n=W?YesNo

Types of System-level Models

Main

SysMod - System Models

this workbook comes with the book

"Capacity Planning for Web Performance: metrics, models, and methods"

Prentice Hall, Upper Saddle River, NJ,1998

PopulationService RateQueue Size

infinitefixedunlimited

infinitefixedlimited

infinitevariableunlimited

infinitevariablefinite

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

Infinite population/constant service rate/infinite queue

This worksheet comes with the book "Capacity Planning for Web Performance",

by D. A. Menasc and V. A. F. Almeida, Prentice Hall, 1998.

Arrival Rate30tps

Service Rate50tps

Utilization60%

Average Number of Requests1.5

Average Response Time0.05

Note that the arrival rate must be less than the service rate!

kPkUtilizationR / S = 1 / (1 - U)

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

Infinite population/constant service rate/finite queue

Arrival Rate30tps

Service Rate50tps

Max. Number of Requests (W)10

Server Utilization59.9%

Server Throughput29.9tps

Average Response Time0.049sec

Fraction of Lost Requests0.24%

kPkWFraction of Lost Requests

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

Infinite population/variable service rate/infinite queue

Arrival Rate30tps

Saturation Point (J)3

Server Average Throughput30.0tps

Note that the arrival rate must be less than X(J)!

Throughput values until saturation

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

Infinite population/variable service rate/finite queue

Arrival Rate30tps

Server Average Throughput28.4tps

Fraction of Lost Requests0.0534300792

Throughput values before saturation

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

Finite population/constant service rate

Average Think Time2sec

Service Rate80tps

Number of Clients50

&A

Page &P

Main Screen

PopFin-VarServ

Finite population/variable service rate

Average Think Time2sec

Saturation Point (J)3tps

Number of Clients50

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

1. This workbook implements the formulas that solve system models

of C/S systems as discussed in Chapter 8 of the book.

2. The input parameters in each worksheet are entered in cells with

purple background.

3. All cells, except for the ones where you must enter values, are protected

to avoid unintended modifications.

Tips:

a. All input parameters should use compatible units (e.g., seconds for

service demands and tps for arrival rates). The results are given in

the units compatible with the input parameters.

b. Do not change the location of any of the cells. This may render your

workbook unusable or generate incorrect results.

c. Always work on a copy of the original workbook provided with the book.

You may open the original workbook and save it under

another name using the Save As option from the File menu.

&A

Page &P

Main Screen

&A

Page &P

System-level models: MethodologyDetermine proper representation for the state of system being modeled (numb. of requests in system, k)Determine set of feasible statesDetermine possible transitions between states, considering possible events (arrival, completion of request)

System-level models: Methodology (2)For each possible state transition, determine transition rate, looking at event that caused the transitionUse flow in = flow out principle to write down equations relating values of pk; remember that sum of all pks is one.

System-level models: Methodology (3)Solve for pks and use them to compute performance metrics:utilization Uaverage throughput Xaverage number of requests Nreqaverage response time Rfraction of requests that are lost because of queue overflow Ploss

Generalized System-level Models0123l0m1m2m3l1l2. . . .l3m4Generalized System-level Models can be solvedusing the flow in = flow out principle

Generalized System-level Models

System-level ModelsExample A Web server receives 30 requests/sec. Itsthroughput function is given below. The serverqueue is limited to five requests. What is theserver utilization, avg. throughput, avg. no. requests, avg. response time, and fraction oflost requests?

System-level ModelsExample (contd)Using the Generalized System-level model equations we get that From Littles Law, avg. response time = avg. no requests/ avg. throughput = 1.85 / 28.4 = 0.065 sec.

System-level ModelsExample (contd)

Practical computation tools

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

Infinite population/constant service rate/infinite queueFormulas: fig. 8.6 pag. 182

30tps

50tps

Utilization U60%

Average Numb. of Requests N1.5

Average Response Time R0.05

kPkUtilizationR / S = 1 / (1 - U)(see fig. 8.5 pag. 180)

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

Infinite population/constant service rate/finite queueFormulas: fig. 8.9 pag. 185

30tps

50tps

Average Number of Requests N1.460

Server Utilization U59.9%

Server Throughput X29.9tps

Average Response Time R0.049sec

Fraction of Lost Requests Ploss0.24%

kPkWFraction of Lost Requests(see fig. 8.8 pag. 184)

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

Infinite population/variable service rate/infinite queueFormulas: (8.5.1) to (8.5.5) pag. 188-189

30tps

Server Average Throughput X30.0tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

Infinite population/variable service rate/finite queueFormulas: (8.5.6) to (8.5.9) pag. 190

30tps

Server Avg. Throughput X28.4tps

Fraction of Lost Requests Ploss0.0534300792

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

Finite population/constant service rateFormulas: (8.5.10), (8.5.11) pag. 191-192

Average Think Time Z2sec

80tps

Number of Clients M50

&A

Page &P

Main Screen

PopFin-VarServ

Finite population/variable service rateFormulas: (8.5.15) to (8.5.18) pag. 193-194

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

30tps

50tps

Utilization U60%

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

30tps

50tps

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

80tps

&A

Page &P

Main Screen

PopFin-VarServ

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

30tps

50tps

Utilization U60%

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

30tps

50tps

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

80tps

&A

Page &P

Main Screen

PopFin-VarServ

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

30tps

50tps

Utilization U60%

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

30tps

50tps

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

80tps

&A

Page &P

Main Screen

PopFin-VarServ

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

30tps

50tps

Utilization U60%

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

30tps

50tps

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

80tps

&A

Page &P

Main Screen

PopFin-VarServ

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

30tps

50tps

Utilization U60%

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

30tps

50tps

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

80tps

&A

Page &P

Main Screen

PopFin-VarServ

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Main

finitefixed

finitevariable

&A

Page &P

Help

PopInf-FixServ-InfQ

30tps

50tps

Utilization U60%

00.400000001.00

10.24000000.11.11

20.14400000.21.25

30.08640000.31.43

40.05184000.41.67

50.03110400.52.00

60.01866240.62.50

70.01119740.73.33

80.00671850.85.00

90.00403110.910.00

100.00241860.9520.00

110.00145120.99100.00

120.0008707

130.0005224

140.0003135

150.0001881

Main Screen

PopInf-FixServ-FinQ

30tps

50tps

00.4014565137.5%

10.2408739218.4%

20.144524339.9%

30.086714645.6%

40.052028853.3%

50.031217361.9%

60.018730471.1%

70.011238280.7%

80.006742990.4%

90.0040458100.2%

100.0024275

&A

Page &P

Main Screen

PopInf-VarServ-InfQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopInf-VarServ-FinQ

30tps

k123

X(k)183550

&A

Page &P

Main Screen

PopFin-FixServ

80tps

&A

Page &P

Main Screen

PopFin-VarServ

k123

X(k)183550

&A

Page &P

Main Screen

Help

Help for SysMod.XLS

purple background.

Tips:

&A

Page &P

Main Screen

&A

Page &P

Part VI: SummarySystem-level models view a server as a black box. Only its arrival process and throughput functions are relevant.State Transition Diagrams (STDs) can be used to find the probability that k requests are in the server. Use the flow in = flow out principle.Littles Law can be used to compute the response time from the average number of requests and from the throughput.

Ó 1998 menascé & almeida. all rights reserved.1 part vi system-level performance models for the...

Documents

x p3systemlevel example

x p0systemlevel example

x p1systemlevel example

systemlevel examplethe

average number of requests

level modelssystem

statesystemlevel example

x p2systemlevel ex