injecting realistic burstiness to a traditional client-server benchmark

© 2006 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

Injecting Realistic Burstiness to a Traditional Client-Server Benchmark

Ningfang Mi College of William and Mary

Giuliano Casale SAP Research

Ludmila Cherkasova Hewlett-Packard Labs

Evgenia Smirni College of William and Mary

Presenter: Lucy Cherkasova

2 International Conference on Autonomic Computing and Communications (ICAC) 2009

Origin of Burstiness

• Enterprise and Internet applications:

Clients DB Server

Front Server

Web + Application

Server

HTTP request

HTTP reply

SQL query

SQL reply

Burstiness

??

Highly Correlated Arrivals

?

3 International Conference on Autonomic Computing and Communications (ICAC) 2009

Client-Server Benchmark

• E.g., TPC-W (On-line bookstore Web site)

• Exponentially distributed user think timesExponentially distributed user think times

Clients DB Server

Front Server

Web + Application

Server

HTTP request

HTTP reply

SQL query

SQL reply

Burstiness

??

Highly Correlated Arrivals

?

4 International Conference on Autonomic Computing and Communications (ICAC) 2009

• Accounts for randomness and variability … • … but not for burstinessbut not for burstiness

Can we ignore burstiness in the arrival process?

Typical Client-Server Benchmark

BurstinessBurstinessVariabilityVariability

Serv

ice t

ime

Serv

ice t

ime

Request number Request number

5 International Conference on Autonomic Computing and Communications (ICAC) 2009

Why Need to Inject Burstiness?

• Burstiness impacts the performance of resource allocation mechanisms.

• Example: Session-based admission control (SBAC)−User session: sequence of transaction requests−Session is a unit of work−Typically, long sessions are “sales”.−Useful system throughput is the number of

completed sessions−Admission controller admits/rejects sessions

based on observed CPU utilization of the server (a combination of last measurement and some history).

L. Cherkasova, P. Phaal. Session Based Admission Control: a Mechanism for

Peak Load Management of Commercial Web Sites. IEEE J. TOC, June 2002.

6 International Conference on Autonomic Computing and Communications (ICAC) 2009

SBAC

Reject a new session when utilization is above the threshold

Abort an accepted session when the server queue is full

highly undesirable

Front ServerWeb +

ApplicationServer

DB Server

New Client Arrival

Requests from already accepted clients

limited server queue

7 International Conference on Autonomic Computing and Communications (ICAC) 2009

Impact of Burstiness

• We performed experiments for the same workload with different arrival patterns: non-bursty vs bursty

• Aborted ratio = aborted sessions/accepted sessions

highly undesirable

Queue Size Non-bursty Bursty

250 0.04% 11.37%

512 0.00% 6.28%

800 0.00% 2.50%

8 International Conference on Autonomic Computing and Communications (ICAC) 2009

Why Need to Inject Burstiness? (2)• Service level agreement (SLA)

−support given response time guarantees for accepted sessions

• SLA of 1.2s can be supported for 98% of requests with queue size =250 for non-bursty traffic

• Only 90% of requests meet SLA=1.2s bursty traffic.

0

0.2

0.4

0.6

0.8

1

1.2

250 512 800

90th

95th

98th

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

250 512 800

90th

95th

98th

Queue Size Queue Size

Resp

onse

Tim

e (

s)

Resp

onse

Tim

e (

s)

Non-Bursty Bursty

9 International Conference on Autonomic Computing and Communications (ICAC) 2009

Limitations of Standard TPC-W

• Think times are drawn randomly from the exponential distribution identical for all clients

• Exponential think times are incompatibleincompatible with the notion of burstiness.

Need to inject burstiness into user think times.

10 International Conference on Autonomic Computing and Communications (ICAC) 2009

Our Methodology

•Basic Idea: modify the distribution of client think time to create bursty arrivals−Regulate the arrivals by using a 2-phase

Markovian Arrival ProcessMarkovian Arrival Process (MAP).• MAPs are variations of popular On/OFF traffic

models that can be easily shaped to create correlated inter-arrival times

• All clients share a MAP(2) to draw think times

• A new module for client-server benchmarks

−Regulate the intensity of traffic surges by using the index of dispersionindex of dispersion. • A simple tunable knob of burstiness

11 International Conference on Autonomic Computing and Communications (ICAC) 2009

Index of Dispersion (I)• Popular burstiness index in networking• Definition

− SCV – the squared coefficient of variation (variance/mean2)− ρk – autocorrelation coefficients

• i.e., correlation of service times− Exponential: I = SCV = 1

)21(1

k

kSCVI variabilityburstines

s

BurstinessBurstinessVariabilityVariability

Serv

ice

tim

e

Serv

ice

tim

e

Request number Request number

12 International Conference on Autonomic Computing and Communications (ICAC) 2009

Markovian Arrival Process (MAP)

• MAPs have ability to provide variabilityvariability and temporal localitytemporal locality.

• We use a class of MAPs with two states only

Normal

Traffic

λlong

Traffic Surge

λshort

2 states: λshort > λlong

pl,s

ps,l

ps,spl,l

time

Num

. of

arr

ivals

pl,s, ps,l, ps,s, pl,s shape correlation

13 International Conference on Autonomic Computing and Communications (ICAC) 2009

MAP Fitting

• Input − Estimated mean service demands at servers: E[Di]

− Mean user think time E[Z]

− The pre-defined index of dispersion I

• Output− A MAP(2) to draw user think times

14 International Conference on Autonomic Computing and Communications (ICAC) 2009

MAP Fitting (2)

Key: determine (Key: determine (λλshortshort,, λλlonglong, , ppl,sl,s,, p ps,ls,l))• Condition for traffic surge

• Condition for normal traffic

• Mean think time

• We use non-linear optimizer to search for such f and ps,l and find a MAP(2) to best match the predefined I

fDEi ishort /)(1

])[),(max(1 ZEDENfi ilong

)][

][(

1

1

,,

short

longlssl ZE

ZEpp

Departure > Arrival

Arrival > Departure the arrival rate is f times higher than the throughput of the system

the arrival rate is f times slower for balanced system throughput

Balancing the height and the width of the burst

15 International Conference on Autonomic Computing and Communications (ICAC) 2009

Realistic values for Burstiness

−What is the range of realistic values for defining burstiness via index of dispersion I ? • Exponential: I = SCV = 1

• Bursty: values of thousands,

−e.g., FIFA World Cup 1998, one of the servers over 10 days, I = 6300

16 International Conference on Autonomic Computing and Communications (ICAC) 2009

TPC-W Testbed

• On-line bookstore Web site • Testbed: clients + front server + DB server

−Constant number of emulated browsers (EBs)

• User session−sequence of transaction requests

−think time (mean=7 sec) between two transaction requests

• 14 transactions types grouped in three mixes:−Browsing mix

−Shopping mix

−Ordering mix

17 International Conference on Autonomic Computing and Communications (ICAC) 2009

Validation – Arrival Process

• Arrival clients to the system (front server)

Think times drawn by a MAP(2) with I create the bursty conditions.

Shopping Mix

Non-bursty (I=1)

Time (s)

Num

ber

of

act

ive c

lients Bursty (I=4000)

Time (s)

Num

ber

of

act

ive c

lients

18 International Conference on Autonomic Computing and Communications (ICAC) 2009

Validation – Utilization DistributionShopping Mix

Non-bursty (I=1) Bursty (I=4000)

pd

fpd

f

pd

fpd

f

Utilization (%)Utilization (%)

Utilization (%) Utilization (%)

Front

DB

19 International Conference on Autonomic Computing and Communications (ICAC) 2009

Validation - Average Latency

0

500

1000

1500

2000

2500

3000

3500

200 400 600 800 1000 1200Number of EBs

non- bursty

I=4000

Browsing Mix

Resp

onse

tim

e (

ms)

0

200

400

600

800

1000

1200

1400

1600

200 400 600 800 1000 1200Number of EBs

non- bursty

I=4000

Shopping Mix

Resp

onse

tim

e (

ms)

20 International Conference on Autonomic Computing and Communications (ICAC) 2009

Validation – Latency Distributions

0%

20%

40%

60%

80%

100%

0 2000 4000 6000 8000 10000

non-bursty

I=4000

0%

20%

40%

60%

80%

100%

0 1000 2000 3000 4000 5000

non-bursty

I=4000

Browsing Mix

CD

F

Shopping Mix

Response time (ms) Response time (ms)

CD

F

0.83

2.98

0.04

1.25

21 International Conference on Autonomic Computing and Communications (ICAC) 2009

Conclusion• Burstiness critical for autonomic system design

− need representative benchmarks for system evaluation− need reproducible and controllable bursty workloads

• Traditional client-server benchmarks ignore burstiness in arrival flows− e.g., TPC-W with exponential think times

• Explicitly inject burstiness − a simple and tunable parameter: index of dispersion− can introduce different intensity of traffic surges

• http://www.cs.wm.edu/~ningfang/tpcw_codes/

• Supported by NSF grants CNS-0720699 and CCF-08114171 and HPLabs gift.