courtesy of r.martin, rutgers univ introduction to queueing theory comp5416 advanced network...

Courtesy of R.Martin, Rutgers Univ

Introduction to Queueing TheoryCOMP5416Advanced Network Technologies

M/M/1-2School of Information Technologies

Queuing Theory

• View network as collections of queues– FIFO data-structures

• Queuing theory provides probabilistic analysis of these queues

• Examples:– Average length– Probability queue is at a certain length– Probability a packet will be lost

M/M/1-3School of Information Technologies

Little’s Formula (aka. Little’s Law)

• Little’s Law:– Mean number tasks in system = arrival rate x mean

residence time• Observed before, Little was first to prove• Applies to any system in equilibrium, as long as

nothing in black box is creating or destroying tasks

SystemArrivals Departures

M/M/1-4School of Information Technologies

Example using Little’s Formula

• Consider 40 customers/hour visit Hungry Jack Restaurant– Average customer spend 15 minutes in the restaurant

• What is average number of customers at at given time?

10/25.0/40 custhrhrcustTr r

M/M/1-5School of Information Technologies

Model Queuing System

• Use Little’s formula on complete system and parts to reason about average time in the queue

M/M/1-6School of Information Technologies

Kendal Notation

• Six parameters in shorthand (x/x/x/x/x/x)– First three typically used, unless specified

1. Arrival Distribution2. Service Distribution3. Number of servers4. Total Capacity (infinite if not specified)5. Population Size (infinite)6. Service Discipline (FCFS/FIFO)

M/M/1-7School of Information Technologies

Distributions

• M: Exponential• D: Deterministic (e.g. fixed constant)• Ek: Erlang with parameter k

• Hk: Hyperexponential with param. k• G: General (anything)

• M/M/1 is the simplest ‘realistic’ queue

M/M/1-8School of Information Technologies

Kendal Notation Examples

• M/M/1:– Exponential arrivals and service, 1 server, infinite capacity

and population, FCFS (FIFO)• M/M/n

– Same as above, but n servers• G/G/3/20/1500/SPF

– General arrival and service distributions, 3 servers, 17 queue slots (20-3), 1500 total jobs, Shortest Packet First

M/M/1-9School of Information Technologies

M/M/1 Queue

Single Isolated Link

•Assume messages arrive at the channel according to a Poisson process at a rate messages/sec.

•Assume that the message lengths have a negative exponential distribution with mean 1/ bits/message.

•Channel transmits messages from its buffer at a constant rate c bits/sec

•The buffer associated with the channel can considered to be effectively of infinite length.

=> an M/M/1 queue.

M/M/1-10School of Information Technologies

M/M/1

• The first M says that the interarrival times to the queue are negative-exponential (ie Markovian or memoryless);

• The second M says that the service times are negative-exponential (ie Markovian or memoryless again);

• The 1 says that there is a single server at the queue.

• Shorthand notation for "a queue with Poisson arrivals, negative exponentially distributed message lengths, a single server, and infinite buffer space".

M/M/1-11School of Information Technologies

• Arrival rate messages/sec,

• Message lengths neg exp, mean 1/ bits/message,

• Transmission rate c bits/sec, – ie messages transmitted at rate = c messages/sec.

• Define the state of the system as the total number of messages at the link (waiting + being transmitted).

• The system is therefore a Markov chain, since both the arrival and message length distributions are memoryless.

0 1 2

M/M/1-12School of Information Technologies

M/M/1 queue model

Tw 1/

Tr

r

w

M/M/1-13School of Information Technologies

Solving queuing systems

• Given: : Arrival rate of jobs (packets) : Service rate of the server (output link)

• Solve:– r: average number of items in queuing system– Tr: avg. time an item spends in whole system

– Tw: avg. time an item waits in the queue

M/M/1-14School of Information Technologies

Equilibrium conditions

0 1 2

Boundary for flux balance

Boundary for global balance

,2,11 ipp ii

M/M/1-15School of Information Technologies

• Solving the flux balance equations recursively, we get

02

2

1 ppppi

iii

0pp ii

Define occupancy

M/M/1-16School of Information Technologies

(normalising condition)

(geometric distribution)

)1(

1

00

i

ip

,2,1,0)1( ip ii

Note:Solution valid only for < 1 (stability condition)For 1, there is no steady state!

M/M/1-17School of Information Technologies

M/M/1 Queue Length Distribution

0

0.1

0.2

0.3

0.4

0.5

0 1 2 3 4 5 6 7 8 9

0

0.1

0.2

0.3

0.4

0.5

0 1 2 3 5 5 6 7 8 9

M/M/1-18School of Information Technologies

M/M/1 Mean Number in the System

1

)1()1(

)1(}{

2

00

i

ii

i

ipiRE

M/M/1-19School of Information Technologies

M/M/1 Mean Delay

1

11}{

111}{1

}|{}{delayMean

0

0

R

ii

iiRR

TE

REp

i

piTETE

0.00

2.00

4.00

6.00

8.00

10.00

E{T

}

0 0.25 0.5 0.75 1

M/M/1 queue

M/M/1-20School of Information Technologies

M/M/1 Mean Waiting Time

• The waiting time is defined as the time in the queue, excluding service time:

1

11

1

111}{}{ RW TETE

M/M/1-21School of Information Technologies

Summary of M/M/1 (infinite buffer)

1

1}{

1

11}{

1}{

1

W

R

TE

TE

RE

M/M/1-22School of Information Technologies

Little’s Formula

• If messages/sec enter a “system”, and each spends, on average, W seconds there, then the total number in the “system” must be given by  W by a simple flow argument.

• Proof for M/M/1:

}{}{

}{1

1

/1

1

11}{

R

R

TERE

RETE

M/M/1-23School of Information Technologies

M/M/1/n – Finite Buffer

0 1

2

n

nipp ii ,,1,00

1

1

00

1

1

n

n

i

ip

nip ini ,,1,0

1

11

M/M/1-24School of Information Technologies

M/M/1/n

• Probability that the buffer will be blocked:

nnnB pP

11

1}blocked is messagePr{

M/M/1-25School of Information Technologies

M/M/1/n exampleExample : = 0.5• Assume that we want

• Choose n so that

• i.e. choose n = 9

310BP

96.91101

105.0

105.01

5.05.0

5.01

5.01

3

31

31

1

1

n

n

n

nn

n

M/M/1-26School of Information Technologies

M/M/1/n – validity of solution

• Solution for the finite buffer queue is valid for all – not just for  < 1 as was the case for the infinite

buffer.

• This is because the finite buffer copes with overloads by blocking messages, rather than by creating a backlog of messages.

M/M/1-27School of Information Technologies

M/M/1/n – Little’s formula

and it is only these messages which contribute to the buffer size. This implies that:

)1( BP

}{)1(}{}{ RBR TEPTERE

To use Little's formula for the finite buffer case, note that the rate of non-blocked messages is given by:

M/M/1-28School of Information Technologies

Summary of M/M/1/n

n

i

in

n

ii

n

n

B

RB

iipRE

P

TEPRE

01

0

1

1

)1(}{

1

)1(

}{)1(}{

1) <or 1 be(can

M/M/1-29School of Information Technologies

State dependent Queues

Let i= arrival rate when the system is in state i

i = service rate when the system is in state i

0 1 2

Balance Equations:,2,111 ipp iiii

,2,1021

110 ippi

ii

 with solution

M/M/1-30School of Information Technologies

M/M/2

• State dependent with

,3,22

1

,1,0

i

i

i

i

i

0 1 2

0101 2)2(ppp

ii

i

i

i

i

M/M/1-31School of Information Technologies

M/M/1 vs M/M/2

1

1

2

1}{ 2 rate server, 1RTE

)1)(1(

11}{ rateeach servers, 2

RTE

12

1

}{

}{

rateeach servers, 2

2 rate server, 1

R

R

TE

TE

Compare M/M/2 (2 servers – each at rate ) to that of a single link at rate 2 .

(M/M/1 result)

(M/M/2 result)

Conclusion: Single fast server is always more efficient!

M/M/1-32School of Information Technologies

M/M/1 vs M/M/2M/M/1 vs M/M/2

0

1

2

3

4

Occupancy

Mea

n D

elay

(u

nit

s o

f si

ng

le p

acke

t se

rvic

e ti

mes

)

M/M/1: 1 server, rate 2

M/M/2: 2 servers, each rate

M/M/1-33School of Information Technologies

M/M/

(a Poisson distribution)

0 1 2

Ai

Aep

ip

iA

i

i

i with !!

0

M/M/1-34School of Information Technologies

M/M/1 delay distribution

(an exponential distribution)

tT etp

R

)1()1()(

• Probability of delay within certain limit:

tR etT )1(1)Pr(

M/M/1-35School of Information Technologies

Example

Suppose that packets arrive at a link at a Poisson rate of 250 packets/second, and the packet lengths are exponentially distributed with mean length 1000 bits/packet. Assume that the buffer length is sufficiently large that it can be assumed to be effectively infinite.

Find the required capacity of the link in bits/second so that the following criteria are satisfied:

(1) Mean delay 10 msec

(2) Pr{delay 20 msec} 0.10

M/M/1-36School of Information Technologies

Solution

bps000,2502500001000250

:Occupancy

ccc

bps000,350

010.0250000

1000

2500001

11000

1

11}{ :DelayMean

c

c

cc

TE R

M/M/1-37School of Information Technologies

Solution

In order to satisfy both criteria, we need to choose capacity >365,129 bps.

(The next highest standard rate is 384 kbps.)

tR etT )1(}Pr{ :Delay of Percentile

10.0}msec 20Pr{ 1000

20

1000

250000

c

R eT

bps 129,365c