3/24//2004 Fairness in Queues, H. Levy, CS, TAU 1

How Fair is your Queue

March, 2004

Hanoch Levy

School of Computer Science, Tel Aviv University

Jointly with

Benjamin Avi-Itzhak, RUTGERS University

David Raz, Tel-Aviv University

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 2

Why QUEUES?

“Not Fair!!!”

To provide FAIRNESS in waiting/service

“This is more Fair…”

Sophisticated queues to address fairness issues

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 3

Queueing Theory, queues and fairness

• Queueing theory: Decades of very deep and elaborate research– Queueing structures / policies, distributions– Focus on delay of individual: Moments/ distribution / optimal

operations, many more!

• Practical Applications: Efficient control / operation of: – Bank, computer system, web server, telecom

• Fairness in queues: – Many statements: “this is fair”, “that system is unfair”. – Very little analysis (job fairness):

• Larson (1988), Palm (1953), Mann (1969), Whitt (1984): Discuss justice related / overtaking

• Morris & Wang (85). • Avi-Itzhak & Levy (96)• Wierman & Harchol-Balter (Sigmetrics 2003):

We don’t know how to quantify queue fairness!

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 4

How Important is fairness in queues?

• Recent studies, Rafaeli et. al. [2003] (experimental psychology):– Experiments on humans in multi-queue and single queue – Fairness in queue is very important to people– Perhaps even more than delay itself!

• WFQ: 10’s of papers – fairness among jobs whose duration is O(1)

microsecond • Economic value of queues of humans:

• O(1%) of GNP?

• FAIRNESS INHERENT/CRUCIAL part of queues

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 5

“So – What is the problem? “

ShortLong

• “Take social-science/economics utility-fairness measures and apply to queues”

The difficulty (B): Size vs. seniority dilemma

• HOW??? What is the “PIE”? A “moving target”…

The difficulty (A): Whom to compare a customer against?

• The problem in a nutshell: Short vs. Long

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 6

Motivation / Questions • Should Long be served ahead of Short?

• Is it fair?

• How fair (how much fair) is it to serve Short Ahead of Long?

• Quantify/ Measure Fairness in Waiting lines!

Short(S)Long (L)

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 7

Motivation / Questions (2):Do You like your supermarket?

• How fair is a scheduling policy?

• E.g:• FIFO• LIFO

Or This?

This?

• Multiple Queue (Multi server)• Single queue (multi server)• Queues by job size?

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 8

Motivation / Questions (3):

and your airport? • Multiple Queue (Multi

server)

• Single queue (multi server)

• Smith & Whitt (81), Larson (87), Rothkopf & Rech (87), Wolff (77, 87, 88)

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 9

Applications (computer world)

• Internet revolution: Service shift to computer systems.

• Responsibility of control: shifts to computer programmer/operator

• Examples:• Call centers:

• Web services:

• How should I operate my web-server? FIFO? LIFO? Priorities?

Call center

Web server

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 10

Related Work(1)

• Avi-Itzhak & Levy (96):• Axiomatic approach • Departure point+emphasis: Seniority (Order of

service) • If service times are identical variance of

waiting time measures fairness• Extend to service times • When is it good?

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 11

Recent Related Work

• Wierman & Harchol-Balter (Sigmetrics 2003):• Propose a Fairness Criterion • Class-based approach: For job of size x compute

E[T(x)/x] – If this is bounded by 1/(1-rho) for all x FAIR.

• Results: Analyze the classification for a large variety of policies.– FIFO (FCFS) – is “Always UNFAIR”– LIFO (preemptive) – is FAIR.

• When is it good?

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 12

Conflicting (disturbing) Views

View

Policy

Ordinary person

Queueing theory (WHB 03)

FIFO

LIFO (preemptive)

Unfair (most?)

Fair

Unfair

Fair

(Fairest?)

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 13

Basic Requirements of a Queue Fairness measure

1. Aim for standard: Have a consistent view/intuition

2. Deal with individual, scenario, and system3. Account for both seniority and service requirement:

• Seniority: Service times are identical: – Fairness is a function of seniority

• FIFO most fair / LIFO most unfair • Senior ahead of junior is more fair

• Service requirement: Arrival times are identical – Fairness is a function of service requirement

• Short ahead of long is more fair

4. Yields to analysis

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 14

RAQFM: A Resource Allocation Queueing Fairness Measure

• Aims at meeting these requirements

• In particular:– Long vs. Short– Seniority vs. service

times

SL

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 15

Approach: individual discrimination

• To whom should a job be compared? (moving target!)• Compare to the “public”.• Focus on server resources (aim at equal division)• Weigh the warranted service with the granted service

• Equal Share philosophy (“axiom”): at t: )(/1)( tNtRi

:iC i

i

d

a

i dttN

R)(

1• Warranted service of

• Granted service of :iC i

i

d

a

iii sdttsG )(

iii RGD • Individual discrimination of

(net service)

:iC

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 16

Basic Properties of RAQFM• Resource allocation is proper = zero-sum discrimination (work conserving, non idling)

0)()()( i

ii

ii

i tRtGtD

0 i

iD 0][ DE

• System measure of discrimination: aggregate statistics of

iD

– Eliminate: Expected discrimination

– Reasonable: distance from mean, Var(D), E[|D-E[D]|].

– Measure of Unfairness

• Individual discrimination: iD

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 17

Property 1: Processor Sharing: Ultimate Fairness

• Single server queue• Processor sharing service policy (Kleinrock (64), (67),

Coffman, Muntz & Trotter (70))

0iD• Theorem 1: Under PS

Var[D] = 0 PS is the most fair policy!!

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 18

Property 2: SENIORITY (identical service times)

• Single server queue• Service times are all identical• Arrival times are arbitrary • Theorem 2:

– Serving by order of seniority (FIFO) is most fair

– Serving in reverse order of seniority (LIFO) is most unfair

– Pushing a junior ahead of senior reduces fairness

L L

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 19

Property 3: Service Time (identical arrival times)

• Single server queue

• Arrival times are all identical

• Service times are arbitrary

• Theorem 3: 2

1

)(1

][ i

N

i

SiN

iNDVar

– Serving shortest job first (SJF) is most fair

– Serving Longest job first (LJF) is most unfair

– Pushing a large job ahead of small job reduces fairness

S

L

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 20

Property 4: More advancedThe Long vs. Short case

• Single server queue

• Long and short arrive at different times

S L

• Fairness of two orders depends on relative seniority and relative service times.

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 21

Property 5: BoundsHow bad (good) can it be?

• Bound on individual discrimination

• Use for scale of reference / sanity check

• How good: ii sD (service time)

• How bad: 2/ii WD (waiting time)

• Bound on system discrimination:

• How bad: 2/][0,2|][|0 max2

max NsDEsDE

..# pbinservedN

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 22

Property 6: Locality of Comparison

• Measure can be evaluated by comparing all customers (across busy periods)

• Unique to RAQFM within a large function family.

• Important for fairness: One should compare only relevant customers (within busy periods).

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 23

Property 7: Discrimination Monotonic in Service time

• THM: For an arbitrary system, if service decision is independent of service time, then: – Discrimination monotonically increases with

service time (deterministic)• Larger customers get preferential service

– Discrimination monotonically increases with service time distribution

justification to the prioritization of short jobs!

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 24

RAQFM is Analyzable

• RAQFM yields to analysis via standard queueing theory techniques

• Can compute

• E[D| x] (class discrimination)

• Var[D] (system unfairness)

• Conducted analysis for M/M/1 type: Variety of service orders (FIFO, LIFO, ROS, more…).

• Conducted analysis for Multi-queue / multi-class systems.

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 25

Individual discrimination under FIFO: M/M/1( conditioned on # customers found ahead)

• Discrimination as a function of # customers found at queue

UtilizationTime to cry

Traffic jam at 4AM

Indifferent

Time to smile

Empty super-market Friday noon

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 26

System Unfairness: Compare operation policies

• System Unfairness as a function of system load

Empty system: everyon

e is alone

LIFO: Severe seniority

discrimination

FIFO: no seniority

discrimination

PS:

Absolute

Fairness!

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 27

How other measures relate:Bridging the gap

LIFO (preemptive)

FIFO

Queueing theory (WHB 2003)

Ordinary person

View

Policy

Unfair

Fair

Unfair

Fair

• Wierman & Harchol-Balter (2003) – FIFO (FCFS) – is UNFAIR

– LIFO (preemptive) – is FAIR

Seniority Service times

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 28

RAQFM: account for all factors - bridge the gap

Seniority + service time differences play role (MOST CASES!)

RAQFM agrees with

ordinary person

Service time differences very radical (A few cases)

RAQFM agrees with

Wierman & Harchol Balter

dwarfsmany

giantsfewa

1.0/1,5.0

10/1,01.0

22

11

FIFO: 0.9

LIFO: 0.15

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 29

Comparison of Methods• [AL96]: (SENIORITY)

• Easy to compute

• Order fairness: When the issue is ORDER

• [WHB03]: (SERVICE TIMES)

• Easy to compute

• When jobs do not see each other / do not care of each other.

• RAQFM: (SENIORITY & SERVICE TIMES)

• Somewhat harder to compute

• When issue is waiting time. Also for ORDER fairness.

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 30

Summary• Fairness is fundamental for queueing systems

• No agreed upon measure exists

• RAQFM is a queueing fairness measure that:1. Has a consistent view 2. Deals with individuals, scenario, and system

3. Accounts for both seniority and service requirement4. Admits logically to special cases:

• Service times are identical: – Senior ahead of junior is more fair

• Arrival times are identical – Short ahead of long is more fair

5. Yields to analysis• We analyzed a large variety of queueing systems• Much more work is needed

3/24//2004 Fairness in Queues, H. Levy, CS, TAU 31

Closing Words

• Thank you

• If you have applications at which fairness is relevant – we will be glad to hear.

• Whenever you “enjoy” the queues of your supermarket / bank / airport / … give us a call…