![Page 1: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/1.jpg)
1
Mor Harchol-BalterCarnegie Mellon
UniversityComputer Science
Heavy Tails: Performance Models & Scheduling Disciplines
![Page 2: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/2.jpg)
2
“size” = service requirement
load < 1
jobs SRPT
jobs
jobs PS
FCFS
Q: Which minimizes mean response time?
![Page 3: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/3.jpg)
3
“size” = service requirement
jobs SRPT
jobs
load < 1
jobs PS
FCFS
Q: Which best represents scheduling in web servers ?
![Page 4: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/4.jpg)
4
IDEA: Use SRPT instead of PS in Web servers
client 1
client 2
client 3
“Get File 1”
“Get File 2”
“Get File 3” Request for file
Requested file
Linux 0.S.
1231000
APACHEWEB
SERVERInternet
![Page 5: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/5.jpg)
5
Objections to SRPT:
Need to know “size” of request
Unfairness to requests for big files
![Page 6: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/6.jpg)
6
Outline of Talk
THEORY
IMPLEMENT
Papers are joint with Adam Wierman & Bianca Schroeder
I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]
II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]
III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]
THEORY
![Page 7: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/7.jpg)
7
Outline of Talk
THEORY
IMPLEMENT
www.cs.cmu.edu/~harchol/
I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]
II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]
III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]
THEORY
![Page 8: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/8.jpg)
8
SRPT has a long history ...
1966 Schrage & Miller derive M/G/1/SRPT response time:
1968 Schrage proves optimality
1979 Pechinkin & Solovyev & Yashkov generalize
1990 Schassberger derives distribution on queue length
BUT WHAT DOES IT ALL MEAN?
![Page 9: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/9.jpg)
9
SRPT has a long history (cont.)
1990 - 97 7-year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN!
1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS!
Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs ...
Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”
![Page 10: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/10.jpg)
10
Real-world job sizes are Heavy Tailed
Job size (x seconds)
log-log plot
xxsizejob 1} Pr{
[Sigmetrics 96]
Heavy-tailedProperty:
“Largest 1% of jobs comprise half the load.”
![Page 11: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/11.jpg)
11
Unfairness Question
SRPT
PS?
?
Let =0.9. Let G: Bounded Pareto(= 1.1, max=1010)
Question: Which queue does biggest job prefer?
M/G/1
M/G/1
![Page 12: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/12.jpg)
12
Results on UnfairnessLet =0.9. Let G: Bounded Pareto(= 1.1, max=1010)
SRPT
PSI SRPT
![Page 13: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/13.jpg)
13
Results on UnfairnessLet G: Bounded Pareto(= 1.1, max=1010)
PS
SRPT
![Page 14: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/14.jpg)
14
Unfairness – General Distribution
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
![Page 15: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/15.jpg)
15
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
Proof:
x
tdt
0 )1 1x
)
x
xFx
2
2
))1(2
(
0x
dttft 2 )(
E[Wait(x)]SRPT
E[Res(x)]SRPT E[T(x)]PS
![Page 16: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/16.jpg)
16
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
Proof cont.
x
tdt
0 )1 1x
Need sufficient condition s.t.
)
x
xFx
2
2
))1(2
(
0x
dttft 2 )(
![Page 17: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/17.jpg)
17
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
Proof cont.
Need sufficient condition s.t.
x t
0 1)
t
yf(y)dy) t
)
x
xFx
2
2
))1(2
(
0x
dttft 2 )(
![Page 18: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/18.jpg)
18
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
Proof cont.
Need sufficient condition s.t.
1
)())( 2x
dttftxx
)())( 2 xFxxx Observe:
)
x
xFx
2
2
))1(2
(
0x
dttft 2 )(
![Page 19: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/19.jpg)
19
Proof cont.
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
1 x 2))1(2
) xFx2 (0x
dttft 2 )( ) xFx2 (0x
dttft 2 )(
Suffices that: 2(1 - (x))2 > 1 - .
Suffices that (x) < 1/2
![Page 20: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/20.jpg)
20
Intuition
E[Wait(x)] E[Res(x)]: small
small
1) All-can-win-theorem: For all distributions, if ½, E[T(x)]SRPT E[T(x)]PS for all x.
2) Under Bounded Pareto distribution ( = 1.1), , E[T(x)]SRPT < E[T(x)]PS for all x. *HT Property*
![Page 21: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/21.jpg)
21
Outline of Talk
THEORY
IMPLEMENT
www.cs.cmu.edu/~harchol/
I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]
II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]
III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]
THEORY
![Page 22: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/22.jpg)
22
What is fair?
xxTxS
1
1PSxSE
Slowdown forjob of size x
Response time forjob of size x
PS does not bias towards any particular job size. Slowdown isindependentof size
Definition: A policy P is fair if E[S(x)]P ≤ E[S(x)]PS for all x. Otherwise, P is unfair.
![Page 23: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/23.jpg)
23
Always Fair Always Unfair
Fair for all loads and distributions
Unfair for all loads and distributions
Sometimes Unfair
Fair for some loads, and unfair for other loads*
Classification of Scheduling Policies
* and distributions
![Page 24: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/24.jpg)
24
Testing your intuition:
* and distributions
Always Fair Always Unfair
Fair for all loads and distributions
Unfair for all loads and distributions
Sometimes Unfair
Fair for some loads, and unfair for other loads*
• Where does SRPT lie? __________• What about other policies that prioritize based on remaining size? ____• What about preemptive policies that prioritize based on age? _____• What about preemptive policies that prioritize based on size? _____
![Page 25: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/25.jpg)
25
Classification of Scheduling Policies
Lots of open problems…
FSPAlwaysFAIR
AlwaysUnfair
Sometimes Unfair
Age-BasedPolicies
Preemptive Remaining-size based Policies
Preemptive Size-basedPolicies
Non-preemptive
PS
PLCFS
FB
PSJFLRPT
FCFS LJFSJF
SRPT
![Page 26: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/26.jpg)
26
Always Unfair
Unfair for all loads and distributions
Theorem: Any preemptive, size based policy, P, is Always Unfair.
Always Unfair
Case1: A finite size, y, receives lowest priority
Case 2: No finite size receives lowest priority (2a) Priorities decrease monotonically -- PSJF (2b) Priorities decrease non-monotonically.
![Page 27: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/27.jpg)
27
Always Unfair
Unfair for all loads and distributions
Theorem: Any preemptive, size based policy, P, is Always Unfair.
Always Unfair
Case1: A finite size, y, receives lowest priority
y
V = Work in System
yVyBEySE P
1
11
1yVBE
![Page 28: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/28.jpg)
28
Always Unfair
Unfair for all loads and distributions
Theorem: Any preemptive, size based policy, P, is Always Unfair.
Always Unfair
Case2a: Priorities decrease monotonically (PSJF)
1
1lim PSJF
xxSE
Infinite sized job has lowest priority ...
… but that job is treated fairly?
![Page 29: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/29.jpg)
29
Always Unfair
Unfair for all loads and distributions
Theorem: Any preemptive, size based policy, P, is Always Unfair.
Always Unfair
Case2a: Priorities decrease monotonically (PSJF)
E[S(x)]
0
PS
PSJF
x
Finding a hump shows PSJF is Always Unfair
![Page 30: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/30.jpg)
30
Always Unfair
Unfair for all loads and distributions
Theorem: Any preemptive, size based policy, P, is Always Unfair.
Always Unfair
Case2b: Priorities decrease non-monotonically
E[S(x)]
0
PS
PSJF
x
Find y beyond which PSJF treats all jobs unfairly.Find x > y, where x has lower priority than y.=> x is treated unfairly.
![Page 31: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/31.jpg)
31
The mysterious hump
0 x
0
PS
PSJF
E[S(x)]
x
This hump appears in many common policies, 1. PSJF2. FB3. SRPT 4. SJF
![Page 32: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/32.jpg)
32
Outline of Talk
THEORY
IMPLEMENT
www.cs.cmu.edu/~harchol/
I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01]
II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03]
III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03]
THEORY
![Page 33: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/33.jpg)
33
What does SRPT mean within a Web server?
• Many devices: Where to do the scheduling?
• Many jobs at once.
From theory to practice:
![Page 34: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/34.jpg)
34
Server’s Performance Bottleneck
5
Linux 0.S.
WEBSERVER(Apache)
client 1
client 2
client 3
“Get File 1”
“Get File 2”
“Get File 3”
Rest ofInternet ISP
Site buyslimited fractionof ISP’s bandwidth
We schedule bandwidth at server’s uplink.
![Page 35: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/35.jpg)
35
Network/O.S. insides of traditional Web server
Sockets take turnsdraining --- FAIR = PS.
WebServer
Socket 1
Socket 3
Socket 2Network Card
Client1
Client3
Client2BOTTLENECK
![Page 36: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/36.jpg)
36
Network/O.S. insides of our improved Web server
Socket corresponding to filewith smallest remaining datagets to feed first.
WebServer
Socket 1
Socket 3
Socket 2Network Card
Client1
Client3
Client2
priorityqueues.
1st
2nd3rd
S
M
L
BOTTLENECK
![Page 37: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/37.jpg)
37
Experimental Setup
Implementation SRPT-based scheduling: 1) Modifications to Linux O.S.: 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design.
Linux 0.S.
123
APACHEWEB
SERVER
Linux
123
200
Linux
123
200
Linux
123
200
switch
WAN
EM
UW
AN E
MU
WAN
EM
U
![Page 38: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/38.jpg)
38
Experimental Setup
APACHEWEB
SERVER
Linux 0.S.
123
Linux
123
200
Linux
123
200
Linux
123
200
switch
WAN
EM
UW
AN E
MU
WAN
EM
U
Trace-based workload: Number requests made: 1,000,000Size of file requested: 41B -- 2 MBDistribution of file sizes requested has HT property.
FlashApache
WAN EMUGeographically-dispersed clients
10Mbps uplink100Mbps uplinkSurgeTrace-basedOpen systemPartly-open
Load < 1Transient overload
+ Other effects: initial RTO; user abort/reload; persistent connections, etc.
![Page 39: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/39.jpg)
39
Load
FAIR
SRPTMea
n R
espo
nse
Tim
e (s
ec)
Results: Mean Response Time
.
.
.
.
.
.
![Page 40: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/40.jpg)
40
Percentile of Request Size
Mea
n R
espo
nse
time
(s)
FAIR
SRPT
Load =0.8
Mean Response Time vs. Size Percentile
![Page 41: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/41.jpg)
41
Transient Overload --Mean response time
SRPTFAIR
![Page 42: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/42.jpg)
42
Transient overloadResponse time as function of job
size
small jobswin big!
big jobsaren’t hurt!
FAIR
SRPT
![Page 43: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/43.jpg)
43
New project:Scheduling dynamic web requests
“$$$buy$$$”
“buy”
“buy”
Web Server(eg: Apache/Linux)
Internet
Internet
Database
Disks
Low-PriorityClient
Locks
High-PriorityClient
CPU(s)
Need to schedule the database ...
![Page 44: Mor Harchol-Balter Carnegie Mellon University Computer Science](https://reader034.vdocuments.us/reader034/viewer/2022051002/56815d46550346895dcb4c07/html5/thumbnails/44.jpg)
44
Conclusion
Misconceptionsabout
unfairness
Discrimination againsthigh-performancescheduling policies
Good news: Many high-performing policies are also fair in practice!
Classifying policies with respect to unfairnessis counter-intuitive.