![Page 1: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/1.jpg)
1
Mor Harchol-BalterCarnegie Mellon UniversitySchool of Computer Science
![Page 2: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/2.jpg)
2
“size” = service requirement
load < 1
jobs SRPT
jobs
jobs PS
FCFS
Q: Which minimizes mean response time?
![Page 3: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/3.jpg)
3
“size” = service requirement
jobs SRPT
jobs
load < 1
jobs PS
FCFS
Q: Which best represents scheduling in web servers ?
![Page 4: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/4.jpg)
4
IDEA
How about using SRPT instead of PS in web servers?
Linux 0.S.
WEBSERVER(Apache)
client 1
client 2
client 3
“Get File 1”
“Get File 2”
“Get File 3”
Internet
![Page 5: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/5.jpg)
5
Many servers receive mostly static web requests.
“GET FILE”
For static web requests, know file size
Approx. know service requirement of request.
Immediate Objections
1) Can’t assume known job size
2) But the big jobs will starve ...
![Page 6: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/6.jpg)
6
Outline of Talk
[BH – Sigmetrics 01] “Analysis of SRPT: Investigating Unfairness”[HSW-Performance 02] “Asymptotic Convergence of Scheduling Policies…”[WH – Sigmetrics 03*] “Classifying Scheduling Policies wrt Unfairness …”
THEORY
IMPLEMENT
www.cs.cmu.edu/~harchol/
[HSBA – TOCS 03] “Size-based Scheduling to Improve Web Performance”[SH – ITC 03*] “Web servers under overload: How scheduling can help”[MSAH – ICDE03] “Priority Mechanisms for OLTP and Web Applications”
(M/G/1)
Schroeder
Wierman
![Page 7: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/7.jpg)
7
THEORY SRPT has a long history ...
1966 Schrage & Miller derive M/G/1/SRPT response time:
1968 Schrage proves optimality
1979 Pechinkin & Solovyev & Yashkov generalize
1990 Schassberger derives distribution on queue length
BUT WHAT DOES IT ALL MEAN?
![Page 8: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/8.jpg)
8
THEORYSRPT has a long history (cont.)1990 - 97 7-year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN!
1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS!
Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs ...
Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”
![Page 9: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/9.jpg)
9
Unfairness Question
SRPT
PS?
?
Let =0.9. Let G: Bounded Pareto(= 1.1, max=1010)
Question: Which queue does biggest job prefer?
M/G/1
M/G/1
![Page 10: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/10.jpg)
10
Results on UnfairnessLet =0.9. Let G: Bounded Pareto(= 1.1, max=1010)
SRPT
PS
I SRPT
![Page 11: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/11.jpg)
11
Unfairness – General Distribution
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
![Page 12: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/12.jpg)
12
All-can-win-theorem:
For all distributions, if ½,
E[T(x)]SRPT E[T(x)]PS for all x.
Proof idea:
x
t
dt
0 )1 1x
)
x
xFx
2
2
))1(2
(
0x
dttft 2 )(
Waiting time (SRPT) Residence (SRPT) Total (PS)
![Page 13: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/13.jpg)
13
Classification of Scheduling Policies
ALWAYS FAIR For all loads, for all service distributions,
ALWAYS UNFAIR For all loads, for all service distributions,
SOMETIMES UNFAIR For some loads:
For other loads :
PSP xTExTEx )]([)]([ ,
PSP xxTExTE ,)]([)]([
PSP xTExTEx )]([)]([ ,
PSP xxTExTE ,)]([)]([
![Page 14: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/14.jpg)
14
Classification of Scheduling Policies
AlwaysFAIR
AlwaysUnfair
Sometimes Unfair
Age-BasedPolicies
Preemptive Size-basedPolicies
Remaining Size-basedPolicies
Non-preemptive
PS PLCFS
FB
PSJF
LRPT
FCFS
LJF
SJF
SRPT Lots of open problems…
FSP
![Page 15: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/15.jpg)
15
What does SRPT mean within a Web server?
• Many devices: Where to do the scheduling?
• No longer one job at a time.
IMPLEMENT From theory to practice:
![Page 16: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/16.jpg)
16
Server’s Performance BottleneckIMPLEMENT
5
Linux 0.S.
WEBSERVER(Apache)
client 1
client 2
client 3
“Get File 1”
“Get File 2”
“Get File 3”
Rest ofInternet ISP
Site buyslimited fractionof ISP’s bandwidth
We model bottleneck by limiting bandwidth on server’s uplink.
![Page 17: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/17.jpg)
17
Network/O.S. insides of traditional Web server
Sockets take turnsdraining --- FAIR = PS.
WebServer
Socket 1
Socket 3
Socket 2Network Card
Client1
Client3
Client2BOTTLENECK
IMPLEMENT
![Page 18: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/18.jpg)
18
Network/O.S. insides of our improved Web server
Socket corresponding to filewith smallest remaining datagets to feed first.
WebServer
Socket 1
Socket 3
Socket 2Network Card
Client1
Client3
Client2
priorityqueues.
1st
2nd
3rd
S
M
L
BOTTLENECK
IMPLEMENT
![Page 19: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/19.jpg)
19
Experimental Setup
Implementation SRPT-based scheduling: 1) Modifications to Linux O.S.: 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design.
Linux 0.S.
1
2
3
APACHEWEB
SERVER
Linux
12
3
200
Linux
123
200
Linux
12
3
200
switch
WA
N E
MU
WA
N E
MU
WA
N E
MU
![Page 20: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/20.jpg)
20
Experimental Setup
APACHEWEB
SERVER
Linux 0.S.
123
Linux
123
200
Linux
123
200
Linux
123
200
switch
WA
N E
MU
WA
N E
MU
WA
N E
MU
Trace-based workload: Number requests made: 1,000,000Size of file requested: 41B -- 2 MBDistribution of file sizes requested has HT property.
Flash
Apache
WAN EMU
Geographically-dispersed clients
10Mbps uplink
100Mbps uplink
Surge
Trace-based
Open system
Partly-open
Load < 1
Transient overload
+ Other effects: initial RTO; user abort/reload; persistent connections, etc.
![Page 21: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/21.jpg)
21
Preliminary Comments
• Job throughput, byte throughput, and bandwidth utilization were same under SRPT and FAIR scheduling.
• Same set of requests complete.
• No additional CPU overhead under SRPT scheduling. Network was bottleneck in all experiments.
APACHEWEB
SERVER
Linux 0.S.
123
Linux
123
200
Linux
123
200
Linux
123
200
switch
WA
N E
MU
WA
N E
MU
WA
N E
MU
![Page 22: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/22.jpg)
22
Load
FAIR
SRPTMea
n R
esp
onse
Tim
e (s
ec)
Results: Mean Response Time (LAN)
.
.
.
.
.
.
![Page 23: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/23.jpg)
23
Percentile of Request Size
Mea
n R
esp
onse
tim
e (
s)
FAIR
SRPT
Load =0.8
Mean Response Time vs. Size Percentile (LAN)
![Page 24: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/24.jpg)
24
Transient Overload
![Page 25: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/25.jpg)
25
Transient Overload - Baseline
Mean response time
SRPTFAIR
![Page 26: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/26.jpg)
26
Transient overloadResponse time as function of job
size
small jobswin big!
big jobsaren’t hurt!
FAIR
SRPT
WHY?
![Page 27: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/27.jpg)
27
Baseline Case
WAN propagation delays
WAN loss
Persistent Connections
Initial RTO value
SYN Cookies
User Abort/Reload
Packet Length
Realistic Scenario
WAN loss + delay
RTT: 0 – 150 ms
Loss: 0 – 15%
Loss: 0 – 15%RTT: 0 – 150 ms,
0 – 10 requests/conn.
RTO = 0.5 sec – 3 sec
ON/OFF
Abort after 3 – 15 sec, with 2,4,6,8 retries.
Packet length = 536 – 1500 Bytes
RTT = 100 ms; Loss = 5%; 5 requests/conn.,RTO = 3 sec; pkt len = 1500B; User abortsAfter 7 sec and retries up to 3 times.
FACTORS
![Page 28: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/28.jpg)
28
Transient Overload - Realistic
Mean response time
FAIR SRPT
![Page 29: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/29.jpg)
29
SRPT scheduling is a promising solution for reducing
mean response time seen by clients, particularly when the load at server bottleneck is high, or under transient overload conditions.
SRPT results in negligible or zero unfairness to large requests.
SRPT is easy to implement and efficient. No CPU overhead. No drop in throughput.
Results corroborated via implementation and analysis.
Conclusion so far …
![Page 30: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/30.jpg)
30
More questions …
STATIC web
requests
Everything so far in talk …
DYNAMICweb
requests
Current work…
SchroederMcWherterSchroeder
Wierman
![Page 31: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/31.jpg)
31
Online Shopping
Internet
client 1
client 2
client 3
“buy”
“buy”
“buy”
Web Server(eg: Apache/Linux)
Database(eg: DB2, Oracle, PostgreSQL)
• Dynamic responses take much longer – 10sec• Database is bottleneck.
![Page 32: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/32.jpg)
32
Online Shopping
Internet
client 1
client 2
client 3
“$$$buy$$$”
“buy”
“buy”
Web Server(eg: Apache/Linux)
Database(eg: DB2, Oracle, PostgreSQL)
Goal: Prioritize requests
![Page 33: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/33.jpg)
33
Isn’t “prioritizing requests” problem already solved?
Internet
“$$$buy$$$”
“buy”
“buy”
Web Server(eg: Apache/Linux)
Database(eg: DB2, Oracle, PostgreSQL)
No.Prior work mostly simulation or RTDBMS.
![Page 34: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/34.jpg)
34
Which resource to prioritize?“$$$buy$$$”
“buy”
“buy”
Web Server(eg: Apache/Linux)
Internet
Internet
Database
Disks LocksCPU(s)
High-Priority client Low-Priority client
![Page 35: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/35.jpg)
35
Q: Which resource to prioritize?“$$$buy$$$”
“buy”
“buy”
Web Server(eg: Apache/Linux)
Internet
Internet
Database
Disks LocksCPU(s)
High-Priority client Low-Priority client
A: 2PL Lock Queues
![Page 36: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/36.jpg)
36
What is bottleneck resource?
• IBM DB2 -- Lock waiting time (yellow) is bottleneck.• Therefore, need to schedule lock queues to have impact.
Fix at 10 warehouses #clients = 10 x #warehouses
![Page 37: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/37.jpg)
37
Why lock scheduling is hardLock resource 1
HHLL L
HHLL L
Lock resource 2
NP H may wait long time
NPinherit Speeding up L may hurt H in long run
Pabort Rollback cost + wasted work + really hurt L’s.
![Page 38: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/38.jpg)
38
Results:
Implementationstudy of
NP, NPinherit, Pabort
under TPC-C workload,Shore DBMS
Developnew policy
POW (Preempt on Wait)
![Page 39: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/39.jpg)
39
Results:
Think time
Responsetime
Pabort
NPinherit
Pabort
NPinherit
![Page 40: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/40.jpg)
40
Results:
Think time
Responsetime
Pabort
NPinherit
Pabort
NPinherit POW:
Best of both
![Page 41: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/41.jpg)
41
More work in SYNC project…
DBMS(eg: DB2, Oracle)QoSInternet
“$$$buy$$$”
“buy”
“buy”
Web Server
QoS from outside the box
Scheduling the TeraGrid
Time-varying load in systems
Impact of closed versus open system models
PSCSDSC
NCSA
![Page 42: Mor Harchol-Balter Carnegie Mellon University School of Computer Science](https://reader036.vdocuments.us/reader036/viewer/2022062301/568149e3550346895db70b10/html5/thumbnails/42.jpg)
42
Scheduling is a very cheap solution… No need to buy new hardware No need to buy more memory Small software modifications
…with a potentially very big win in some situations.
Conclusion
Thank you!