1 web server performance in a wan environment vincent w. freeh computer science north carolina state...
TRANSCRIPT
![Page 1: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/1.jpg)
1
Web Server Performance in aWAN Environment
Vincent W. FreehComputer Science
North Carolina State
Vsevolod V. PanteleenkoComputer Science & Engineering
University of Notre Dame
![Page 2: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/2.jpg)
2
Large web site
Complex design and interaction
Multiple tiers Appliance Web, app, & DB servers
Study performance of web server Cached pages
Most testing Simulated load LAN environment
Our evaluation adds Simulated WAN environment Small MTU, BW limits, latency Shows some optimization
aren’t
Appliance Web Servers Application Servers
Database Servers
Clients
![Page 3: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/3.jpg)
3
Evaluating a web server
Three parts Measuring the server Loading the server Supporting the server
Appliance Web Servers Application Servers
Database Servers
Clients
Net
Server load
Server demand
Tiers2&3
![Page 4: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/4.jpg)
4
Two ways to load server
Synthetic load Controlled Reproducible Flexible Only as good as assumptions, mechanisms Hard to replicate real world
Real-world load Uncontrolled Not reproducible (can use traces) Accurate model of system Hard to produce extreme or rare conditions
Discussion Need both Validate simulations with real-world tests
Net
![Page 5: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/5.jpg)
5
Loading the server
Our tests use synthetic load Three load-generating tools Micro-benchmarking tool
Requests a single object at a constant rate Tests delivery of static, cached documents Establishes base line
Net
![Page 6: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/6.jpg)
6
Modified SURGE
SURGE Scalable URL reference generator Barford & Crovella, U Boston Emulates statistical distribution
Object & request sizeObject popularityEmbedded object referencesTemporal localityUse idle periods
Modifications Converted from process based to event based
To increase number of clients Server-throttling problem eliminated
Net
![Page 7: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/7.jpg)
7
Delays and limits
Emulate WAN parameters in a LAN Network delays Bandwidth limits
Modified kernel and protocol stack Separate delay queue per TCP connection Necessary for accurate emulation More accurate than Dummynet & NISTnet (per interface)
Net
![Page 8: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/8.jpg)
8
Measuring a web server
OS
Network
HTTP
request reply
drivers
TCP/IP
Apache, TUX
![Page 9: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/9.jpg)
9
Measuring a web server
OS
Network
HTTP
request reply
Measure utilization usingHW performance counters
![Page 10: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/10.jpg)
10
Test environment
OS: Linux 2.4.8 Node: (server & clients)
Pentium III, 650MHz 512MB main memory
NIC: 3COM 3C590 100 Mbps ethernet Direct connect
Software: Client: microbenchmarking, SURGE, delay/limits Server: Apache, Tux
Warmed client No cache misses
Client
Client
Server
NIC
NIC
NIC
NIC
![Page 11: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/11.jpg)
11
Cost breakdown – file size, Apache
Majority of time in interrupt (recv’g)But most data is sent.
•MTU = 536 bytes•Delay = 200 ms•BW = 56 Kbps•Data send rate = 3MB/s
![Page 12: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/12.jpg)
12
Cost breakdown - file size, TUX
•Twice data send rate as Apache.•Essentially all cost in interrupts.
•MTU = 536 bytes•Delay = 200 ms•BW = 56 Kbps•Data send rate = 6 MB/s
![Page 13: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/13.jpg)
13
Apache versus TUX
Apache TUX
Server send rate 3.0 MB/s 6.0 MB/s
Packets rec’d / s 5738 11,991
Packets sent / s 6156 11,878
Interrupts / s 7482 13,974
Concurrent connections 784 1451
![Page 14: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/14.jpg)
14
Cost breakdown vs. MTU
Surge parameters•Size = 10 KB•Delay = 200 ms•BW = 56 Kbps•Data send rate = 6 MB/s
![Page 15: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/15.jpg)
15
Effects of network delay
Surge parameters•MTU = 536 bytes•Size = 10 KB•BW = 56 Kbps•Data send rate = 6 MB/s
![Page 16: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/16.jpg)
16
Effects of bandwidth limits
Surge parameters•MTU = 536 bytes•Size = 10 KB•Delay = 200 ms•Data send rate = 6 MB/s
20% decrease in overhead from 28kbps to infinity
![Page 17: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/17.jpg)
17
Persistent connections
Surge parameters•MTU = 536 bytes•Size = 10 KB•Delay = 200 ms•Size = 10 KB•Data send rate = 6 MB/s
10% decrease going from 1 to 16 requests per connection
![Page 18: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/18.jpg)
18
Copy and checksumming
0
0.1
0.2
0.3
0.4
0.5
0.6
zero copy & HW
checksumming
copy & checksum
kernel mode
socket write
soft intr.
hard intr.
Surge parameters•MTU = 536 bytes•Size = 10 KB•Delay = 200 ms•Size = 10 KB•Data send rate = 6 MB/s
![Page 19: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/19.jpg)
19
Re-assess value of some optimizations
Copy & checksumming avoidance LAN: 25-111% copy or 21-33% copy & 10-15% checksum WAN: 10% combined
Select optimization LAN: 28% WAN: < 10%
Connection open/close avoidance (HTTP 1.1) LAN: “greatly”, “significantly” WAN: < 10%
![Page 20: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/20.jpg)
20
Conclusion
Most processing in protocol stack and drivers Small MTU size increases processing cost Little effect from
Network delay Bandwidth limitations Persistent connections
End-user request latency depends Primarily on connection bandwidth Secondarily on network delay
Future Dynamic & uncached pages Add packet loss
Work supported by IBM UPP & NSF CCR9876073
www.csc.ncsu.edu/faculty/freeh/
![Page 21: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/21.jpg)
21
End
![Page 22: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/22.jpg)
22
Persistent connections - packets/s
![Page 23: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/23.jpg)
23
Number of Packets vs. MTU
![Page 24: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/24.jpg)
24
Web (HTTP) servers
Apache Largest install base User space Process-based model
TUX Niche server Kernel space Event-based model Aggressive
optimizations Copy/checksum
avoidance Object, name caching
![Page 25: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/25.jpg)
25
Measuring a web server
OS
Network
HTTP
request reply
![Page 26: 1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering](https://reader035.vdocuments.us/reader035/viewer/2022081514/56649d1f5503460f949f3136/html5/thumbnails/26.jpg)
26
Interrupt coalescing
0
0.1
0.2
0.3
0.4
0.5
0.6
Apache:normal
Apache: intr.coalescing
TUX: normal TUX: intr.coalescing
CP
U u
tiliz
atio
n
user modekernel mode
socket w ritesoft. intr.hard. intr.
Decreases interrupt scheduling overhead Interrupt every 2 ms