a case study of web server benchmarking using parallel wan emulation carey williamson rob simmonds...
TRANSCRIPT
![Page 1: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/1.jpg)
A Case Study ofA Case Study ofWeb Server Web Server
Benchmarking UsingBenchmarking UsingParallel WAN Emulation Parallel WAN Emulation
Carey WilliamsonRob Simmonds Martin Arlitt
University of Calgary
![Page 2: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/2.jpg)
Web Benchmarking with IP-TNE 2
Network Emulation A hybrid performance evaluation
methodology that combines aspects of implementation with simulation modeling
A network emulator is a network simulator with an interface that allows client applications to interact with it in real-time
“A simulator that talks back” (IP packets) Why? Provides a reliable, repeatable test
environment for distributed applications Internet games, video conferencing, Web, ...
![Page 3: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/3.jpg)
SimulationSimulation
Real WorldReal World
![Page 4: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/4.jpg)
Web Benchmarking with IP-TNE 4
IP-TNE The Internet Protocol Traffic and Internet Protocol Traffic and
Network EmulatorNetwork Emulator (IP-TNE) is a network emulator based on IP-TN (packet-level IP network simulator)
Enables interaction between IP based clients via an IP-TN simulated network (in real time!) Distributed applications can interact
with IP-TNE without modification
![Page 5: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/5.jpg)
Web Benchmarking with IP-TNE 5
IP-TNE Overview
![Page 6: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/6.jpg)
Web Benchmarking with IP-TNE 6
IP-TNE Overview (cont’d) CCTKit with real-time extensions
provides an environment for fast network emulation (PDES)
IP-TNE provides routing methods suitable for shared environments and dedicated test environments
Now has HTTP and TCP client models that can be used for Web server benchmarking
![Page 7: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/7.jpg)
Web Benchmarking with IP-TNE 7
So What? Flexible routing model support High-performance packet reading and
writing via raw sockets (1 Gbps) Can model an arbitrary IP internetwork Detailed IP protocol models
IPv4, ICMP, ping, traceroute, pchar, MTU, ... Supports parallel execution on
shared memory multiprocessors “Blazingly fast!” - CLW, 2002
![Page 8: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/8.jpg)
Web Benchmarking with IP-TNE 8
Example: Web Benchmarking
Web Server
Client 1
Client 2
Client 3
Client C
...
![Page 9: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/9.jpg)
Web Benchmarking with IP-TNE 9
WAN Emulation (1 of 3)
Web Server
Client 1
Client 2
Client 3
Client C
...
“Centralized” Approach
![Page 10: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/10.jpg)
Web Benchmarking with IP-TNE 10
WAN Emulation (2 of 3)
Web Server
Client 1
Client 2
Client 3
Client C
...
“Shim” Approach(NISTnet, DummyNet, WASP)
![Page 11: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/11.jpg)
Web Benchmarking with IP-TNE 11
WAN Emulation (3 of 3)
Web Server
Client 1
Client 2
Client 3
Client C
...
Our IP-TNE Approach
![Page 12: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/12.jpg)
Web Benchmarking with IP-TNE 12
Objectives of Case Study Evaluate new approach to WAN
emulation, and demonstrate feasibility
Confirm prior results by Nahum et al. on effects of WAN conditions on Web server performance
How fast can Apache Web server go? How fast can IP-TNE go?
![Page 13: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/13.jpg)
Web Benchmarking with IP-TNE 13
Experimental Setup
IP-TNE on Compaq ES-40 (4 CPU) Apache (1.3.23) on another ES-40 Gigabit Ethernet (1 Gbps) in
between OS is Compaq Tru64 (v5.1A)
ANML for defining network model e.g., simple regular WAN topology
![Page 14: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/14.jpg)
Web Benchmarking with IP-TNE 14
![Page 15: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/15.jpg)
Web Benchmarking with IP-TNE 15
![Page 16: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/16.jpg)
Web Benchmarking with IP-TNE 16
Web Workload Model Static content only Closed-loop workload generator Fixed-size Web objects
Small (1 KB) Large (64 KB)
Variable-size Web objects Median 3 KB Mean 9 KB Pareto heavy tail (alpha = 1.2) Zipf-like document popularity profile
![Page 17: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/17.jpg)
Web Benchmarking with IP-TNE 17
Performance Metrics
Two primary metrics HTTP transaction rate (trans/sec) Network throughput (Mbps)
Several secondary metrics Response time Connection failure rate Packet loss rate ...
![Page 18: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/18.jpg)
Web Benchmarking with IP-TNE 18
Results (Fig. 4a) For 1 KB transfers with HTTP/1.0:
Single client: 170 transactions/sec Transaction rate scales up with number of
clients up to about H = 32 Transaction rate flattens, then drops sharply
as num clients is increased more (closed loop)
Peak rate achieved: 3800 trans/sec Peak throughput approximately 40 Mbps Transaction rate is (strongly) inversely
related to the client round trip time (RTT)
![Page 19: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/19.jpg)
Web Benchmarking with IP-TNE 19
![Page 20: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/20.jpg)
Web Benchmarking with IP-TNE 20
![Page 21: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/21.jpg)
Web Benchmarking with IP-TNE 21
![Page 22: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/22.jpg)
Web Benchmarking with IP-TNE 22
![Page 23: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/23.jpg)
Web Benchmarking with IP-TNE 23
![Page 24: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/24.jpg)
Web Benchmarking with IP-TNE 24
Results (Fig. 4b) For 64 KB transfers with HTTP/1.0:
Single client: 18 transactions/sec Transaction rate scales up with number of
clients up to about H = 32 Transaction rate flattens, then drops slightly
as num clients is increased more Peak rate achieved: 220 trans/sec Peak throughput approximately 115 Mbps Transaction rate is (weakly) inversely
related to the client round trip time (RTT)
![Page 25: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/25.jpg)
Web Benchmarking with IP-TNE 25
![Page 26: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/26.jpg)
Web Benchmarking with IP-TNE 26
Results (Fig. 4c) For variable-size transfers with HTTP/1.0:
Single client: 60 transactions/sec Transaction rate scales up with number of
clients up to about H = 32 Transaction rate flattens, then drops
as num clients is increased more Peak rate achieved: 1300 trans/sec Peak throughput approximately 90 Mbps Transaction rate is inversely related to the
client round trip time (RTT) Behaviour is in between 1 KB and 64 KB results
![Page 27: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/27.jpg)
Web Benchmarking with IP-TNE 27
![Page 28: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/28.jpg)
Web Benchmarking with IP-TNE 28
Results (Fig. 5a)
Concurrent connections with HTTP/1.0: Single client: 600 transactions/sec Qualitatively similar results to before,
except that fewer clients are needed to drive the server to full load
Conceptually concurrent connections are no different than adding more clients
![Page 29: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/29.jpg)
Web Benchmarking with IP-TNE 29
![Page 30: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/30.jpg)
Web Benchmarking with IP-TNE 30
Results (Fig. 5b) Persistent connections with
HTTP/1.1: Single client: 300 transactions/sec Qualitatively similar results to before,
except that transaction rate is about 70% higher than for HTTP/1.0 (since multiple HTTP req’s per TCP conn)
Peak transaction rate 6500 trans/sec Much less dependency on RTT effects
![Page 31: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/31.jpg)
Web Benchmarking with IP-TNE 31
![Page 32: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/32.jpg)
Web Benchmarking with IP-TNE 32
Results (Fig. 5c)
Pipelined persistent connections with HTTP/1.1: Single client: 800 transactions/sec Qualitatively similar results to before,
except that transaction rate is about 100% higher than for HTTP/1.0
Peak transaction rate 7600 trans/sec Much less dependency on RTT effects
![Page 33: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/33.jpg)
Web Benchmarking with IP-TNE 33
![Page 34: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/34.jpg)
Web Benchmarking with IP-TNE 34
Results (Fig. 6a)
Effect of WAN RTT delays: Increasing the per-link propagation
delay increases the client RTT delay, which in turn reduces the transaction rate and throughput (as expected)
As RTT increases, more and more clients are needed in order to drive the Web server to full load
Similar to [Nahum et al. 2001]
![Page 35: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/35.jpg)
Web Benchmarking with IP-TNE 35
![Page 36: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/36.jpg)
Web Benchmarking with IP-TNE 36
Results (Fig. 6b) Effect of bandwidth asymmetry:
For asymmetric access technologies such as ADSL (Asymmetric Digital Subscriber Line), the upstream link from the client to the server can sometimes be the bottleneck for TCP, even though it is primarily carrying ACKs only
Depends on normalized bandwidth ratio Greater asymmetry, worse performance
![Page 37: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/37.jpg)
Web Benchmarking with IP-TNE 37
![Page 38: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/38.jpg)
Web Benchmarking with IP-TNE 38
Results (Fig. 6c)
Effect of WAN packet losses: Decreasing the router queue size at
the bottleneck link increases the packet loss ratio (as expected)
As the level of packet loss increases, the HTTP transaction rate and the network throughput decrease
Similar results to [Nahum et al. 2001]
![Page 39: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/39.jpg)
Web Benchmarking with IP-TNE 39
![Page 40: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/40.jpg)
Web Benchmarking with IP-TNE 41
Summary and Conclusions The IP-TNE is a useful tool for Web
server benchmarking Demonstrates feasibility of WAN
emulation using a single computer Confirms prior results by Nahum et
al. studying the effects of WAN conditions on Web server performance
Demonstrates performance advantages of HTTP/1.1
![Page 41: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/41.jpg)
Web Benchmarking with IP-TNE 42
Future Work? Increase traffic to your Web site!!! Guarantee 20% increase in traffic! 1000’s of new clients per month!!!!!
Send $$$ to [email protected]
![Page 42: A Case Study of Web Server Benchmarking Using Parallel WAN Emulation Carey Williamson Rob Simmonds Martin Arlitt University of Calgary](https://reader035.vdocuments.us/reader035/viewer/2022062620/551b3807550346cf5a8b64a2/html5/thumbnails/42.jpg)
Web Benchmarking with IP-TNE 43
Future Work with IP-TNE Validation of IP-TNE (and IP-TN) Benchmarking IP-TNE vs IP-TNE Benchmarking Web caching appliances Evaluating SRPT scheduling in WAN setting Connection/packet-level scheduling algorithms Evaluating CATNIP approach to TCP/IP Evaluating portable (wireless) Web servers Workload sensitivities (Zipf, Pareto, corr,
mods) Experiments with dynamic content (CGI, etc) Asymmetric networks, Ensemble-TCP Parallel TCP connections: friend or foe? Evaluating effect of TCP SACK in WAN