jaws(web server framework using patterns)

JAWS(A FRAMEWORK FOR HIGH-PERFORMANCE WEB SERVERS)

TP VERSION 1.1

JAWS(A FRAMEWORK FOR HIGH-PERFORMANCE WEB SERVERS)

TP VERSION 1.1

YoungSu, [email protected]

Microsoft MVPDevpia Architecture Sysop

EvaCast Study Leader (http://www.EvaCast.net)

YoungSu, [email protected]

Microsoft MVPDevpia Architecture Sysop

EvaCast Study Leader (http://www.EvaCast.net)

What’s the What’s the

Good Web Good Web Server?Server?

9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 22

ArchitectArchitect

Concurrency

Thread-per-Session

I/O

Synchronous

Cache

LRU

Protocol

&

Filter

Read Request

Parse Request


Thread-per Request

Thread Pool

Reactvie

Asynchronous

LFU

Structured Cache

Hinted Cache

Parse Request

Parse Header

Perform Request

Log Request

Seed Paper

� James C. Hu, Douglas Schmidt,

JAWS: A Framework for High-Performance Web Servers,

Adobe Acrobat 7.0 Document

Domain-Specific Application Frameworks: Frameworks Experience By Industry,

John Wiley & Sons, October, 1999.


Agenda

� Motivation

� Applying Patterns & Frameworks to Web Server� Common Pitfalls of Developing Web Server

� Overcoming Web Server Pitfalls

� Relationship Between Frameworks, Patterns & Other Reuse Technique

� The JAWS Web Server� The JAWS Web Server� Overview of the JAWS Framework

� Overview of the Design Pattern in JAWS

� Concurrency Strategy

� I/O Strategy

� Protocol Pipeline Strategy

� File Caching Strategy

� Web Server Benchmarking

� Concluding Remarks9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 66

� Common Pitfalls of Developing Web Server Software

� Overcoming Web Server Pitfalls � Overcoming Web Server Pitfalls

with Patterns and Frameworks

� Relationship between

Frameworks, Patterns, and Other Reuse Techniques


2.1 Common Pitfalls

of Developing Web Server SW

� Excessive low-level detail

� Continuous rediscovery & reinvention of

incompatible higher-level programming abstraction.

� High potential for errors� High potential for errors

� Lack of portability

� Steep learning curve

� Inability to handle increasing complexity


2.2 Overcoming Web Server Pitfalls

with Patterns & Frameworks

� Limitations of Class Library

� do not capture

the control flow and collaboration

among families of related SW components.

� Class Library reuse often

re-invent and re-implement

the overall Software architecture

for each new Application.


2.2 Overcoming Web Server Pitfalls

with Patterns & Frameworks

� The Benefits of Patterns

� Expert’s Design Experiences

� Good Methods to use Framework

� The infrastructure of Framework

� The Benefits of Frameworks

� Expert’s intellectual products

� Help developers avoid costly re-invention


2.3 Relationship Between Frameworks, Patterns,

and Other Reuse Techniques

Class Libraries vs Framework

App SpecificApp SpecificLogicLogic

DATABASEDATABASE ADTsADTs

MATHMATH NETWORKINGNETWORKINGInvocationsInvocations

Class LibraryClass Library

ReactorReactor

NETWORKINGNETWORKING

Active ObjectActive Object

GUIGUI

StateState

MATHMATH


OO DesignOO Design

EventEventLoopLoop

GRAPHICSGRAPHICS GUIGUI

SingletonSingleton StrategyStrategy

ReactorReactor AdapterAdapter

StateState Active ObjectActive Object

Class Library Class Library Component ArchitectureComponent Architecture

SelectionsSelections

Design PatternDesign Pattern

DATABASEDATABASE

SingletonSingleton

GRAPHICSGRAPHICS

AdapterAdapter

EventEventLoopLoop

App SpecificApp SpecificLogicLogic

ADTsADTs

CallbacksCallbacks

InvocationsInvocations

Application FrameworkApplication FrameworkComponent ArchitectureComponent Architecture

2.3 Relationship Between Frameworks, Patterns,

and Other Reuse Techniques

� Frameworks define “semi-complete” application

that embody domain-specific object structures and functionality.� Class Libraries don’t offer explicit guidance to system design

� Components in a Framework manage

the canonical flow of control within App

� Frameworks are active and exhibit “IoC (Inversion of Control)” at runtime.� Template Method Pattern (Hollywood Principle)

� In Practice, Frameworks, Class Libraries,

and Components are complementary technologies.


� Overview of the JAWS Framework,

� Overview Design Pattern in JAWS.

� 4 Core Components in JAWS� Concurrency.� Concurrency.

� I/O.

� Protocol/Pipeline.

� File Caching.

� JAWS Framework Revisited


3.1 Overview of the JAWS Framework


3.2 Overview Design Pattern in JAWS

� Strategic Pattern� Message Composition/Decomposition

� Pipe & Filter , Layer, Composite Message

� Dispatcher� Acceptor –Connector / Reactor / Proactor

� Active Object

� Service Configurator

� Tactical Pattern� Strategy

� Adapter

� State

� Singleton


Strategic Pattern Overview

DataData

LayerLayer

DecompositeDecomposite

MessageMessage

DispatchingDispatching

LayerLayer

Composite MessageComposite Message

MessagesMessages

UnMarshalingUnMarshaling

MarshalingMarshaling

MessageMessage

3.2.1 Strategic Pattern

� Marshaling / UnMarshaling

� Composite Message / Layer / Pipe & Filter

� Dispatcher � Dispatcher

� Acceptor-Connector / Reactor / Proactor

� Active Object

� Service Configurator


Composite Message


Acceptor/Connector Pattern


Acceptor Pattern

� Acceptor is Factory.

� Creates, Accepts and Activates a new Protocol Handler

Whenever the Event Dispatcher notifies Acceptor

that a connection has arrived from a Client.

Protocol Handler

Protocol Handler


Event Dispatcher Concurrency

TaskAcceptor

peer_acceptor_accept()

Protocol Handler

peer_stream_open()

Reactor Pattern

Reactor

handle_events()register_handler(in handle)remove_handler(in handle)

Event Handler

handle_event(in type)get_handle()

handlehandlesetset

ownsowns

dispatchdispatch


SynchronousEvent

Demultiplexer

select()

Concrete Event Handler


Concrete Event Handler


Handle<<uses>><<uses>>

notifiesnotifies

ownsowns

Reactor Pattern� decouples the synch demultiplexing and dispatching logic.

� the Single-Thread Web Server Concurrency model

InputOutput Handler Protocol PipelineInputOutput

Reactive IO Filecache Handle


Synchronous IO Reactor

send_file()recv_file()send_data()recv_data()

Asynchronous IO

Initiation Dispatcher

handle_event()register_handler(h)remove_handler(h)

Timer_Queue

schdule_timer(h)cancel_timer(h)expire_timers(h)

Reactive IO Handler

handle_input()handle_output()

IO Handle

Proactor Pattern

Handle

<<uses>><<uses>>

is associated withis associated with

AsynchronousOperation Processor

AsynchronousOperation

Initiator

Completion Handler

handle_event()

<<uses>><<uses>> <<invokes>><<invokes>>

<<uses>><<uses>>


Proactor

handle_events()

AsynchronousEvent Demultiplexer

get_completion_event()

Concrete Completion Handler

**

<<enqueues>><<enqueues>>

CompletionEvent Queue

execute_async_operation() async_operation()

handle_event()

<<executes>><<executes>>

<<dequeues>><<dequeues>>

<<demultiplexes & <<demultiplexes & dispatches>>dispatches>>

Proactor Pattern (IOCP)

Socket ASocket A

Socket BSocket B Completion PortCompletion Port

2.1 Create Completion Port2.1 Create Completion Port

5. 1 Send 5. 1 Send Completion Socket InfoCompletion Socket Info

3. Create Thread3. Create Thread

7. Thread Returns Thread Pool7. Thread Returns Thread Pool


Socket BSocket B

Socket CSocket C

Completion PortCompletion Port

2.2 Create 2.2 Create Completion QueueCompletion Queue

ThreadThread

ThreadThread

ThreadThread

1. Create Socket1. Create Socket

4. Link 4. Link Socket and PortSocket and Port 5. 2 Send 5. 2 Send

Completion PacketCompletion Packet

Thread PoolThread Pool

6. Process Completion Events6. Process Completion Events

Proactor Pattern� decouples the Asynch demultiplexing and dispatching logic.

� The Asynch variant of the Thread Pool

� Use IOCP (IO Completion Port)

InputOutput Handler Protocol PipelineInputOutput

Asynchronous IO Filecache Handle


Synchronous IO Proactor

send_file()recv_file()send_data()recv_data()

Reactive IO

Asynchronous IO

Completion Dispatcher

handle_event()register_handler(h)remove_handler(h)

Timer_Queue

schdule_timer(h)cancel_timer(h)expire_timers(h)

Filecache Handle

Proactive IO Handler

handle_read_file()handle_write_file()

IO Handle

Asynch Op

open()cancel()

Active Object Pattern

Scheduler

Dispatch() Insert()

Proxy

method_1()method_n()

ActivationQueue

Insert()Remove()

createscreates createscreates maintainsmaintains

invokeinvoke

11 1111 11


Future MethodRequest

Can_run()Call()

**

Servant

method_1()method_n()

createscreates createscreates maintainsmaintains

ConcreteMethodRequest 1

ConcreteMethodRequest N

Client

write towrite to

obtain result fromobtain result from

Active Object Pattern� decouples method invocation from method execution.

� Thread-per-Request, Thread Pool, Thread-per-Session Model

use Active Object pattern.

Protocol Handler

http_request()ftp_request()

delegates

delegates

for(;;){

m = aq->remove();

dispatch(m); }aq->insert(http)


Scheduler

http_request()ftp_request()snmp_request()dispatch()

Activation Queue

insert()remove()

Method Object

ftp_request()snmp_request()

Resource Representation

Protocol Pipeline

delegates

delegates

Service Configurator Pattern

Component

+Init()+Fini()+Suspend()+Resume()

ComponentRepository

Service


+Resume()+Info()

CoreComponent

CoreComponent

ServiceConfigurator

Service Configurator Pattern

� Provides System Configuration at Runtime.

� Dynamically optimize, control and reconfigure

the behavior of Web server strategies a installation-time or during run-time.

ServiceFilecache

Protocol Handler

Service Repository


+init()+fini()+suspend()+resume()+info()

Protocol Handler

Filter Repository

Cache Strategy Repository

Protocol Pipeline DLL Cache Strategy

Read Request Filter LRU Strategy LFU Strategy

Parse Request Log Request

3.2.2 Tactical Patterns

� Strategy� Configure Cache replacement strategies

without afffecting the core SW Architecture.

� Adapter� Uniformly encapsulate the operation

of synch, asynch and reactive I/O.

� State� Event Dispatcher use State

to seamlessly support different concurrency strategies

and both synch and asynch.

� Singleton� To provide a global point to access object.

� Exist one copy of Cached Virtual File System.


Review


3.3 4 Components in JAWS

Concurrency

Thread-per-Session

I/O

Synchronous

Cache

LRU

Protocol

&

Filter

Read Request

Parse Request


Thread-per Request

Thread Pool

Reactvie

Asynchronous

LFU

Structured Cache

Hinted Cache

Parse Request

Parse Header

Perform Request

Log Request

3.3 Concurrency Strategy

� Design Challenge

� A large portion of non-I/O related Web server overhead is due to

the Web server’s concurrency strategy.

� Key Overhead� Key Overhead

� Synchronization

� Thread/Process Creation

� Context Switching

� Choosing an efficient concurrency strategy

is crucial to achieve high-performance


3.3 Concurrency Strategy

� Alternative Solution Strategies

� Thread-per-Request

� Thread-per-Session

� Thread Pool� Thread Pool

� Single-Thread

� Experiments

� JAWS Concurrency Strategy Framework


3.3.1 Alternative Solution Strategies

� Concurreny Strategy of Web Servers

� Single Thread

� Roxen

� Process-BasedDynamic FactorsDynamic Factors

Machine LoadMachine Load

# of simultaneous requests# of simultaneous requests Memory UseMemory Use

Memory WorkloadMemory WorkloadProcess-Based

� Apach

� Zeus

� MultiThread

� PHTTD

� JAWS


Static FactorsStatic Factors

H/W ConfigurationH/W Configuration OS PlatformOS Platform

Web Server Use CaseWeb Server Use Case

Machine LoadMachine Load Memory WorkloadMemory Workload

Thread-per-Request

� Allows each client request to run concurrently in a separate thread.

� Each Request arrives,

a new thread is crated to process request.

� This design appropriate synch I/O mechanismsThis design appropriate synch I/O mechanisms



Task

Protocol Handler

Protocol Handler

Acceptor

Protocol Handler

Protocol Handler

Thread-per-Request (cont’d)

� Advantage� Simplicity

� Exploit parallelism on multi-processor platform

� LimitationThe # of running threads may grow without bound, exhausting� The # of running threads may grow without bound, exhaustingavailable memory and CPU resources

� Compatible Fields� Use Light loaded servers with low latency

� Don’t use time consuming task based system


Thread-per-Session

� Allows each client connection to run concurrently

� All of these requests are submitted through a connection

between each client and a separate thread in the web server process

� Advantage� Advantage

� Does not spawn a separate thread for each request.

� Limitation

� This model still vulnerable to unbounded resource consumption as the # of clients grows.


Thread-per-Session (cont’d)

� Suited for HTTP 1.1 but not HTTP 1.0

� HTTP 1.0� support open/operation/close

� One URL Fetch per Connection� Needs Reconnection

� Transporting Data Size Limitation� Transporting Data Size Limitation

� URL Size Limitation

� HTTP 1.1� Persistent Connection

� Pipeline Functionality � Many Requests -> Support Serial Response

� Compressed Data Transporting

� Use Proxy Server and Cache


Thread Pool

� Allows up to N requests to execute concurrently within a

pool of threads.

� Support prespawning of Thread Creation



Task

Protocol HandlerProtocol Handler

Acceptor Protocol Handler

Protocol Handler

Thread Pool (cont’d)

� Advantage

� Lower overhead that Thread-per-Request.

� Thread Prespawning

� thread creation cost is amortized

� Pool Size is Fixed

� Resource Consume is bounded

� Limitation

� Small Pool Size

� New incoming requests be dropped or wait indefinitely.

� Large Pool Size

� Resource Consumption may be no better than using Thread-Per-Request


Single-Threaded

� All connections and requests

are handled by the same thread of control.

� Single-Threaded servers process requests iteratively.

� Compatible Filed

� Asynch or Reactive I/O Multiple Request

� Inadequate for high volume server

� Subsequent requests are blocked until they are reached


Experiments

� Static adaptivity

� Select the concurrency strategy

that best meets

The static requirements

of the system.

� Dynamic adaptivity

� Adapt dynamically

to current server condition

� multi-threaded concurrency

� More suited to

A multiple-processor machinethan

a uni-processor machine

� In order to accommodate Unanticipated

load

� Support to increase

the # of available threads in the Pool


JAWS Concurrency Strategy

FrameworkEvent Dispatcher

dispatch()

Concurrncy

Task

Protocol Handler

open()

Acceptor

accept()

server->dispatch()

serverserver

tasktask

acceptoracceptor

handlerhandler


Concurrncy

dispatch()

accept()activiate()enque()deque()svc()

Thread Pool

dispatch()

Thread-per-Connection

dispatch()

task->accept()

task->activate()

task->enque()

acceptor->accept()

3.4 I/O Strategy� Design Challenge

� is to devise efficient data retrieval and delivery strategies,

collectively referred to as I/O.

� Key Overhead

� Arranging Multiple I/O operation.

� Example

� Synch

� Web Transaction involving monetary fund transfer

� ASynch

� Web access to static information (Web page – ASP..)


3.4 I/O Strategy


� The Synchronous I/O Strategy

� The Reactive I/O Strategy

� The Asynchronous I/O Strategy� The Asynchronous I/O Strategy

� Experiments

� JAWS I/O Strategy Framework


The Synch I/O Strategy

� Synch I/O describes the model of I/O interaction between a Web server process and kernel.

� Kernel doesn’t return the thread of control to the server until the requested I/O operation either complete , completes partially, or fails.

� Advantage (Easy Use)

� Disadvantage (Resource Exhaust)� If combined with a single threaded concurrency strategy,

It is not possible to perform multiple synch I/O operation simultaneously.

� When using multiple thread,

It is still possible for an I/O request to block indefinitely.


The Reactive I/O Strategy

� Reactive I/O alleviates the blocking problems

of synchronous I/O without resorting to polling.

� Web Server uses

an OS event demultiplexing system call.

� select() in UNIX

� WaitForMultipleObjects() in Win32

� Refers Reactor Pattern

� Disadvantage

� May not make effective use of multiple CPU.


The Asynch I/O Strategy

� Asynch I/O simplifies

the de-multiplexing of multiple events

in one or more thread of control

without blocking the Web server.

� Advantage� The Web Server not block on I/O request.

� Disadvantage� Asynch I/O is not available on many OS Platform.

� more complicated than writing synch program.


Experiments

� The Result reveal that

each I/O strategy behaves differently

under different load condition.

� JAWS I/O Strategy Framework

addresses this issue

by allowing the I/O Strategy

to adapt dynamically

to run-time server condition.


JAWS I/O Strategy Framework

Perform Request

svc()

fhfhInputOutput Handler

receive_file()receive_data()send_file()send_data()

InputOutput


Filecache Handle

iohioh

ioio

ioioio->send_file(“index.html”)


complete()error()

send_data()

Synch IO


Asynch IO


Proactor

Reactive IO


Reactor

write(fh)Proactor->

send_file(fh)

Reactor->

send_file(fh)

3.5 Protocol Pipeline Strategies

� Design Challenge� Early Web Server

� Simply retrieved the requested file.

� Transferred its contents to the request.

� Modern Web Server� Modern Web Server� Support Various File Format� text, image, audio, video.

� Introducing Server Side Engine (ASP, JSP..)� search engine, map generation, interfacing database,

security, financial transaction.


3.5 Protocol Pipeline Strategy


� Pipeline.

� Reflection + Component Configurator.

� Examples� Examples

� Advanced Search Engine.

� Image Servers.

� Adaptive Web Contents.

� JAWS Protocol Pipeline Strategy Framework


Using PipelineHTTP/ 1.0 200 OK

Date: Thu, 09 Oct 01:26:00 GMT

Server: Eva/1.0

Last-Modified: Wed, 08 Oct 1997 ..

Content-Length : 12345

Content-Type : text/html

<HTML>

<TITLE> Homepage </TITLE>

..

</HTML>

CONNECTION

USER_AGENT

HOST

ACCEPT

ACCEPT

ACCEPT

Keep-Alive

Mothra/0.1

any.net

image/gif

image/jpeg

*/*


PerformPerform

RequestRequest

LogLog

RequestRequest

any.net - -

[09/Oct/1997:01:26:00 -0500]

“GET /~jxh/home.html HTTP/1.0

ReadRead

RequestRequest

GET /~jxh/hoem.html HTTP/1.0

Connection: Keep-Alive

User-Agent: Mothra/0.1

Host: any.net

Accept: image/gif, image/jpeg, */*

ParseParse

RequestRequest

GET

/users/jxh/public_thml/home.html

HTTP /1.0

ParseParse

HeadersHeaders

Static vs Dynamic

� Static Configuration� HTTP 1.0 Request

� limited # of processing operation

� Marsharing & Demarsharing

� Demutlipelxing� Demutlipelxing

� Dynamic Configuration� Unlimited # of Processing operation

� Requires a lot of process arranging cost.


Example

� Advanced Search Engine

� May construct data filters dynamically.

� “(perfomrnace AND (NOT symphonic))”

� Requesting for Web pages containing the word

“performance” but not the word “symphonic”.

� Could be implemented as a pipeline of

� a positive match component coupled with

� a negative match component


Example (cont’d)

� Image Servers

� User may request that

� Image be cropped, scaled, rotated, and dithered.

� Adaptive Web Contents� Adaptive Web Contents

� Dynamic Delivery for End User.

� PDA User

Web Content Overviews and Smaller Image.

� Workstation & Personal Computer

Full Multimedia Enriched Pages.


JAWS Protocol Pipeline Strategy

Protocol Handler InputOutput Handler

Protocol Pipeline

svc()

Read Request

svc()

Filter

component->svc()io->receive_data()

pipepipe ioio

componentcomponent


svc()

Filter

svc()parse()

Parse Headers

svc()parse()

Perform Request

svc()perform()

Log Request

svc()log()

Filter::svc()

parse()

Filter::svc()

perform()

Filter::svc()

log()

3.6 File Caching Strategies

� Design Challenges� Accessing the File System is

a significant source of web server overhead.

� Cache is a storage medium

that provides more efficient retrieval.

� Alternative Solution Strategies� Least Recently Used (LRU) Caching.

� Least Frequently used (LFU) Caching.

� Hinted Caching.

� Structured Caching.

� JAWS Cached Virtual File System Framework


Alternative Solution Strategies

� Least Recently Used Caching

� Least Frequently Used Caching

� Structured Caching

� Refers the Next Page.� Refers the Next Page.

� Hinted Caching

� Analysis of Web page retrieval pattern.

� Web pages have spatial locality.

� Using statistical information about links (hinted).

� Using Pre-fetching Method.

� Refers “Hinted Caching in the Web”9/10/20089/10/2008http://www.EvaCast.nethttp://www.EvaCast.net 6060

Alternative Solution Strategies

� Structured Caching


JAWS Cached Virtual FS Framework

InputOutput

receive_file()send_file()

filename

InputOutput

handle()mapaddr()name()size()open()

Filecache Object

handle()mapaddr()name()size()

fhfh fofo

fh->open(file)cvf->open(file,fo)

cvfcvf


HashTable

Filecache

find()fetch()remove()create()findish()

rw_lock[]

Cache Strategy

insert()

LRU Strategy

insert()

LFU Strategy

insert()

replace least

recently used

Replace least

Frequently used

hthtcachecache

JAWS Cached Virtual FS Framework

Perform Request

svc()

fhfhInputOutput Handler


InputOutput


Filecache Handle

iohioh

ioio

ioioio->send_file(“index.html”)


complete()error()

send_data()

Synch IO


Asynch IO


Proactor

Reactive IO


Reactor

write(fh)Proactor->

send_file(fh)

Reactor->

send_file(fh)

3.7 The JAWS Framework Visited

Concurrency

Thread-per-Session

I/O

Synchronous

Cache

LRU

Protocol

&

Filter

Read Request

Parse Request


Thread-per Request

Thread Pool

Reactvie

Asynchronous

LFU

Structured Cache

Hinted Cache

Parse Request

Parse Header

Perform Request

Log Request

� Hardware Testbed

� Software Request Generator

� Experimental Results

� A Summary of Techniques

for Optimizing Web Servers.


4.1 Hardware Testbed

� System Specification

� Two Micron Millerinia PRO2 plus workstation

� 128M RAM

� Client – 200 MHZ

� Server – 180 MHZ� Server – 180 MHZ

� ENI-155P-MF-S ATM Card

� ATM Network running through

� a FORE Systems ASX-200BX (bandwith 622 Mbps)

� Limitation of LAN emulation mode = 120 Mbps


4.2 Software Request Generator

� Experiment Purpose� Using WebSTONE Stress Tool

� Collect Client and Server Side Metric

� average server throughput, client latency

� Gather Loads and Statistics

� using various particular files sizes

� Refers the next page.

� Using various concurrency & dispatching strategies

� research the impacts of results.


4.2 Software Request Generator

� File Access Patterns

Document Size

500 bytes

Frequency

35%


500 bytes

5 Kbytes

50 Kbytes

5M Bytes

35%

50%

14%

1%

4.3 Experimental Results

� Using 3 Models

� Definition of Terms

� Throughput Comparisons

� Latency Comparisons� Latency Comparisons

� Summary of Benchmark Results

� A Summary of Techniques

for Optimizing Web Servers



� Using 3 Models

� Synchronous Thread-per-Request

� Using Synchronous I/O

Synchronous Thread Pool� Synchronous Thread Pool

� Using Synchronous I/O

� Asynchronous Thread Pool

� Using TransmitFile() for Asynchronous I/O

� TransmitFile() is a custom Win32 Function()



� Definitions of Terms� Throughput

� The average # of bits received per second by client.

� Start Time

� HTTP Request received by client.

� End Time

� Connection is closed at the client end.� Connection is closed at the client end.

� Latency

� The average amount of delay in milliseconds

� Start Time

� HTTP GET Request received by client.

� End Time

� Completely Received File.



� Throughput Comparisons.

� Connection ↑ - Throughput ↓

� As expected,

The throughput for each connection degrades

as the connections per second increase.



� Throughput Comparisons.� Refer Figure 3.

� Experiment Results from 50K Files.

� Thread-Per-Request Model� Thread-Per-Request Model

� Connection load ↑ -> Throughput ↓ rapidly

� Incurs higher thread creation overhead.

� Synchronous Thread Pool

� Connection load ↑ -> Throughput ↓ more gracefully

� Uses Pre-spawning thread.




� Refer Figure 1 through 5.

� TransmitFile() performs extremely poorly

for small files ( i.e. < 50K bytes)

� TransmitFile() is forced

� to wait while the kernel services incoming request.

� As the Size of the file grows,

� TransmitFile() rapidly outperforms the synch model.




Fig 1. 500 Byte File


Fig 1. 500 Byte File

Fig 2. 5K Byte File




Fig 3. 50K File

Fig 4. 500K File




Fig 5. 5M File

4.4 A Summary of Techniques (for Optimizing Web Server)

� Lightweight Concurrency� Don’t use Process-based concurrency Mechanism.

� Using POSIX thread.

� Specialized OS Feature� Using IOCP� Using IOCP

� Request Lifecycle system call overhead.� Reducing synchronization.

� Caching files.� Reducing Iterative System I/O Call (open, read, write..)

� Using “gather-write”� Using writev (Unix System Call)

� Pre-computation HTTP response



� Logging Overhead� File System access

� Makes a significant number of I/O Calls.

� Synchronization overhead� Thread/Process are required to log request to a common shared log file, � Thread/Process are required to log request to a common shared log file,

access to the log file needs to be synchronized.

� Reverse hostname lookups� Hostname is typically more useful information in the log file.

� Ident Lookups � For identifying User Name

� Make setting up a new TCP/IP Connection



� Transport layer optimizations

� The listen backlog

� Give Queue Size Higher.

� Socket Send buffers� Socket Send buffers

� Should be set to the highest permissible limit.

� On Solaris, 64KB limit

� Nagle’s Algorithms

� Congestion Control Algorithm

� Server latency critical App (such as X-Windows)

disable this algorithm.


5. Concluding Remarks


References

� POSA1

� Layer , Pipe & Filter

� POSA2� POSA2

� Component Configurator , Active Object,

Dispatcher (Reactor, Con-Acc, Proactor)

� Composite Message Pattern


jaws(web server framework using patterns)

Technology

architect http

good web server

web servers tp version