apache multipurpose infrastructure for network …€¦ · red5 quickfix/j apache directory ... xml...
TRANSCRIPT
![Page 1: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/1.jpg)
Apache Multipurpose Infrastructure for Network ApplicationsBuilding Scalable Network Applications
Dan PritchettRearden Commerce
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 1
![Page 2: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/2.jpg)
What are network applications?
Application logic is primarily protocol handling
Good Examples•SMTP•LDAP•FTP
Tend to have a high ratio of I/O to application logic.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 2
![Page 3: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/3.jpg)
How have we solved this?Worker thread per request pattern
Listener thread accepts connections.Connection established with client, delegated to a worker thread.Worker remains bound to client until service request is complete.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 3
![Page 4: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/4.jpg)
Why doesn’t that work?Majority of thread time in I/O waitThreads are expensive
500KB stack per threadContext switching overhead
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 4
![Page 5: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/5.jpg)
Enter SEDAStaged Event Driven Architecture
Non-blocking I/O generates eventsStages perform operations on events
Stage1
Stage2
Stage3
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 5
![Page 6: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/6.jpg)
Benefits of SEDADecouple threads from I/O
One thread can service many I/O channelsImproved resource utilization
Higher throughputLinear degradation in service response time.Equally important more balanced performance
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 6
![Page 7: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/7.jpg)
SEDA Performance
050
100150200250
1 16 64 128 256 1024
Throughput
Apache Haboob
00.20.40.60.81.0
0 10 100 1000 1000 100000
Response Time Distribution (ms)
0.5
0.7
0.9
64 128 256 1024
Jain Fairness
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 7
![Page 8: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/8.jpg)
SEDA Implementations
Apache MINAHaboob/SandstormLimeWireTerraLycosRimfaxe Web Server
Apache ExcaliburSwiftMQMULEOcean Store
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 8
![Page 9: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/9.jpg)
Apache MINAMultipurpose Infrastructure for Network Applications
Apache implementation of SEDA
A pattern and framework for developing network services.Java 5.0 NIO implementation
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 9
![Page 10: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/10.jpg)
Projects Using MINA
Apache QpidAvisSubEthaSMTPJiveOpenLSDRed5
QuickFIX/JApache Directory ProjectBeep4j
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 10
![Page 11: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/11.jpg)
MINA Application Components
Acceptor/Connector
Filter
Handler
Accept inbound connections
Establish outbound connections
Transform data
Ancillary Processing
Protocol Logic
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 11
![Page 12: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/12.jpg)
MINA FiltersFilters are the primary source of code factoringCategories of filter methods
Data Flow•messageReceived, filterWrite, messageSentSession Management•sessionCreated, sessionOpen, sessionClosed, sessionIdle
Filter Management•onFilterPre/PostAdd, onFilterPre/PostRemoveErrors•exceptionCaught
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 12
![Page 13: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/13.jpg)
Data Flow Events
messageReceived•Inbound events
filterWrite•Outbound events, prior to I/O
messageSent
•Outbound events, after I/O
Filter
Connector
mes
ageR
ecev
ied
filterWrite
mes
ageS
ent
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 13
![Page 14: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/14.jpg)
Session Management Events
sessionCreated•Prior to I/O connection established
sessionOpened•Once connection is established
sessionClosed•Once connection is closed
Filter
Acceptor
Clien
t Co
nn
ectsessio
nC
reated
Co
nn
ection
sessionO
pen
ed
sessionC
losed
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 14
![Page 15: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/15.jpg)
Filter Lifecycle & Errors
Filter additionsonPreAdd, onPostAdd
Filter removalsonPreRemove, onPostRemove
ErrorsexceptionCaught - All calls from framework catch Throwable and route through this method.•This method should avoid throwing exceptions
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 15
![Page 16: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/16.jpg)
Handlers
Handlers implement the service logic.Handlers are a special filter.
Operations provided by IoHandler interface.Same methods as filter, except filter lifecycle.Installed automatically as last filter in chain•Last to receive inbound events•First to receive outbound events
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 16
![Page 17: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/17.jpg)
Filter Chains
Filters are attached to acceptors and connectors.Events pass through the chain in order.
Filter can control if events propagate or stop.
Filter1
Filter2
Filter3
Handler
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 17
![Page 18: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/18.jpg)
Typical MINA Application
Transform streams to objects
Track collection of objects as logical
Administration and management.
Protocol Semantics
Codec
SessionManagement
Logging &Statistics
ProtocolHandler
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 18
![Page 19: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/19.jpg)
Codec
Encoder/DecoderTranslates byte streams into objectsStreams arrive as byte buffers•Buffers and parsing units orthogonal.
For exampleXML document may be in many buffersOne buffer may contain many HTTP headers
Don’t assume anything about buffer boundaries and protocol boundaries.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 19
![Page 20: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/20.jpg)
Session Management
Protocols may involve multiple events in a logical session.
HTTP for example:•Request•Headers•0 or more body blocks
Session filter provides aggregation of events for downstream filters.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 20
![Page 21: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/21.jpg)
Logging/Administration
Loosely coupled metricsLog protocol informationStatisticsFlow control
Each capability can be written as a single purpose, reusable filter.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 21
![Page 22: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/22.jpg)
Example ServiceSimple Time Server
TCP/IP protocolClient sends their time, ISO formatted GMT, line terminated.Service responds with two lines:•Server time, ISO formatted GMT•Delta in milliseconds between the client and server.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 22
![Page 23: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/23.jpg)
Step 1: Service Configuration
SocketAcceptorConfig config = new SocketAcceptorConfig();config.setBacklog(5);config.setReuseAddress(true);DefaultIoFilterChainBuilder chain = config.getFilterChain();TextLineCodecFactory factory = new TextLineCodecFactory();ProtocolCodecFilter codec = new ProtocolCodecFilter(factory);chain.addLast(“codec”, codec);chain.addLast(“date”, new ISODateFilter());chain.addLast(“logger”, new LoggingFilter());chain.addLast(“counter”, new ServiceCounterFilter());
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 23
![Page 24: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/24.jpg)
Step 2: Acceptor Setup
TimeServiceHandler handler = new TimeServiceHandler();SocketAcceptor acceptor = new SocketAcceptor();InetSocketAddress listenAddr = new InetSocketAddress(8011);acceptor.bind(listenAddr, handler, config);
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 24
![Page 25: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/25.jpg)
ISODateFilterpublic class ISODateFilter extends IoFilterAdapter {
private static final DateFormat formatter = newSimpleDateFormat(“yyyy-MM-ddTHH:mm:ss.S”);
public void messageReceived(NextFilter next,IoSession session, Object message) {
Date date = formatter.parse((String)message);next.messageReceived(session, date);
}
public void filterWrite(NextFilter next,IoSession session, WriteRequest request) {
Response response = (Response)request.getMessage();StringBuilder text = new StringBuilder();text.append(formatter.format(response.date)).append(“\r\n”);text.append(Long.toString(response.delta)).append(“\r\n”);WriteRequest encoded = new WriteRequest(text.toString());next.filterWrite(session, encoded);
}}
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 25
![Page 26: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/26.jpg)
ServiceCounterFilter
public class ServiceCounterFilter extends IoFilterAdapter {private int requests = 0;
public void messageReceived(NextFilter next,IoSession session, Object message) {
requests += 1;next.messageReceived(session, message);
}
public int getRequests() {return requests;
}}
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 26
![Page 27: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/27.jpg)
TimeServiceHandler
public class TimeServiceHandler extends IoHandlerAdapter {public void messageReceived(IoSession session,
Object message) {Date client = (Date)message;Response response = new Response(client);session.write(response);
}}
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 27
![Page 28: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/28.jpg)
Response Object
public class Response {Date date;long delta;
public Response(Date client) {date = new Date();delta = client.getTime() - date.getTime();
}}
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 28
![Page 29: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/29.jpg)
Tips: ThreadsI/O Threads - Associated with Acceptors & Connectors
Manage NIO channels and deliver events into the chain.Take care to avoid starving NIO channels.
ExecutorFilter dispatches event on separate thread
Can be installed anywhere in filter chainRemember context switching adds to client latency
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 29
![Page 30: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/30.jpg)
Tips: Factoring
Filters are the unit of factoringFilter over head is small.Small, single purpose filters are preferable to handler logic.
Filters vs ObjectsNo simple rule for using one vs. the other.In fact, they are complimentary. Hierarchy of filters is reasonable.
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 30
![Page 31: Apache Multipurpose Infrastructure for Network …€¦ · Red5 QuickFIX/J Apache Directory ... XML document may be in many buffers One buffer may contain many HTTP headers Don’t](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b7966a07f8b9a534c8d429c/html5/thumbnails/31.jpg)
Further Reading
SEDAhttp://www.eecs.harvard.edu/~mdw/proj/seda/
MINAhttp://mina.apache.org/
Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Rearden Commerce
Dan Pritchett — Apache Multipurpose Infrastructure for Network Applications Building Scalable Network Applications Slide 31