fix with quickfix/j and epam fix - tech talks
TRANSCRIPT
1CONFIDENTIAL
FIX with QuickFIX/J and
EPAM FIX
Balázs Kollár
APRIL 25, 2015
2CONFIDENTIAL 2
About EPAM Systems, Inc
EPAM
Systems
1993 founded, US-based
Public (NYSE: EPAM)
B2BITS© EPAM's Capital Markets
Competency Center delivering a
broad range of solutions and
consulting services in Capital
Markets
Development
Centers
Belarus
Ukraine
Russia
Hungary
Kazakhstan
Poland, Bulgaria
Headcount >10000 (engineers)
Products:
FIX engines
FIX servers
Solutions for Direct Exchange Access
OMS & Matching engines
Market data feeds adaptors
FIX Testing & Operational Tools
Services:
Consulting
Integration
FIX on-boarding services
Cloud services
3CONFIDENTIAL
Table of contents
What is FIX protocol1
Systems utilizing FIX2
FIX messages, types3
Transports, threading4
Advantages & drawbacks of QFJ5
Benefits of EPAM FIX5
4CONFIDENTIAL
• Financial Information Exchange, in use since 1992
• Covers pre-trade, trade, post-trade communication
• Much more than another markup language:
• It’s types, value sets give a comprehensive business dictionary
• Standard names make understanding between parties easier
What is FIX protocol
5CONFIDENTIAL
SYSTEMS UTILIZING FIX
6CONFIDENTIAL
1. Enter, change, cancel orders
2. Order event processing
3. Provide order state and history
4. Get a list of orders (open, done)
OMS: Order Lifecycle Management
7CONFIDENTIAL
1. Clients subscribe to Market Data updates
– Clients need to know prices to submit orders
– Machines vs humans
2. Quote engines
– Clients request a “price offer” before submitting quoted orders
MarketData Streaming
8CONFIDENTIAL
1. Reconciliation modules get a notification of each trade. Later they check if all trades
arrived properly to the middle office.
2. Post-trade enrichment modules
– Client Identifying Data
– fees
3. Booking, allocation: splitting trades
4. Position and risk management
– How much USD I owe to the market? What amount I need to hedge?
Post-trade
9CONFIDENTIAL
Questions?
10CONFIDENTIAL
FIX MESSAGE TYPES
11CONFIDENTIAL
• A client would like to see the actual prices
• Market Data Request ->
– initiate subscription for stream of market data of an instrument
• <- Market Data (Snapshot/Incremental)
– Bid/Ask best price and amount: top-of-the book
– May be a full order book: best 5-10 prices and amounts
Pre-trade: Market Data
12CONFIDENTIAL
„A quote is an offer with validity. The sell side is obliged to hold this offer.”
1. Quote request: before entering an order clients can request an „offer”.
2. Quote: an offer containing price and amount, that’s valid until a certain time.
– Example: give me buy price for 2m$. Quote time is shown with progress bar.
Pre-trade: Quoting Orders
13CONFIDENTIAL
• New Order Single
– Side, Instrument, quantity, quantity unit: USD/EUR BUY 1m$
– OrdType: market, limit, stop: Buy @1.11 or better
– Price, StopPx
– TimeInForce
• IOC
• resting: day, good-till-cancel, good-till-date
• Detailed explanation: @Investopedia.com
Order entry: Single-leg (35=D)
14CONFIDENTIAL
• New Order Single
– Effective Time
– Expire Time
– Settlement Type/Date: T+2, T+3 days
– Parties
• Originating Firm
• Trader ID
• Account Nr
Order entry: Single-leg (35=D)
15CONFIDENTIAL
One message type for many purposes
1. Confirmation / Rejection
2. Replaced
3. Cancelled (may be unsolicited)
4. Expired
5. Filled: partial fill or last fill
6. DoneForDay: daily stats at end of the trading day
Order event notifications: ExecutionReport (35=8)
16CONFIDENTIAL
Notification attributes
Filled-, Open-, Cumulative quantity
Filled price
Avarage Price
Order event notifications: ExecutionReport
17CONFIDENTIAL
1. OrderCancelReplaceRequest (35=G)
– Not amendable: instrument, side, quantity unit
2. OrderCancelRequest (35=F)
– Only the open quantity can be canceled, the executed quantity can not
3. Both can be rejected: OrderCancelReject (35=9)
– Nothing happened: the original order remains active
Order amendment and cancellation
19CONFIDENTIAL
Questions?
20CONFIDENTIAL
SOCKET TRANSPORT,
REPLAYING
21CONFIDENTIAL
• Traditional Server-Client communication over sockets
• The server is called Acceptor
• The client is called Initiator
• The Sessions are preconfigured on the server:
• SenderCompID, TargetCompID
• Sequence numbers: both sides has a counter, always increasing
Socket transport
22CONFIDENTIAL
Socket transport
•ASCII, separator 0x00 or SOH
23CONFIDENTIAL
• Session acts like a queue on both sides: store and forward technique
• While offline both parties may produce messages, especially the one producing Executions
• Orders can be rejected while offline or delivered later
• After reconnection the offline messages will be delivered
Offline messages
24CONFIDENTIAL
1. When session was lost, the last SeqNr seen from the Acceptor side was 1000
2. While offline 500 trades are executed on the Acceptor
3. Initiator logs in to the Acceptor -> gets a login confirmation with SeqNr=1501
4. Now the FIX library knows that it missed 500 messages!
5. Initiator sends ResendRequest from 1001 to 1500
6. Messages >1500 will be queued in the FIX stack until the replay is finished.
• You may want to reject any new, replace, cancel requests until eveything is back in sync!
How message replaying works
25CONFIDENTIAL
Questions?
26CONFIDENTIAL
OTHER TRANSPORTS
27CONFIDENTIAL
1. Not plain text anymore, don’t need socket connection
2. Parsing, binding is for you to do: SAX, JAXB, JibX
3. You can send it via any transport: message queues, HTTP
4. QFJ transports it via 212/213 tags (XmlData) but still over Socket transport
5. No other support in QFJ, won’t bind it to it’s model
FIXML
28CONFIDENTIAL
1. Not supported by QuickFIX/J
2. You can use Apache Camel’s QuickFIX/J endpoints and route the messages to any other
transport: JMS, HTTP, file, etc.
3. Or you can use FIXML and generate the java model from XSD
FIX over tunnels
29CONFIDENTIAL
Questions?
30CONFIDENTIAL
THREADING
31CONFIDENTIAL
• QuickFIX/J does transport with async networking
• The thread you’re sending from (Session.send()) won’t deal with networking
• Always check the return value of send(), it may return false and won’t throw an exception
if session is not logged in
Threading: sending
32CONFIDENTIAL
1. You’ll receive messages on the QuickFIX/J listener thread
2. Dispatch all work to another thread if possible
– Throwing an exception on the listener thread you can break QFJ, trigger resending, etc.
3. Single thread or
4. Sticky thread assignment: Given a thread pool for processing. Make sure events of an order
are always processed on the same thread.
Threading: receiving
33CONFIDENTIAL
Questions?
34CONFIDENTIAL
MEMORY
35CONFIDENTIAL
1. Lots of wrappers, even around primitives
2. Produces lots of garbage
QFJ model
36CONFIDENTIAL
1. Replace FIXMessageEncoder
– Write an adaptor to read your internal model directly
2. Replace FIXMessageDecoder
– Bind to your model while parsing
Advanced: skipping QFJ model
37CONFIDENTIAL
Questions?
38CONFIDENTIAL
PROS/CONS FOR QFJ
39CONFIDENTIAL
1. Free and open source.
2. You can extend it.
3. Well tested. Not too many bugs faced.
4. We use it in prod for years now.
PROs
40CONFIDENTIAL
1. Memory footprint, GC overhead
2. Over-a-day scheduling not supported
– Early start on Sunday
3. HA limitations: only file, jdbc and sleepycat
– no cache-based message store, but easy to write one
4. Parser: no annotation-based binding to custom model
– But extensible: you can implement your own parser
CONs
41CONFIDENTIAL
1. Defalt parser creates lots of objects
2. Replay handling is hidden
– Hard to wait until replay is finished. You can add a listener.
3. No monotoring admin UI
– Just a simple JMX interface
4. No synchronuous sending.
CONs
42CONFIDENTIAL
EPAM FIX
43CONFIDENTIAL
EPAM FIX
44CONFIDENTIAL
FIX ANTENNA JAVA ENGINE API WORKING WITH MESSAGES
• Generic model to work with the abstract FIXMessage
class via fields and groups getters and setters
• Provides the highest performance
• Each FIX message type is a class with FIX fields as
members
• Use of intelligence to make work with FIX business
object more pleasant
• Message template to increase speed of sending
messages of the same structure but with different
values
• Allows working with values as with primitive type to
reduce garbage production
CREATE MESSAGE SINGLETON BY FIX
PROTOCOL VERSION &
MESSAGE TYPE
CREATE MESSAGE FROM RAW FIX STRING (PARSE)
SERIALIZE FIX OBJECT MESSAGE
TO RAW FIX STRING
ADD, REMOVE, MODIFY FIX FIELDS AND REPEATING
GROUPS
VALIDATE MESSAGE
API TO WORK
WITH FIX
MESSAGES
FIX FLAT MESSAGE MODEL
FIX OBJECT MODEL
PREPARED MESSAGES
EFFICIENT GETTERS AND SETTERS
45CONFIDENTIAL
FIX Antenna Java Low-Latency Design
1. Zero-GC design for FIX message parse/send/receive paths
2. Avoid the use of garbage producing calls in standard java library
3. Code instrumented GC-free regression testing of new releases
4. Use of pre-allocated object pools
5. Use of Java off-heap memory for data buffers
6. Custom data structures packed in byte / integer arrays (no standard java collections)
7. Use of memory mapped files in persistence layer
46CONFIDENTIAL
FIX ANTENNA JAVA INTEGRATION
• Fully functional applicationsupporting all features ofnew MDP3.0 platform
• Functional core to support FAST
• Set of libraries for FIX FIXML conversion
•Links FIX Antenna with the Java Message Service
• Contains a server for routing FIX messages to and from JMS server
JMS ADAPTOR
FIXML CONVERTOR
CME MDP 3.0
ADAPTOR
FAST ENGINE
Set of adapters extending FIX Antenna Java functionality
47CONFIDENTIAL
FIX ANTENNA JAVA THREADING MODEL
• Send in the same thread where FIXSession.sendMessagewas called
• No additional delays for threads communication (better latency)
• No blocking on sending (till queue has free space)
• FIX Antenna can send messages in batches (better throughput)
• Current queue size is accessible for analysis
SYNCHRONOUS SENDING
USER
THREADSERIALIZE FIX
MESSAGE
OUTGOING MESSAGE
STORAGE
MESSAGE
QUEUE
INCOMING MESSAGE
STORAGE
PER TYPE
HANDLERS
USER
DEFINED
HANDLERS
SYSTEM
HANDLERS
READ &
PARSE FIX
MESSAGE
READING
THREAD
FIX ANTENNA
SYSTEM
THREAD POOL
INCOMING MESSAGE
STORAGE
USER
DEFINED
HANDLERS
SYSTEM
HANDLERS
READ &
PARSE FIX
MESSAGE
READING
THREAD
FIX ANTENNA
SYSTEM
THREAD POOL
USER THREAD
MESSAGE
QUEUE
SERIALIZE
FIX
MESSAGE
OUTGOING
MESSAGE
STORAGESENDING
THREAD
ASYNCHRONOUS SENDING
48CONFIDENTIAL
FIX ANTENNA JAVA THREADING MODEL
FIX ANTENNA
FIX SESSION
FIX MESSAGE
SENDER
FIX MESSAGE
READER
Input
Disruptor
SENDER
OUTPUT
DISRUPTOR
RECEIVER SENDER
FIX
FIX
FIX
• Input and output disruptors
• Asynchronous sending – stay not affected by slow FIX clients
COOPERATION WITH DISRUPTOR
FIX SESSION
FIX MESSAGE
SENDER
FIX MESSAGE
READER
FIX SESSION
FIX MESSAGE
SENDER
FIX MESSAGE
READER
49CONFIDENTIAL
FIX Antenna Java “Hello World!”
SessionParameters details = new SessionParameters();
details.setFixVersion(FIXVersion.FIX42);
details.setSenderCompId("BLP");
details.setTargetCompId("SCHB");
FIXSession fixSession =
connectionDetails.createNewFIXSession();
message.set(11, clordid);
message.set(44, 100); // Price
message.set(38, 10); // OrderQty
fixSession.sendMessage(message);