SESSION 1:

MIDDLEWARE HISTORY

AND TECHNICAL OVERVIEW

Michael Schonberg

OpenMAMA Maintainer

April 4, 2012

NYSE Technologies

Agenda

Session 1 (09:00-10:00)

Introduction

A Brief History of Message Oriented Middleware

Messaging Concepts

OpenMAMA: Highlights

OpenMAMA: Architecture

OpenMAMA: Key Constructs Overview

Session 2 (10:00-11:00)

OpenMAMA: Key Constructs Details

OpenMAMA: Writing Code

Introduction

Introduction

OpenMAMA is:

Middleware Agnostic Messaging API

Supports a variety of Message Oriented Middleware (MOM) platforms

MOM allows applications to exchange information without establishing explicit connections

De-facto standard in financial services

A consistent abstraction layer

High performance

Open Source

Hosted by the Linux Foundation

Supported by a diverse group of industry stakeholders

Presented by Feargal O’Sullivan

History Lesson

The Idea

The hardware BUS

architecture… applied

to software

Two Main Lines

Publish/Subscribe Made Famous By: Teknekron Software Systems (nka TIBCO Software)

• Client-Server • Daemon Based • Broadcast/Multicast • Event Driven • High-speed • Topic Naming Scheme • ‘Reliable’ Delivery

Queues Made Famous By: International Business Machines (aka IBM)

• Mainframe to Client-Server • Hub & Spoke • Point-to-point • Guaranteed Delivery • Slow (but steady) • Queue Names • Queue over-flow

Common Features • Decoupling • Fault tolerance • Fire and forget • Payload encoding • Entitlement control

Hybrids

Certified Delivery on Pub/Sub

Hub & Spoke on Pub/Sub

Content Routing

Daemonless Pub/Sub

Replay Capabilities

Last Known Value Cache – Distributed Caches

Multiple Offerings

TIBCO Rendezvous

IBM MQ Series (WebSphereMQ)

JMS: TIBCO EMS; Sonic MQ etc.

29West LBM (now Informatica)

AMQP: Red Hat MRG

ZeroMQ – SonicMQ

Solace Systems (FPGA Acceleration)

Real-time Innovations (RTI)

NYSE Technologies Data Fabric

Etc.

Applied Middleware

10 Market

Gateway

Feed Handler

Algo

Cache

Quote

Conflation Cache

Desktop Traders

M

o

n

i

t

o

r

i

n

g

Historical Tick

Capture

Trade

Custom FX Rates

SuperFeed

Market Gateway

Market Gateway

Algo Algo

Presented by: Mike Schonberg

Messaging Concepts

Message Oriented Middleware

Publish-subscribe semantics

Publishers send messages

• Broadcast semantics

Subscribers receive messages

• Topic vs. content based

• OpenMAMA supports topic based

Request-reply

Publishers broadcast

Subscribers optionally send reply directly to publisher

Location independent

Publish-Subscribe

Topic resolution is automatic

Publishers require no awareness of subscribers

Subscribers have no explicit knowledge of publishers

Request-Reply

Request is a normal message with “return address”

Subscribers optionally reply directly to requester

• Point-to-point

Transports and Implementations

Transports

TCP

UDP Multicast

RDMA

Shared Memory/IPC

IB-Verbs

Implementations

NYSE Technologies Data Fabric

• TCP, RDMA, Shared Memory, IB-Verbs

TIBCO Rendezvous and FTL

AMQP

IBM Websphere MQ

OpenMAMA: Highlights

What is OpenMAMA?

Open Source Middleware Agnostic Messaging API

Hosted by Linux Foundation

LGPL 2.1

Single API for event-driven messaging

Multiple middlewares and payloads

High performance

Mature

Widely deployed in financial services

Potential applications outside finance

Why Use OpenMAMA?

Eliminate vendor-lock in

Flexibility

Write once for multiple middlewares

• Develop to a single open API

• Deploy over multiple middlewares

Simplify IPC

Focus on critical business logic

More than an just an API

Advanced data quality support

Gap detection

State recovery

Dictionary support

Initial images

Advanced feature implementation

Queues and dispatchers

Timers

IO events

Etc.

Applications

Financial Services

Market data distribution

Order routing

Risk management

Other

HPC/parallel data processing

Logistics

Any application requiring data distribution to

heterogeneous consumers

OpenMAMA: ARCHITECTURE

OpenMAMA Architecture

OpenMAMA: Key Constructs

Key Constructs

Transports

Queues

Publishers

Events

Subscriptions

Inbox (for request/reply)

Timers

IO events

Q & A

mschonberg@nyx.com

SESSION 2 – UNDER THE HOOD

Mike Schonberg

Agenda

Transports

Queues

Publishers

Events

Value Added Components

Code

Introduction to Bridges

Transports

Establish connections connections between

publishers and subscribers

Mechanism varies widely between middlewares

• NSD (NYSE Technologies Data Fabric)

• Broker based (Avis, AMQP, TIBCO Rendezvous)

• Multicast topic resolution (Informatica LBM)

Global configuration and tuning

Buffer sizes

Network parameters

Throttle rates

Queues

Queue based threading model

One dispatch thread per queue

Several dispatch modes

• Single event: invoke one callback and return

• Timed: dispatch until empty or timeout

• Continuous: dispatch forever

Callback based

Advanced features

Watermarks to monitor queue depth

Enqueue callback to integrate with other event loops

Custom callbacks

Publishers

Send messages and requests

One publisher per topic

Associated with a specific transport

Thinnest possible wrapper to native publishing

Support throttled publishing

Advanced publishing framework

dqpublisher in source

Stateful (market data publishing)

Subscriptions

Subscription to a single topic

Receive message from all publishers of topic

onMsg() callback invoked for each message

Creation throttled

Optional wildcard support

Basic vs. stateful (market data)

Stateful receive initial state

Stateful monitor sequence numbers

Inbox

Receives replies through onMsg() callback

Multiple replies possible

• From different subscribers

• From the same subscriber

Parameter to publisher_sendFromInbox()

Supports multiple requests

Timers

Repeating timers

Interval is floating point in seconds

Resolution dependent on native middleware or OS

onTimerCallback()

Enqueued on most once per interval

Queue depth makes non-deterministic

OpenMAMA provides default implementation

IO Events

Asynchronous IO support

For non-blocking file descriptors and sockets

Invokes callbacks for read and write readiness

Support is optional

Integrates asynchronous IO into OpenMAMA event

dispatching

OpenMAMA: Value-added Components

Other Components

Stateful market data subscriptions/data

quality

Advanced publishing framework

Data dictionary

Fault tolerance

Stateful Subscription

Initial value

delivered point-to-

point

Complex sequence

number logic

Other potential

applications

Data Quality

Essential for Stateful/Market Data Subscriptions

Sequence number monitoring

Request recap on gap detection

Recap is an initial image

Pre-initial spooling

Service disruptions from native middleware

Callbacks inform application of data quality events

STALE -> data definitely lost

POSSIBLY_STALE -> data may be lost

Advanced Publishing

Framework for publishing to stateful/market data

subscribers

Sequence number and status fields for data quality

Initial value requests

Recap requests

Refresh messages

Opitional

Data Dictionary

OpenMAMA supports self describing messages

Field type, name, id (integer) and value

Sending names with every msg -> big messages

Dictionary maps id’s to names, types and values

From a server as a message

Loaded from a file

Built dynamically

Allows integration of non-self describing payloads

Fault Tolerance

Applications can run in fault tolerant groups

Every instance has a unique weight

Heart beat messages determine failure

UDP multicast

Bridge using OpenMAMA and native middleware

Callbacks inform instances of transitions

Application responsible for recovery logic

OpenMAMA in Action

Let’s write a program!

Create a publisher

Open a session

Pack a payload

Send message

Create a subscriber

Open interest in a topic

Receive message callback

Unpack message

OpenMAMA Publisher Example

Taken from mamapublisherc.c example

Publishes a message at regular interval

Subscribes to a request topic

Sends reply on receiving a request

Key concepts Publishing

Subscribing

Replying to requests

Creating messages

Timers

int main(void) {

initializeMama ();

createIntervalTimer ();

createInboundSubscription ();

createPublisher ();

mama_start(gMamaBridge);

return 0;

}

Initialization

Load middleware

bridge

By name “wmw”,

“lbm”, “tibrv” or “avis”

Initialize OpenMAMA

Create or get queue

Create a transport

• Two step

1. Allocate

2. Create (initialize)

void initializeMama(void) {

mama_status status;

status = mama_loadBridge(&gMamaBridge,

gMiddleware);

if (status != MAMA_STATUS_OK) {

printf ("Error loading bridge: %s\n",

mamaStatus_stringForStatus (status));

exit (status);

}

status = mama_open();

mama_getDefaultEventQueue(gMamaBridge,

&gMamaDefaultQueue);

status =

mamaTransport_allocate(&gTransport);

status = mamaTransport_create(gTransport,

gTransportName,

gMamaBridge);

}

Create a Timer

Initialize timer

Default queue from

initialization

1 second interval

Repeating

Callback publishes

message

static void createIntervalTimer(void){

status = mamaTimer_create(&gTimer,

gMamaDefaultQueue,

timerCb,

1.0,

NULL);

}

static void MAMACALLTYPE

timerCb(mamaTimer timer, void* closure){

publishMessage (NULL);

}

Create a Subscription

Subscription for

requests

Send reply for each

request

Callbacks

No messages before

createCb

Two step create

msgCb for each

message

static void createInboundSubscription (void){

gInboundCb.onCreate = createCb;

gInboundCb.onError = errorCb;

gInboundCb.onMsg = msgCb;

gInboundCb.onQuality = NULL;

gInboundCb.onGap = NULL;

gInboundCb.onRecapRequest = NULL;

mamaSubscription_allocate(&gSubscription);

mamaSubscription_createBasic(gSubscription,

gTransport, gMamaDefaultQueue, gInboundCb,

“REQUEST_TOPIC”, NULL);

}

static void MAMACALLTYPE

inboundMsgCb(mamaSubscription subscription,

mamaMsg msg, void* closure, void* itemClosure){

if (!mamaMsg_isFromInbox (msg)) {

return;

}

publishMessage (msg);

}

Publishing

Publisher requires transport and topic.

Optional source and root

Sending Messages

Static message reused

Standard MAMA fields: Status and SeqNo

Custom field: Publisher Topic

request != NULL: called from msgCb(). Point-to-point reply

Request == NULL: called from timerCb(). Multicast message.

static void createPublisher(void){

mamaPublisher_create(&gPub, gTransport,

“MAMA_TOPIC”, NULL, NULL);

}

static void publishMessage(mamaMsg request){

static int msgNumber = 0;

static mamaMsg msg = NULL;

if (msg == NULL) {

mamaMsg_create(&msg);

}

else {

mamaMsg_clear(msg);

}

mamaMsg_addI32(msg, MamaFieldMsgStatus.mName,

MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_OK);

mamaMsg_addI32(msg, MamaFieldSeqNum.mName,

MamaFieldSeqNum.mFid, msgNumber++);

mamaMsg_addString (msg, "PubTopic", 10002, “MAMA_TOPIC”);

if (request){

mamaPublisher_sendReplyToInbox(gPublisher, request, msg);

}

else {

mamaPublisher_send(gPublisher, msg);

}

}

What Does it Do?

Timer callback fired every second

Sends a message on “MAMA_TOPIC”

Listens for messages on “REQUEST_TOPIC”

Message callback fired

• If inbound message is a request then send reply

Middleware Bridges - Overview

Bridges

Payload and Message Bridges

Directly reflect top-level API

• Messages, subscriptions, queues, timers, etc.

One-to-one function mapping

Virtual function table

Dynamic loading and static linking

Multiple bridges simultaneously

OpenMAMA macros build/load table

• Bridge implementations just provide methods

Questions?

mschonberg@nyx.com