Messaging Systems and Technology

Introduction

• Synchronous middleware doesn’t suit all applications

• Message Oriented Middleware (MOM) provides features like:

• asynchronous communications between processes• store-and-forward capabilities• transactional messaging

Basic Asynchronous Operations

• Send (dest, message)• Receive ( target, message)

receivesendqueue

MOM usage

• Applications often need:• deferred processing of some ‘slow’ transaction (eg

printing an invoice)• support efficient 1-to-many and many-to-many

communications• to send messages whether the server is available or not• to provide event notification to a dynanic user

community• loosely coupled client-server systems

Example - Transactions

//transaction 2transaction {

GetMessage (....)} onCommit { .... }onAbort {.....}

DBMS Queue

//transaction 1transaction {

UpdateDBMS(…..)PutMessage (....)

} onCommit { .... }onAbort {.....}

MOM Technologies

• Broadly two (somewhat intersecting) categories

• Message brokers (IBM, BEA, Microsoft, etc)• Broadcast-based publish-subscribe (TIBCO, IONA)

• Both categories differ greatly in their features and capabilities

Some Application Examples

• Stock price update notification• General workflow systems• Application integration• System management

Some Example Products

• TIBCO/Rendezvous• IONA’s OrbixTalk• IBM’s MQ Series (Publish-Subscribe)

TIB/Rendezvous• Pioneers of broadcast/multicast publish-

subscribe technology• Publishers and subscribers communicate

using subjects

PubSubject

SubSub

SubCreate/Publish

Register/Subscribe

Subjects

• Hierarchical names identify a subject of interest

• /CSIRO• /CSIRO/gossip• /CSIRO/gossip/ADSaT• /CSIRO/work/ADSaT

• Wildcards can be used• /CSIRO/**• /CSIRO/*/ADSaT

Multicast/Broadcast

• A published message is sent across the network only once

• All subscribers receive the same message• IP Multicast or broadcast is used on LANs• Important scalability issue as

bandwidth/processor usage is low

LAN Architecture

BroadcastNetwork

rvd

rvd

Subscribe

Publisher

Publisher

Subscribe

WAN Architecture

LocalBroadcastNetwork

rvd

Subscribe

Publisher

rvrdrvrd

LocalBroadcastNetwork

rvd

Publisher

Subscribe

Wide Area Network

Filtered,Point-to-Point

Quality-of-Service (QOS)

• Reliable• publisher informed after time-out period if message

cannot be delivered to a subscriber

• Certified• guaranteed delivery or both parties informed of

failure• to survive process failure messages can be logged to

disk until specified time-out period expires

Programming Options

• C/C++/Java• Self-describing message format• TIB/ObjectBus layers a standard CORBA-

compliant ORB upon the TIBCO protocols. • TIBIOP provides broadcast messages in a

CORBA environment.

Security

• Security is a problem in publish-subscribe multicast technology

• TIBCO Data Security (DS) product:• publisher controls secure key distribution to

subscribers using digital signatures• subscribers use key to encrypt messages on a subject• choice of encryption algorithm available

OrbixTalk

• Uses IP multicast in CORBA environment

OrbixTalkApplication

OrbixTalk Multicast Transport Protocol

OrbixTalk API

CORBAEvent Service

OrbixTalkApplication

Topics• Communication based on topics (read

subjects, wildcards, etc)• 3 QOS levels:

– otmcp: UDP IP multicast. Limits packet size to 1280 bytes and no guarantee of receipt.

– otrmp: Augments IP multicast to ensure message delivery to all subscribers until configurable time-out expires

– otsfp: OrbixTalk MessageStore daemon uses a store-and-forward protocol to provide guaranteed message delivery.

OrbixTalk Architecture

Sender

Sender

Sender

Sender

MulticastProtocols

Listener

Listener

Listener

Listener

OrbixTalkDaemon

MessageStore

Daemon

I P Address

otmcp

otrmp

Topic

otsfp

Topic

IPAddress

Architecture Issues

• Daemons can have hot backups to support failure of primary daemon

• Several MessageStore daemons can be used to facilitate load balancing:

• each OrbixTalk process only use one MessageStore

• Compaction utility must be scheduled to remove old messages

Security/WANs

• No specific security or WAN support• security can be programmed using Orbix Transformers

• WANs traffic needs appropriately configured routers, or an IIOP bridge between LANs to be written

MQSeries PubSub

• MQSeries is probably most widely deployed MOM product

• Basic technology provides a message queue architecture:

• communication via shared queues managed by Queue Managers

• persistent/non-persistent messages• transactional queue access with XA• broad platform support

MQSeries PubSub

BROKER A

BROKER B

Subscriber1

Subscriber2Publisher

1

Publisher2

Topic A

Topic B

Topic B

Topic B

Topic A

1 PubSub Broker per QM

Topics interchanged by Brokers if needed

LAN/WAN support

Architecture

• Brokers can be organized to communicate hierarchically

USA

West East

HQ

Asia

HK

Features

• Typical hierachical topic names, wildcards, etc

• Streams can be used as higher level topic partitioning scheme

• DEFAULT stream per broker• optional additional streams for specified topics• broker allocates a thread to handle each stream

Quality-of-Service

• Persistent• messages written to log file• survive broker/QM failure

• Non-persistent• messages lost if broker fails• fast

• Message priority supported in queues

Other features

• MQ can act as an XA transaction manager• Basic security provided, integrated with

native operating system• Security exits support integration with 3rd

party products• Other products, eg MQ Integrator

Performance Test

• Publishers and subscribers run on different machines

• Dual pentium 500Mhz NT 4.0 boxes, 0.5GB memory

• MQ QM/broker and Orbix daemons run on same machine as publishers

• Reliable, non-presistent protocols used

256 Byte Message

0100200300400500600700

10 20 30 40 50 No. Of Subscribers

Milliseconds

OrbixTalkTIBCOMQ

512 Byte Message

0

200

400

600

800

1000

10 20 30 40 50No. Of Subscribers

Milliseconds

OrbixTalkTIBCOMQSeries

Interpretation

• Multicast technologies are fast!• TIBCO rvd becomes bottleneck

• ‘Publisher too fast’ exceptions

• OrbixTalk generates large number of interrupts

• one per message per listener

• MQ scales well, but queue/broker architecture is inherently slower

Summary• MOM provides excellent solution to many

business problems• Range of technologies available, with very

different:• architectures/technologies• features• performance

• Important to understand issues and select right one for your business