easy enterprise application integration with rabbitmq and amqp
DESCRIPTION
VMware vFabric RabbitMQ Technical Webinar December 2010 by VMware engineer Emile Joubert. Covers common integration patterns, and how RabbitMQ makes these easily implemented, using AMQP as a communications mechanism. You can view a recording of this presentation on YouTube: http://www.youtube.com/user/SpringSourceDev#p/c/5956C6D9EC319817/0/ABGMjX4K0D8TRANSCRIPT
![Page 1: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/1.jpg)
© 2010 VMware Inc. All rights reserved
Emile Joubert
December 2010
The Advanced Message Queuing Protocol (AMQP) is an application layer
protocol for messaging. This talk introduces RabbitMQ, itsprominent multi-lingual open-source implementation.
![Page 2: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/2.jpg)
© 2010 VMware Inc. All rights reserved
Disclaimer
This session may contain product features that are currently under development.
This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
"THESE FEATURES ARE REPRESENTATIVE OF FEATURE AREAS UNDER DEVELOPMENT. FEATURE COMMITMENTS ARE SUBJECT TO CHANGE, AND MUST NOT BE INCLUDED IN CONTRACTS, PURCHASE ORDERS, OR SALES AGREEMENTS OF ANY KIND. TECHNICAL FEASIBILITY AND MARKET DEMAND WILL AFFECT FINAL."
![Page 3: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/3.jpg)
© 2010 VMware Inc. All rights reserved
About me...
Emile studied computer science and mathematics before following a career that included electronic publishing; joining LShift in 2008 and SpringSource in 2010.
![Page 4: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/4.jpg)
© 2010 VMware Inc. All rights reserved
Outline
• Messaging• AMQP• Publishing walk-through• RabbitMQ and the benefits of Erlang• Large scale sample uses
![Page 5: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/5.jpg)
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Decoupling
![Page 6: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/6.jpg)
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Bidirectional Decouplinge.g. Remote Procedure Call
![Page 7: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/7.jpg)
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Pipelining and Decoupling
![Page 8: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/8.jpg)
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Work-distribution and Decouplinge.g. duplication and round-robin
![Page 9: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/9.jpg)
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Work aggregation, distribution and decoupling
![Page 10: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/10.jpg)
© 2010 VMware Inc. All rights reserved
Why Messaging?
SQL : : Databases : : FilesystemAMQP : : Messaging : : Network
• Logical decoupling• Physical decoupling• Temporal decoupling
![Page 11: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/11.jpg)
© 2010 VMware Inc. All rights reserved
AMQP
• An open, wire-level binary protocol for
Message Orientated Middleware• Created by users and technologists
working together, to satisfy real needs• A straight-forward and complete
solution for business messaging: SMTP for business
![Page 12: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/12.jpg)
© 2010 VMware Inc. All rights reserved
Transport
Framing
AMQP Wire Format
AMQP Model
Network-friendly
Network-optimized
Multi-vendorinteroperability
Flexible routing/queuing parameters
AMQP Model
![Page 13: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/13.jpg)
© 2010 VMware Inc. All rights reserved
AMQP Specification Sample
<method name="publish" content="1" index="40"> <chassis name="server" implement="MUST"/> <field name="reserved-1" type="short“ /> <field name="exchange" domain="exchange-name"/> <field name="routing-key" domain="shortstr"/> <field name="mandatory" domain="bit"/> <field name="immediate" domain="bit"/> </method>
![Page 14: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/14.jpg)
© 2010 VMware Inc. All rights reserved
AMQP Model
• Producers• Exchanges• Bindings• Queues• Consumers
![Page 15: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/15.jpg)
© 2010 VMware Inc. All rights reserved
Publishing: Setup
Exchanges, queues & bindings are created dynamically by producers & consumers
![Page 16: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/16.jpg)
© 2010 VMware Inc. All rights reserved
Publishing
![Page 17: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/17.jpg)
© 2010 VMware Inc. All rights reserved
Publishing
![Page 18: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/18.jpg)
© 2010 VMware Inc. All rights reserved
Publishing
Publishing a second message
![Page 19: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/19.jpg)
© 2010 VMware Inc. All rights reserved
Consuming
![Page 20: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/20.jpg)
© 2010 VMware Inc. All rights reserved
Consuming
Consuming second message
![Page 21: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/21.jpg)
© 2010 VMware Inc. All rights reserved
Publishing Multiple Messages
![Page 22: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/22.jpg)
© 2010 VMware Inc. All rights reserved
Publishing Multiple Messages
...to several consumers on the same queue
![Page 23: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/23.jpg)
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
![Page 24: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/24.jpg)
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
Sending
![Page 25: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/25.jpg)
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
Messages go to every queue bond to a fanout exchange
![Page 26: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/26.jpg)
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
Queues can be consumed from at different rates
![Page 27: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/27.jpg)
© 2010 VMware Inc. All rights reserved
Publishing: Topic Routing
* matches 1 word
# matches 0 or more words
![Page 28: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/28.jpg)
© 2010 VMware Inc. All rights reserved
Publishing: Topic Routing
![Page 29: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/29.jpg)
© 2010 VMware Inc. All rights reserved
AMQP Model Elements
Exchange Properties•passive•durable•auto-delete
Exchange Types•direct•fanout•topic•headers
Queue Properties•passive
•durable
•exclusive
•auto-delete
Message Properties•routing key
•immediate
•delivery-mode
•priority
•expiration
![Page 30: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/30.jpg)
© 2010 VMware Inc. All rights reserved
Prominent AMQP Implementations
RabbitMQ (Erlang)
OpenAMQ (C++)
Apache Qpid (Java and C++)
RedHat MRG (Messaging, Realtime & Grid) (C++)
![Page 31: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/31.jpg)
© 2010 VMware Inc. All rights reserved
Why Erlang?
• Concise (binary matching)• Sane concurrency (no shared state)• Built-in distributed database (Mnesia)• Multi-platform (Windows, Unix)
![Page 32: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/32.jpg)
© 2010 VMware Inc. All rights reserved
Erlang binary matching
decode_method('basic.publish', <<F0:16/unsigned, F1Len:8/unsigned, F1:F1Len/binary, F2Len:8/unsigned, F2:F2Len/binary, F3Bits:8>>)
->F3 = (F3Bits band 1) /= 0,F4 = (F3Bits band 2) /= 0, #'basic.publish'{ticket = F0, exchange = F1, routing_key = F2, mandatory = F3, immediate = F4};
![Page 33: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/33.jpg)
© 2010 VMware Inc. All rights reserved
What is RabbitMQ?
RabbitMQ consists of the broker (server) and several clients (Java, .net and others)
They speak the AMQP protocol to each other
Anyone can write other clients which also speak AMQP and work with RabbitMQe.g. Ruby, Java, Spring, Python, C, C#, Perl,
Erlang, Lisp, Haskell, PHP, <your language here>
It's in many popular Linux distributions and can easily be installed on OS X
Released under MPL
![Page 34: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/34.jpg)
© 2010 VMware Inc. All rights reserved
What is RabbitMQ?
17k lines of Erlang code in the brokerSupports clustering for increased
scalabilityCan work with Pacemaker and
associated tools to provide High Availability
Supports several extension points via plug-ins, e.g. exchange types and backing queue storage
![Page 35: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/35.jpg)
© 2010 VMware Inc. All rights reserved
AMQP compared to JMS
AMQP has a wire-level binary protocol, instead of an API
AMQP offers multi-language interoperability
AMQP message payloads are opaqueUse Spring-AMQP to migrate
![Page 36: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/36.jpg)
© 2010 VMware Inc. All rights reserved
Protocols
AMQP (SSL)STOMPXMPP (Rabbiter)0MQREST
![Page 37: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/37.jpg)
© 2010 VMware Inc. All rights reserved
Real-world features
•Unroutable messages•Client acknowledgements•Persistence•Transactions•Flow control•Security
![Page 38: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/38.jpg)
© 2010 VMware Inc. All rights reserved
Management & Monitoring
RabbitMQ plugin
• Inspect and manage resources
• Load & save configuration
• HTTP API
• Commandline tool
![Page 39: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/39.jpg)
© 2010 VMware Inc. All rights reserved
Management & Monitoring
![Page 40: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/40.jpg)
© 2010 VMware Inc. All rights reserved
Management & Monitoring
![Page 41: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/41.jpg)
© 2010 VMware Inc. All rights reserved
Cloud Ready
• Messaging service hosted in cloud• Cloud infrastructure services
• e.g. Nasa Nebula• Providing a service in the cloud
• e.g. SQS on Amazon
![Page 42: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/42.jpg)
© 2010 VMware Inc. All rights reserved
Nebula Architecture
![Page 43: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/43.jpg)
© 2010 VMware Inc. All rights reserved
UIDAI
![Page 44: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/44.jpg)
© 2010 VMware Inc. All rights reserved
UIDAI
Unique Identification Authority of India (UIDAI) will issue a12-digit for all residents
•One of the largest IT projects ever•Largest online identity DB ever•200 million messages per day peak•600 million residents over 4 years
• Spring• Tomcat• Hibernate• MySQL• Hadoop• Mule• Apache httpd• RabbitMQ
![Page 45: Easy enterprise application integration with RabbitMQ and AMQP](https://reader033.vdocuments.us/reader033/viewer/2022052215/554f5f1bb4c9058a148b4649/html5/thumbnails/45.jpg)
© 2010 VMware Inc. All rights reserved
The future
• AMQP 1.0• Connect-a-thon
• <Your suggestion here>• Get in touch!• http://www.rabbitmq.com/
This presentation will be available at
http://www.springsource.com/webinars
from Monday 13 Deccember 2010