interoperability with rabbitmq

Interoperability with RabbitMQ

by Alvaro Videlaat Shanda Innovation - March 2010

Thursday, March 18, 2010

About Me

• Lead Developer at TheNetCircle.com

• Twitter: @old_sound

• Blog: http://obvioushints.blogspot.com/

• 老外

Thursday, March 18, 2010

The Interoperability

Problem

Thursday, March 18, 2010

The Problem

MoviePublisher

Thursday, March 18, 2010

The Problem

NewMovie

MoviePublisher

Thursday, March 18, 2010

The ProblemWebService

XMLJava

NewMovie

MoviePublisher

Thursday, March 18, 2010

The ProblemWebService

XML

Image ThumbnailGenerator

C++

Java

NewMovie

MoviePublisher

Thursday, March 18, 2010

The ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

C++

PHP

Java

NewMovie

MoviePublisher

Thursday, March 18, 2010

The ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

StaticHTML

Publisher

C++

PHP

PHP

Java

NewMovie

MoviePublisher

Thursday, March 18, 2010

The ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

StaticHTML

Publisher

C++

PHP

PHP

Java

TrailersGenerator

PHP

NewMovie

MoviePublisher

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Written In Different Languages

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Written In Different Languages

Different Operating Systems

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Written In Different Languages

Different Operating Systems

Written By Different People

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Written In Different Languages

Different Operating Systems

Written By Different People

They Need To Share Information

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Written In Different Languages

Different Operating Systems

Written By Different People

They Need To Share Information

They Need Real Time Data

Thursday, March 18, 2010

SYSTEMSThe ProblemWebService

XML

Image ThumbnailGenerator

PHP CacheGenerator

TrailersGenerator

StaticHTML

Publisher

Written In Different Languages

Different Operating Systems

Written By Different People

They Need To Share Information

They Need Real Time Data

They Need To Scale

Thursday, March 18, 2010

Is There a Solution?

Thursday, March 18, 2010

AMQPand

RabbitMQ

Thursday, March 18, 2010

AMQP

• Advanced Message Queuing Protocol

• Suits Interoperability

• Completely Open Protocol

• Binary Protocol

• AMQP Model

• AMQP Wire Format

Thursday, March 18, 2010

AMQP Model

• Exchanges

• Message Queues

• Bindings

• Rules for binding them

Thursday, March 18, 2010

AMQP Wire Protocol

• Functional Layer

• Transport Layer

Thursday, March 18, 2010

Message Flow

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html

Thursday, March 18, 2010

Exchange Types

• Fanout

• Direct

• Topic

Thursday, March 18, 2010

Default Exchanges

• amqp.fanout

• amqp.direct

• amqp.topic

Thursday, March 18, 2010

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-Fanout_Exchange.html

Thursday, March 18, 2010

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-Direct_Exchange.html

Thursday, March 18, 2010

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-Topic_Exchange.html

Thursday, March 18, 2010

Sample usage

Thursday, March 18, 2010

Sample Usage II

Thursday, March 18, 2010

Other Advantages

• Fire and Forget

• Stop/Start consumers safely (no need for last processed id)

• No need for Cronjobs locks

• Load Distribution

• Transactions

Thursday, March 18, 2010

RabbitMQ

Thursday, March 18, 2010

RabbitMQ

• Enterprise Messaging System

• Open Source MPL

• Written in Erlang/OTP

• Commercial Support

Thursday, March 18, 2010

Features

• Reliable and High Scalable

• Easy To install

• Easy To Cluster

• Runs on: Windows, Solaris, Linux, OSX

• AMQP 0.8

Thursday, March 18, 2010

Client Libraries

• Java

• .NET/C#

• Erlang

• Ruby, Python, PHP, Perl, AS3, Lisp, Scala

Thursday, March 18, 2010

Docs/Support

• http://www.rabbitmq.com/documentation.html

• http://dev.rabbitmq.com/wiki/

• #rabbitmq at irc.freenode.net

• http://www.rabbitmq.com/email-archive.html

Thursday, March 18, 2010

Our Setup:

Publishers - 28 PHP Frontend Machines

2 PHP Machinesrunning 8 consumers each

Thursday, March 18, 2010

Our Setup

• Deployed in mid 2009

• Reduced site SPOF

• 400.000+ messages routed a day (and increasing)

• Very easy to install/deploy

Thursday, March 18, 2010

Easy To Install?

Thursday, March 18, 2010

Easy To Install?

http://gist.github.com/334219

Thursday, March 18, 2010

Show me the code!

Thursday, March 18, 2010

Resources:

• http://www.rabbitmq.com/

• https://www.amqp.org/confluence/display/AMQP/About+AMQP

• http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/index.html

• http://wso2.org/library/articles/using-axis2-c-amqp-transport-part-i

• http://www.skaag.net/2010/03/12/rabbitmq-for-beginners/

• http://github.com/careo/rabbitmq-erlang-client-examples

Thursday, March 18, 2010

Questions?

Thursday, March 18, 2010

Thanks!Alvaro Videla

http://twitter.com/old_sound

http://www.slideshare.net/old_sound

Thursday, March 18, 2010