rabbitmq is the new king
DESCRIPTION
Speakers: Jan Machacek and Alvaro Videla Jan will show how to use RabbitMQ to connect components on different platforms. After a few introductory slides explaining the main concepts, the rest of the talk will be live code. The coding will begin by showing Spring Integration code to connect to RabbitMQ to send and receive messages--think byte[] values back and forth to start with. Jan will then show the real power of RabbitMQ by replacing the RPC server component by native code in C++: we will write image processing component. To make matters even better--faster--we will then use CUDA to perform the image processing. Come to Jan's talk and demo if you want to find out how to use RabbitMQ in the core of your application, especially if your application needs to tie together code on completely different platforms and if it performs heavy number crunching or image processing. You do not need to have deep knowledge of Spring Integration, AMQP or OpenCV, …; all that you will need is your inner geek.TRANSCRIPT
© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
RabbitMQ is the new kingJan Machacek | Alvaro Videla
Jan Machacek
• Works at Cake Solutions
• Author of Pro Spring, Pro Spring 2.5 and other books & articles
• Author of Akka Patterns, Specs2 Spring, Scalad;Spring Extensions, Spock Spring Integration
• Editor of the Open Source Journal
• @honzam399 | [email protected]/janm399
Alvaro Videla
• Developer Advocate at Pivotal / RabbitMQ
• Co-Author of RabbitMQ in Action
• Creator of the RabbitMQ Simulator
• Blogs about RabbitMQ Internals: http://videlalvaro.github.io/internals.html
• @old_sound | [email protected]/videlalvaro
Why do we need messaging?
Classic Web Apps
Implement a Photo Gallery
Two Parts:
Pretty Simple
‘Till new requirements arrive
The Product Owner
Can we also notify the user friends when she uploads a new
image?
Can we also notify the user friends when she uploads a new
image?
I forgot to mention we need it for tomorrow…
The Social Media Guru
We need to give badges to users for each picture upload
We need to give badges to users for each picture upload
and post uploads to Twitter
The Sysadmin
Dumb! You’re delivering full size images!
The bandwidth bill has tripled!
Dumb! You’re delivering full size images!
The bandwidth bill has tripled!
We need this fixed for yesterday!
The Developer in the other team
I need to call your Java stuff but from Python
I need to call your PHP stuff but from Python
And also PHP starting next week
The User
I don’t want to waittill your app resizes
my image!
You
(╯°□°）╯（ ┻━┻
Let’s see the code evolution
%% image_controllerhandle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
Pseudo Code
Comments
%% image_controllerhandle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
Pseudo Code
Function Name
%% image_controllerhandle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
Pseudo Code
Arguments
%% image_controllerhandle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
Pseudo Code
Function Body
%% image_controllerhandle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
Pseudo Code
Return Value
%% image_controllerhandle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
First Implementation:
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image),
ok.
Second Implementation:
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image),
notify_friends(ReqData:get_user()),ok.
Third Implementation:
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image),
notify_friends(ReqData:get_user()),add_points_to_user(ReqData:get_user()),ok.
Fourth Implementation:
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image),
notify_friends(ReqData:get_user()),add_points_to_user(ReqData:get_user()),tweet_new_image(User, Image),ok.
Final Implementation:
Can our code scale to new requirements?
What if
• We need to speed up image conversion
What if
• We need to speed up image conversion• User notifications sent by email
What if
• We need to speed up image conversion• User notifications sent by email• Stop tweeting about new images
What if
• We need to speed up image conversion• User notifications sent by email• Stop tweeting about new images• Resize in different formats
What if
• We need to speed up image conversion• User notifications sent by email• Stop tweeting about new images• Resize in different formats• Swap Language / Technology (No Down Time)
What if
Can we do better?
Sure.Using messaging
DesignPublish / Subscribe Pattern
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),
Msg = #msg{user = ReqData:get_user(), image = Image},publish_message('new_image', Msg).
First Implementation:
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),
Msg = #msg{user = ReqData:get_user(), image = Image},publish_message('new_image', Msg).
First Implementation:
%% friends notifieron('new_image', Msg) ->
notify_friends(Msg.user, Msg.image).
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),
Msg = #msg{user = ReqData:get_user(), image = Image},publish_message('new_image', Msg).
First Implementation:
%% friends notifieron('new_image', Msg) ->
notify_friends(Msg.user, Msg.image).
%% points manageron('new_image', Msg) ->
add_points(Msg.user, 'new_image').
%% image_controllerhandle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),
Msg = #msg{user = ReqData:get_user(), image = Image},publish_message('new_image', Msg).
First Implementation:
%% friends notifieron('new_image', Msg) ->
notify_friends(Msg.user, Msg.image).
%% points manageron('new_image', Msg) ->
add_points(Msg.user, 'new_image').
%% resizeron('new_image', Msg) ->
resize_image(Msg.image).
Second Implementation:
Second Implementation:
THIS PAGE INTENTIONALLY LEFT BLANK
What is RabbitMQ
RabbitMQ
RabbitMQ
• Multi Protocol Messaging Server
RabbitMQ
• Multi Protocol Messaging Server• Open Source (MPL)
RabbitMQ
• Multi Protocol Messaging Server• Open Source (MPL)• Polyglot
RabbitMQ
• Multi Protocol Messaging Server• Open Source (MPL)• Polyglot• Written in Erlang/OTP
Multi Protocol
http://bit.ly/rmq-protocols
Polyglot
Polyglot
• Java
Polyglot
• Java• node.js
Polyglot
• Java• node.js• Erlang
Polyglot
• Java• node.js• Erlang• PHP
Polyglot
• Java• node.js• Erlang• PHP• Ruby
Polyglot
• Java• node.js• Erlang• PHP• Ruby• .Net
Polyglot
• Java• node.js• Erlang• PHP• Ruby• .Net• Haskell
Polyglot
Even COBOL!!!11
Some users of RabbitMQ
Some users of RabbitMQ
Some users of RabbitMQ
• Indeed.com
Some users of RabbitMQ
• Indeed.com
• MailboxApp
Some users of RabbitMQ
• Indeed.com
• MailboxApp
• Mercado Libre
Messaging with RabbitMQ
A demo with the RabbitMQ Simulator
https://github.com/RabbitMQSimulator/RabbitMQSimulator
What are we building?
What are we building?
AMQP
Server
CV
What are we building?
AMQP
Server
CV
onCoinResponse(CorrelationId, CoinResponse): Unit
RecogServiceActivatormjpegChunk(CorrelationId) (ChunkData): UnitimageChunk(CorrelationId) (ChunkData): Unit
RecogService
Demo
int:chain
ChunkData Collection[FrameData]FrameData
Array[Byte] StringFrameData
FrameDataFrameData
int:chain
decodeFrame(ChunkData): Collection[FrameData]
ChunkDecoder
DefaultMessageSplitter
AmqpOutboundEndpoint
ObjectToStringTransformer
ServiceActivatingHandler
ServiceActivatingHandler
onCoinResponse(CorrelationId, CoinResponse): Unit
RecogServiceActivator
mjpegChunk(ChunkData): UnitimageChunk(ChunkData): Unit
recogChannel: MessageChannelRecogService
DirectChannel
mjpegChunk(CorrelationId) (ChunkData): UnitimageChunk(CorrelationId) (ChunkData): Unit
RecogService
mjpegChunk(SessionId, ChunkData): UnitimageChunk(SessionId, ChunkData): Unit
recogService: RecogServiceRecogController
sessionEnded(SessionId): UnitonCoinResponse(CorrelationId, CoinResponse): Unit
sessions: Map[SessionId, CoinResponse]messageSender: MessageSendingOperations[...]
RecogSessions
onCoinResponse(CorrelationId, CoinResponse): Unit
recogSessions: RecogSessionsRecogServiceActivator
SubscribableChannel dispatchChannel
SubscribableChannel webSocketHandlerChannel
MessagingWebSocketHandler
AnnotationMessageHandler
SimpleBrokerMessageHandler
RecogSessions
RecogController
send
send
send
send
subscribe
subscribe
send
send
subscribeMessageSendingOperations[] DispatcherServlet
HTTP
WebSocket
SubProtocolWebSocketHandler
RecogService
Questions
https://github.com/eigengo/springone2gx2013
