simplify, isolate and scale with rabbitmq
TRANSCRIPT
Message Queues• Simplify: Reduce Complexity
• Isolate: Decouple Systems
• Scale: Enable Scalability
John Blackmore - @johnblackmore
Simple Queue
• Producer sends messages to a queue
• Consumer receives messages from a queue
• Queue buffers messages un8l they are processed
John Blackmore - @johnblackmore
Why is this Useful?• "Fire and Forget" Commands
• Decoupling Systems
• Asynchronous Processing
• Parallel Processing
• Fault Tolerance
John Blackmore - @johnblackmore
Exchange Types
Most common Exchange types:
• Direct: Messages are routed to queues along bindings that exactly match the rou9ng key.
• Fanout: Messages are routed to all bound queues, rou9ng key ignored.
• Topic: Messages are routed to queues along bindings that pa@ern match the rou9ng key.
John Blackmore - @johnblackmore
New User Emails• As part of new registra0on a welcome email is sent
• Sending the email takes 1-3 seconds on average
• Under high traffic, the web server can max-out
Underlying Problem:
• Sending email can take a long 0me
John Blackmore - @johnblackmore
New User Emails - Exis/ng Code// register the new user, then send the welcome emailmail('[email protected]', 'Welcome to GroupBuyWidgets!', 'Thanks for signing up.');
• Pro: Simple, easy to read
• Con: Takes a long .me, Blocks un.l email is sent
Solu%on:
• Queue the emails for asynchronous processing
John Blackmore - @johnblackmore
New User Emails - Diagram
• Producer: Registra1on Script on Web Server
• Consumer: Background Worker on Applica7on Server
• Direct Exchange with Single Queue
John Blackmore - @johnblackmore
New User Emails - Produceruse PhpAmqpLib\Connection\AMQPStreamConnection;use PhpAmqpLib\Message\AMQPMessage;
// Connect to RabbitMQ and open a channel$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');$channel = $connection->channel();
// We will push onto the "email" exchange$channel->exchange_declare('email', 'direct', false, false, false, false, false);
// Compose the message i.e the email.$payload = json_encode([ 'to' => '[email protected]', 'subject' => 'Welcome to GroupBuyWidgets!', 'body' => 'Thanks for signing up.',]);
// Wrap the payload and publish to the exchange$message = new AMQPMessage($payload, ['delivery_mode' => 2]);$channel->basic_publish($message, 'email', 'email.send');
John Blackmore - @johnblackmore
New User Emails - Benchmarks
Apache Benchmark, 100 requests, 5 connec6ons:
Non Queued==========Time taken for tests: 41.232 secondsRequests per second: 2.43 [#/sec] (mean)Time per request: 2061.575 [ms] (mean)
Queued======Time taken for tests: 6.186 secondsRequests per second: 16.17 [#/sec] (mean)Time per request: 309.310 [ms] (mean)
John Blackmore - @johnblackmore
Batch Processing Payments
Problem:
• Lots of data records to process.
• Processing is 3me cri3cal.
Solu%on:
• Parallel Processing
• Mul%ple Consumers
John Blackmore - @johnblackmore
Batch Processing Payments
Number of Consumers | Time to Process-----------------------|---------------------- 1 | 4 - 7 Hours 10 | 25 - 40 Minutes 100 | < 4 Minutes
John Blackmore - @johnblackmore
More RabbitMQ Examples• Publish/Subscribe - Sending the same message to
mul7ple queues.
• Rou7ng & Topic - Receiving messages selec7vely, or based on a paAern.
• Remote Procedure Calls - Receiving replies using anonymous exclusive callback queues.
h"p://www.rabbitmq.com/getstarted.html
John Blackmore - @johnblackmore
Ques%ons?
Twi$er: @johnblackmoreWeb: johnblackmore.com
Email: [email protected]
github.com/johnblackmore/rabbitmq-examples-php
John Blackmore - @johnblackmore
Sources• RabbitMQ Tutorials
• AMQP 0-9-1 Model Explained
• Ge=ng Started with RabbitMQ and CloudAMQP
• DigitalOcean - Install and Manage RabbitMQ
John Blackmore - @johnblackmore