smart energy systemsaiellom/pdf/5 messaging.pdfspring client-server syncronous communication...
TRANSCRIPT
![Page 2: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/2.jpg)
Client-server
syncronouscommunication
communicationinitiated by client
spikes in server load
messages get lost
client must knowserver‘s location
server must be up and running, ready for requests
client server
![Page 3: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/3.jpg)
Indirect communication
![Page 4: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/4.jpg)
Messaging
queue
broker
consumerproducer
syncronouscommunication
communicationinitiated by client
spikes in server load
messages get lost
client must knowserver‘s location
server must be up and running, ready for requests
message message
![Page 5: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/5.jpg)
Messaging
queue
broker
consumerproducer
producer
producer
consumer
consumer
![Page 6: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/6.jpg)
AMQPAdvanced Message Queuing Protocol
![Page 7: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/7.jpg)
AMQP
producer consumerqueue AMQP connection
channel
channel
chan
chan
exchange
broker
virtual host
binding
![Page 8: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/8.jpg)
RabbitMQ
• Erlang implementation of AMQP
• AMQP v0.9.1 with custom extensions
• Data persistance– In-memory/file-persisted embeded database
– Specific message storage and index files
• Clustering– Erlang´s capabilities
• Plugins– Web-based management
![Page 9: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/9.jpg)
Tips
• Connecting to RMQ
– Your application should start irrespective ofwhether a connection to RMQ can be establishedor not
– If the connection to RMQ is lost, your applicationshould reconnect by itself
– If the connection is down, producing/consumingmessages should fail gracefully
![Page 10: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/10.jpg)
AMQP Channels
• AMQP connection is a TCP connection
• A channel is a logical connection
– commands are sent via channels
– all channels reuse the same TCP connection
– hundreds of thousands of new channels a second
AMQP connection
channel
channel
chan
chan
![Page 11: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/11.jpg)
Tips
• Open a channel, do something with thechannel, close the channel
• Channels instances are technically threadsafe– It is recommended to avoid having several
threads using the same channel concurrently
• Channels are terminated by exceptions– Cascade of failures: if some part of the program
tries to use a terminated channel, it will fail too
![Page 12: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/12.jpg)
AMQP Message
properties:content-type::stringcontent-encoding::stringdelivery-mode::1|2priority::0-9correlation-id::stringreply-to::stringexpiration::stringmessage-id::stringtimestamp::timestamptype::stringuser-id::stringapp-id::stringreserved::nullheaders::map[string,object]
body:byte[]
• No information about the producer and consumer
![Page 13: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/13.jpg)
RMQ Message
• Two parts
– Payload
• Anything
– Label
• Describes the payload
• Information about where the message should go– Exchange name
– Topic/routing key (optional)
![Page 14: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/14.jpg)
AMQP Broker
![Page 15: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/15.jpg)
Queue
• Place for messages ready to be consumed
• How to create a queue?– queue.declare
– Queue name• Used by consumers to subscribe to the queue• Used when creating a binding• Otherwise, RMQ assigns a random name
– Other properties• durable
• exclusive
• auto-delete
• arguments
![Page 16: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/16.jpg)
Queue
• Who can create a queue?
– Both consumers and producers
– Consumers can’t create a queue while subscribed to another one on the same channel
• What if a queue already exists when trying to declare it?
– Idempotent if the properties are exactly the same
![Page 17: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/17.jpg)
Tips
• Messages published into an exchange without a queue to be routed to are discarded by RMQ
• Who should create the queue?– Can you afford losing messages created by your
producers? If not, both consumers and producers should create the needed queue
• If you want to check the existence of a queue, set the passive option of queue.declare to true– check then act pattern discouraged
![Page 18: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/18.jpg)
Queue and consumers
• Messages are sent immediately to consumers subscribed to the queue– basic.consume
– basic.get
• Consumer acknowledges a message iff it is done with processing (to RMQ and not to producer!)– basic.ack
– auto_ack parameter set to true when subscribing to a queue (acceptable risk of losing messages)
• Reject a message (rather than acknowledge it)– basic.reject
![Page 19: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/19.jpg)
Tips
• Use basic.consume if the consumer is processing many messages out of a queue
• Use basic.consume if the consumer needs to automatically receive messages from a queue as soon as they arrive
• Don’t use basic.get in a loop as an alternative to basic.consume (performance overhead)
• Don´t forget to acknowledge a message, RMQ won´t send that consumer any more messages
![Page 20: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/20.jpg)
Exchanges and bindings
• Exchange– Place where producers send messages
• Routing key– Rules upon which RMQ decides to which queue
it should deliver the message
• Binding– A queue is said to be bound to an exchange by a
routing key
![Page 21: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/21.jpg)
Direct exchange
• Routing algorithm: if the routing key of a message matches therouting key of a queue, then the message is delivered to thatqueue
• If Q2 is bound to X by the routing key black, then all messageswith the routing key black will be deliverd to Q2
![Page 22: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/22.jpg)
Direct exchange
• A default exchange
• Empty string as its name
• A newly declared queue will beautomatically bound to this exchangeusing the queue name as routing key
• channel.basic_publish(exchange=´´,
routing_key=´queue-name´, body=msg)
![Page 23: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/23.jpg)
Fanout exchange
• Routing algorithm: a message isdelivered to all queues bound to theexchange (broadcast)
• Routing keys are ignored
![Page 24: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/24.jpg)
Topic exchange
• Routing algorithm: if the routing key of a message matches the routing keyof one or more queues, then the message is deliverd to all the queues
• Routing key must be a list of words delimeted by dots• * can substitute for exactly one word• # can substitute for zero or more words
![Page 25: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/25.jpg)
Topic exchange: example
• timeseries.<building>.<floor>.<room>.<area>.<type-id>.<instance-id>
• timeseries.iaas.0.353,window.light.12345
• Exchange– Name: timerseris.exchange.topic– Type: topic
• Queue– Name: timeseries.all– Type: durable– Routing key: timerseries.#
![Page 26: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/26.jpg)
Virtual host
• Logical subdivision of RMQ´s execution space– Virtual message broker– Its own queues, exchanges and bindings– Its own persmissions
• Allow for multiple applications to one RMQ server
• Default vhost called /
• Vhosts are to RMQ what VMs are to physical servers
![Page 27: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/27.jpg)
Tips
• Identify common functionality groups in your application and give each one itsown vhost
• Use the default one for testing
– guest/guest as credentials
– Guest has full permissions
![Page 28: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/28.jpg)
Reliability: durability
• Queues and exchanges together with messagesdon´t survive a reboot of the RBM server
• The durable property of queues and exchanges isset to false by default
• For messages to survive reboot, queues and exchanges must be durable, but this is not sufficient– Messages must be persistent– Set the delivery-mode option of the message to 2
![Page 29: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/29.jpg)
Tips
• Should you use persistent messaging forall messages?
– Writing a message to disk is much slower thansaving it in RAM
– Slows down RMQ in terms of the number ofmessages it can process per second
![Page 30: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/30.jpg)
Reliability: redundancy
• Multiple RMQ instances
– Performance
– Scalability
– Fault tolerance
![Page 31: Smart Energy Systemsaiellom/pdf/5 Messaging.pdfSpring Client-server syncronous communication communication initiated by client spikes in server load messages get lost client must know](https://reader036.vdocuments.us/reader036/viewer/2022070721/5ee16f87ad6a402d666c5224/html5/thumbnails/31.jpg)
References
• Coulouris, G., Dollimore, J., Kindberg, T., Blair, G., Distributed Systems: Concepts and Designs. Fifth Edition.
• Videla, A. and Williams, J. W. RabbitMQ in Action: Distributed messaging for everyone. Manning Publications Co. 2012.
• Dossot, D. RabbitMQ Essentials. Packt Publishing. 2014.
• RabbitMQ Tutorial, https://www.rabbitmq.com/tutorials/.