monitoring with sensu
DESCRIPTION
Overview of the Sensu monitoring frameworkTRANSCRIPT
![Page 1: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/1.jpg)
Monitoring with
![Page 2: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/2.jpg)
First of all...
![Page 3: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/3.jpg)
First of all...
![Page 4: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/4.jpg)
Why we (me) don’t like Nagios?
• No active-active cluster
![Page 5: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/5.jpg)
Why we (me) don’t like Nagios?
• No active-active cluster (nor easy active-passive one)
![Page 6: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/6.jpg)
Why we (me) don’t like Nagios?
• No active-active cluster (nor easy active-passive one)
![Page 7: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/7.jpg)
Why we (me) don’t like Nagios?
• No active-active cluster (nor easy active-passive one)
• Restart after adding a host (auto-scaling anyone??)
![Page 8: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/8.jpg)
Why we (me) don’t like Nagios?
• No active-active cluster (nor easy active-passive one)
• Restart after adding a host (auto-scaling anyone??)
• (Really) difficult to hack on
![Page 9: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/9.jpg)
Why we (me) don’t like Nagios?
• The master executes the checks (all of them!)
![Page 10: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/10.jpg)
Why we (me) don’t like Nagios?
• The master executes the checks (all of them!)
• Painful config reloads if using slaves
![Page 11: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/11.jpg)
Why we (me) don’t like Nagios?
• The master executes the checks (all of them!)
• Painful config reloads if using slaves
• (Quite) hard to configure
![Page 12: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/12.jpg)
Why we (me) don’t like Nagios?
• First release: March 14, 1999
![Page 13: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/13.jpg)
Why we (me) don’t like Nagios?
• First release: March 14, 1999
![Page 14: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/14.jpg)
![Page 15: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/15.jpg)
What’s Sensu?
• A monitoring framework
• They say “monitoring router”
• It takes results of checks and passes them to handlers
![Page 16: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/16.jpg)
Sensu facts
• Active-active cluster
![Page 17: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/17.jpg)
Sensu facts
• Active-active cluster
• Hosts automagically registered when agent starts
![Page 18: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/18.jpg)
Sensu facts
• Active-active cluster
• Hosts automagically registered when agent starts
• Written in Ruby
![Page 19: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/19.jpg)
Sensu facts
• Clients execute all the checks (all of them!)
![Page 20: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/20.jpg)
Sensu facts
• Clients execute all the checks (all of them!)
• No need to have slaves (but you can if you want to)
![Page 21: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/21.jpg)
Sensu facts
• Clients execute all the checks (all of them!)
• No need to have slaves (but you can if you want to)
• Configured with small JSON files
![Page 22: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/22.jpg)
Sensu facts
• Clients execute all the checks (all of them!)
• No need to have slaves (but you can if you want to)
• Configured with small JSON files
• Compatible with Nagios plugins!
![Page 23: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/23.jpg)
Sensu facts
• IT’S 2014
![Page 24: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/24.jpg)
Sensu facts
• IT’S 2014
![Page 25: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/25.jpg)
What does Sensu provide?
• Agent (client) to gather check results
• Server to take actions (or not) on these results
• REST API to query the state
• Optional dashboard to query the API
![Page 26: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/26.jpg)
What does Sensu NOT provide?
• Storage for the results history
• Notifications
• Graphing
![Page 27: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/27.jpg)
Architecture overview
![Page 28: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/28.jpg)
Two modes of operation
![Page 29: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/29.jpg)
Subscription
subscription messages
![Page 30: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/30.jpg)
Subscription
check request
![Page 31: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/31.jpg)
Subscription
check results
![Page 32: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/32.jpg)
Standalone
![Page 33: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/33.jpg)
Standalone
check results
![Page 34: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/34.jpg)
• Modes can be mixed
• The client also listens for results in a socket– Throw JSON to it from your application
![Page 35: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/35.jpg)
![Page 36: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/36.jpg)
Handlers
• Each check can have its own handler
• Sensu considers only results flagged as “metrics” and not-ok’s
![Page 37: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/37.jpg)
Handler types
• Pipe: the server executes a script and throws the event as JSON to the standard input
• TPC and UDP: the server connects to a host and port and delivers the event
• AMQP: the event can be reinjected in the broker for further routing
![Page 38: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/38.jpg)
![Page 39: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/39.jpg)
Handling events selectively
• What if you don’t want to handle some events?
• REST API to the rescue
![Page 40: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/40.jpg)
Stashes
• A stash is associated with an ID (a “path”)
• You can store here as much shit as you want
• Put a “silence” in the stash, and make your handlers check for it.
![Page 41: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/41.jpg)
Mutators
• A filter to modify event data before passing it to a handler.
• Example: manipulate events to make them suitable for graphite before passing them to a TCP handler
![Page 42: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/42.jpg)
Extensions
• Mutators and Handlers running in the same process than Sensu
• They must be written also in Ruby
• Run in the same event loop
![Page 43: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/43.jpg)
![Page 44: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/44.jpg)
Things missing / rant
• You can’t monitor something if you can’t install an agent on it.– There’s a ticket open since more than one year
ago...– Some patches, but still not a solution
• As with nagios, the state is decided by the plugins.
![Page 45: Monitoring with sensu](https://reader033.vdocuments.us/reader033/viewer/2022061103/53fe1d0b8d7f72db2d8b45ac/html5/thumbnails/45.jpg)
Further research
• http://sensuapp.org/docs/0.12
• https://github.com/sensu
• http://sensuapp.org/support/
• https://metacpan.org/pod/Sensu::API::Client