configuration management and #monitoringlove
DESCRIPTION
I will tell a story about how we changed from our old monitoring monolith to monitoring based on Sensu and friends driven by configuration management, how #monitoringsucks become #monitoringlove for us. Monitoring in virtual environments like the cloud has changed the way how monitoring solution needs to work and look like. The configuration of monitoring needs to be driven by configuration management and not by discovering or manually processes via UIs. The monitoring solutions needs to be flexible, scalable and easy to configure. I will tell how we have done at Recorded Future to implement a state-of-the-art solution based on Sensu, RabbitMQ, Redis, Graphite, collectd, mcollective, etc driven by Chef.TRANSCRIPT
![Page 1: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/1.jpg)
Configuration management and #monitoringlove
#devops
#monitoringlove
#opschef
#dadops
@ulfmansson the #swedishchef
![Page 2: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/2.jpg)
![Page 3: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/3.jpg)
![Page 4: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/4.jpg)
![Page 5: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/5.jpg)
![Page 6: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/6.jpg)
![Page 7: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/7.jpg)
![Page 8: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/8.jpg)
![Page 9: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/9.jpg)
![Page 10: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/10.jpg)
![Page 11: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/11.jpg)
![Page 12: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/12.jpg)
![Page 13: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/13.jpg)
![Page 14: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/14.jpg)
![Page 15: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/15.jpg)
![Page 16: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/16.jpg)
![Page 17: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/17.jpg)
![Page 18: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/18.jpg)
![Page 19: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/19.jpg)
![Page 20: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/20.jpg)
![Page 21: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/21.jpg)
![Page 22: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/22.jpg)
![Page 23: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/23.jpg)
![Page 24: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/24.jpg)
![Page 25: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/25.jpg)
![Page 26: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/26.jpg)
![Page 27: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/27.jpg)
![Page 28: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/28.jpg)
![Page 29: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/29.jpg)
![Page 30: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/30.jpg)
![Page 31: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/31.jpg)
![Page 32: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/32.jpg)
![Page 33: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/33.jpg)
![Page 34: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/34.jpg)
![Page 35: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/35.jpg)
![Page 36: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/36.jpg)
![Page 37: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/37.jpg)
![Page 38: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/38.jpg)
![Page 39: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/39.jpg)
Some example from Monigusto – monitoring up by @patrickdebois
git clone https://github.com/monigusto/vagrantmonigusto.git
![Page 40: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/40.jpg)
Handler config
sensu_handler "default" do
type "set"
handlers [ "pagerduty", “mail” ]
end
sensu_handler "pagerduty" do
type "pipe"
command "pagerduty.rb"
severities ["ok", "critical"]
end
![Page 41: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/41.jpg)
Creating a check
sensu_check "redis_process" do
command "checkprocs.rb p redisserver C 1"
handlers ["default"]
subscribers ["redis"]
interval 30
additional(:notification => "Redis is not running")
end
![Page 42: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/42.jpg)
Sensu admin (state, db)
![Page 43: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/43.jpg)
Questions?
Community Track – Bayview B
The Sensu team (- Sean Porter)
![Page 44: Configuration management and #monitoringlove](https://reader033.vdocuments.us/reader033/viewer/2022042714/5558233bd8b42a25588b4c8c/html5/thumbnails/44.jpg)
Sensu-dashboard (stateless, no db)