serverspec and sensu - testing and monitoring collide
DESCRIPTION
Lightning talk from Sydney DevOps MeetupTRANSCRIPT
![Page 1: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/1.jpg)
+
MICHAEL RICHARDSON - @M_RICHO
Testing and Monitoring Collide
![Page 2: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/2.jpg)
WHAT IS IT?
RSpec Tests for checking your servers are configured correctly
![Page 3: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/3.jpg)
CM/PROVISIONING TOOL AGNOSTIC
![Page 4: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/4.jpg)
CM/PROVISIONING TOOL AGNOSTIC
SERVERS CAN BE BUILT USING• CHEF• PUPPET• CF-ENGINE• ANSIBLE
• SALT STACK• SHELL SCRIPTS• MANUAL HACK• WHAT EVER.
![Page 5: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/5.jpg)
26 RESOURE TYPES INCLUDING• COMMAND• CRON• DEFAULT_GATEWAY• FILE• GROUP• HOST• INTERFACE
• IPTABLES• KERNEL_MODULE• PACKAGE• PORT• PROCESS• SERVICE• USER
![Page 6: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/6.jpg)
![Page 7: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/7.jpg)
That’s super fast
![Page 8: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/8.jpg)
WHEN TO RUN SERVERSPEC TESTS?
??
![Page 9: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/9.jpg)
WHEN TO RUN SERVERSPEC TESTS?
That’s super useful
• When developing or updating config management code (eg puppet/chef)
• When validating a node or environment
![Page 10: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/10.jpg)
WHEN TO RUN SERVERSPEC TESTS?
Hell yeah. That’s a great idea.
• What about every 2 minutes on all hosts orchestrated via your monitoring tool
![Page 11: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/11.jpg)
WHEN TO RUN SERVERSPEC TESTS?
Hell yeah. That’s a great idea.
• What about every 2 minutes on all hosts orchestrated via your monitoring tool
Why?• Fast• Lightweight• Overlaps with traditional
monitoring checks. (Eg. Is the apache service running)
• Simplifies monitoring tool config
![Page 12: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/12.jpg)
WE USE SENSU FOR MONITORING
It’s • Simple• Extensible• Lightweight• Simple to install on linux and windows.
![Page 13: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/13.jpg)
ITERATION 1+GET SENSU TO JUST RUN THE RSPEC TESTS
![Page 14: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/14.jpg)
ITERATION 1+
RSpec stdout isn’t great to simply pass back to a monitoring tool.
![Page 15: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/15.jpg)
ITERATION 1+SURE, WE CAN TIDY UP RSPEC STD OUT
But now have no idea which tests are failing
![Page 16: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/16.jpg)
2 USEFUL FEATURES TO THE RESCUE
1. Rspec has a JSON format option
2. Sensu-Client has a local socket to accept ‘arbitrary’ check results
![Page 17: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/17.jpg)
RSPEC JSON OUTPUT
4 exampletests
Summary results
![Page 18: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/18.jpg)
SENSU CLIENT SOCKET
Send results here
Failures appear here
![Page 19: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/19.jpg)
ITERATION 2+
![Page 20: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/20.jpg)
ITERATION 2+RSpec in json format
Send each test result to sensu socket
Report summary serverspec results to back to sensu defined check
Iterate over results
![Page 21: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/21.jpg)
ITERATION 2+
Summary results Individual test failures
![Page 22: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/22.jpg)
SUMMARY
1. Write comprehensive tests2. Don’t duplicate effort by writing the same thing
again as a monitoring check.3. Simply run your tests via a monitoring tool
![Page 23: Serverspec and Sensu - Testing and Monitoring collide](https://reader033.vdocuments.us/reader033/viewer/2022061104/540ddd4b8d7f72747e8b4bb4/html5/thumbnails/23.jpg)
THANK YOU
Michael Richardson - @m_richohttps://github.com/m-richo/sensu_check-rspec