20181013-olf-intro to sensu - ohio linuxfest 20192018/10/13 · sensu enterprise • sensu...
TRANSCRIPT
![Page 1: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/1.jpg)
Garrett Honeycutt [email protected]
learnpuppet.com @learnpuppet
2018-10-13 Ohio LinuxFest Columbus, OH
Intro to Sensu
![Page 2: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/2.jpg)
@learnpuppet
![Page 3: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/3.jpg)
@learnpuppet
![Page 4: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/4.jpg)
@learnpuppet
![Page 5: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/5.jpg)
@learnpuppet
0 - OK1 - Warning2 - Critical3 - Unknown
![Page 6: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/6.jpg)
/sensu/sensu-puppet
@learnpuppet
![Page 7: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/7.jpg)
@learnpuppet
![Page 8: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/8.jpg)
/sensu/sensu-puppet
@learnpuppet
![Page 9: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/9.jpg)
What is Sensu?
@learnpuppet
![Page 10: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/10.jpg)
Flexible monitoring framework
@learnpuppet
![Page 11: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/11.jpg)
Router for your metrics and alerts
@learnpuppet
![Page 12: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/12.jpg)
Status checks and telemetry at scale
@learnpuppet
![Page 13: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/13.jpg)
Designed for dynamic environments
@learnpuppet
![Page 14: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/14.jpg)
Why Sensu?
@learnpuppet
• Collect metrics and status checks with one system
![Page 15: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/15.jpg)
Why Sensu?
@learnpuppet
• Collect metrics and status checks with one system• Interface with other monitoring and metrics systems
![Page 16: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/16.jpg)
Why Sensu?
@learnpuppet
• Collect metrics and status checks with one system• Interface with other monitoring and metrics systems• Reuse existing Nagios plugins
![Page 17: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/17.jpg)
Why Sensu?
@learnpuppet
• Collect metrics and status checks with one system• Interface with other monitoring and metrics systems• Reuse existing Nagios plugins• Plugins can be written in any language
![Page 18: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/18.jpg)
Why Sensu?
@learnpuppet
• Collect metrics and status checks with one system• Interface with other monitoring and metrics systems• Reuse existing Nagios plugins• Plugins can be written in any language• Decoupled model for easy-to-scale architecture
![Page 19: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/19.jpg)
Why Sensu?
@learnpuppet
• Collect metrics and status checks with one system• Interface with other monitoring and metrics systems• Reuse existing Nagios plugins• Plugins can be written in any language• Decoupled model for easy-to-scale architecture• Great user community! (http://sensucommunity.slack.com/)
![Page 20: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/20.jpg)
@learnpuppet
•5+ Years•500 Checks, handlers, mutators•400 Contributors•1100 Pull requests•3000 commits
/sensu-plugins
![Page 21: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/21.jpg)
Monitoring at Scale
1:N execution scheduling
One execution request yields results from N clients
@learnpuppet
![Page 22: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/22.jpg)
Dynamic Environments
@learnpuppet
Monitoring ephemeral systems/resources
![Page 23: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/23.jpg)
@learnpuppet
Driven by configuration management
Dynamic Environments
![Page 24: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/24.jpg)
@learnpuppet
Open Source under MIT license
Sensu Core
![Page 25: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/25.jpg)
@learnpuppet
Written in Ruby
Sensu Core
![Page 26: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/26.jpg)
@learnpuppet
First commit July 10, 2011
Sensu Core
![Page 27: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/27.jpg)
@learnpuppet
1.0 released July 7th, 2017
Sensu Core
![Page 28: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/28.jpg)
@learnpuppet
2.0 alpha released February 27th, 2018
Sensu Core
![Page 29: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/29.jpg)
Sensu Enterprise
• Sensu Enterprise built on Sensu Core
• Drop-in replacement for sensu-server & sensu-api
• Proprietary license with commercial support
• Runs on JRuby for improved performance
• Adds enhanced integrations, contact routing and more
@learnpuppet
![Page 30: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/30.jpg)
Architecture Overview
• Transport
• Data Store
• Monitoring Agent (client)
• Event Processor (server)
• RESTful API
• Dashboard
@learnpuppet
![Page 31: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/31.jpg)
@learnpuppet
Message bus that enables decoupling
Transport
![Page 32: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/32.jpg)
@learnpuppet
Reduces need for service discovery
Transport
![Page 33: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/33.jpg)
RabbitMQ as Transport
• TLS with peer verification
• Multi-user with ACLs
• Clustering and federation (WAN)
@learnpuppet
![Page 34: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/34.jpg)
@learnpuppet
Sensu services are stateless
Data store
![Page 35: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/35.jpg)
Data Store
@learnpuppet
Redis
![Page 36: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/36.jpg)
Data Stored
@learnpuppet
Stored data is generally ephemeral
![Page 37: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/37.jpg)
Data Stored
@learnpuppet
![Page 38: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/38.jpg)
Data Stored
@learnpuppet
Some history for context
![Page 39: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/39.jpg)
Monitoring Agent
@learnpuppet
Sensu Core package provides sensu-client (agent)
![Page 40: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/40.jpg)
Check execution scheduler
Subscription (server)
• Sensu server publishes execution requests• Sensu agent executes check commands• Check result messages are published to transport
@learnpuppet
![Page 41: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/41.jpg)
Check execution scheduler
@learnpuppet
Stand alone (client)
• Sensu agent schedules check execution• Sensu agent executes check commands• Check result messages are published to transport
![Page 42: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/42.jpg)
Event Processor• Maintains client and incident registries
• Processes check result messages to create events
• Processes events according to configuration
• Filters improve signal-to-noise ratio
• Mutators transform event data format
• Handlers act on event data
@learnpuppet
![Page 43: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/43.jpg)
Check Result Life Cycle
@learnpuppet
![Page 44: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/44.jpg)
@learnpuppet
![Page 45: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/45.jpg)
RESTful API
@learnpuppet
No Click Fail
![Page 46: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/46.jpg)
RESTful API Endpoints/clients
/checks
/incidents
/results
/request
/silenced
@learnpuppet
![Page 47: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/47.jpg)
Dashboard
@learnpuppet
Uchiwa open source dashboard
![Page 48: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/48.jpg)
Uchiwa
@learnpuppet
Demo time
![Page 49: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/49.jpg)
Review: Sensu Flow• Check result is published to Transport
@learnpuppet
![Page 50: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/50.jpg)
Review: Sensu Flow• Check result is published to Transport
@learnpuppet
• Event processor (server) reads result from Transport• Data Store is updated based on result• Result combined with additional context becomes an
event• Filters, mutators and handlers evaluate and take action
on event
![Page 51: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/51.jpg)
Review: Sensu Flow• Check result is published to Transport
@learnpuppet
• Event processor (server) reads result from Transport• Data Store is updated based on result• Result combined with additional context becomes an
event• Filters, mutators and handlers evaluate and take action
on event
• API and Dashboard expose state from Data Store
![Page 52: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/52.jpg)
@learnpuppet
![Page 53: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/53.jpg)
@learnpuppet
![Page 54: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/54.jpg)
@learnpuppet
![Page 55: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/55.jpg)
@learnpuppet
![Page 56: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/56.jpg)
@learnpuppet
![Page 57: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/57.jpg)
Configuration Philosophy
@learnpuppet
inspired by UNIX-style chaining of tools
![Page 58: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/58.jpg)
Configuration Conventions
@learnpuppet
JSON configuration
/etc/sensu/config.json /etc/sensu/conf.d/**/*.json
![Page 59: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/59.jpg)
Sensu Primitives
@learnpuppet
![Page 60: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/60.jpg)
Checks
@learnpuppet
Monitor status or collect measurement
![Page 61: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/61.jpg)
Checks
@learnpuppet
Configured on agent or server
![Page 62: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/62.jpg)
Checks
@learnpuppet
Configuration on disk can provide client-specific overrides and/or security
![Page 63: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/63.jpg)
Example: Check Definition
$ cat /etc/sensu/conf.d/checks/check_haproxy.json{ "name": "check_haproxy", "command": "check_haproxy.rb -s app1 -w 40 -c 25", "subscribers": ["load_balancer"], "interval": 30 }
@learnpuppet
![Page 64: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/64.jpg)
Example: Check Result
{ "name": "check_haproxy", "command": "check_haproxy.rb -s app01 -w 40 -c 25", "subscribers": ["load_balancer"], "interval": 30, "timestamp": 1445569640, "output": "UP - 85% of 10 /app01/ services", "status": 0, "duration": 0.87 }
@learnpuppet
![Page 65: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/65.jpg)
Metric Checks
@learnpuppet
Collect measurements
![Page 66: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/66.jpg)
Metric Checks
@learnpuppet
Unlike status checks, OK events are sent to handlers
![Page 67: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/67.jpg)
Events
@learnpuppet
Created for every check result
![Page 68: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/68.jpg)
Events
• An event object is created is created for every check result
• Events with ok status update history in data store• Events should be passed to handlers when they:• Indicate an unhealthy state• Contain formatted data (metrics)• Change from an unhealthy state to healthy (resolve)
@learnpuppet
![Page 69: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/69.jpg)
Example: Event Data
{ "id": "42ccc91a-5449-4060-b7c3-c444db177fd7", "client": {"...": "..."}, "check": {"...": "..."}, "occurrences": 1, "action": "create", "timestamp": 1445569640 }
@learnpuppet
![Page 70: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/70.jpg)
Check hooks
@learnpuppet
Auto remediation
![Page 71: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/71.jpg)
@learnpuppet
{ "checks": { "nginx_process": { "command”: "check-process.rb -p nginx", "subscribers": ["proxy"], "interval": 30, "hooks": { "critical": { "command": "sudo /etc/init.d/nginx start", "timeout": 30 }, "non-zero": { "command": "ps aux", "timeout": 10 } } } }}
![Page 72: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/72.jpg)
Filters
@learnpuppet
Improve signal to noise
![Page 73: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/73.jpg)
Example: Filter Definition$ cat /etc/sensu/conf.d/filters/office_hours.json{ "filters": { "nine_to_fiver": { "negate": false, "attributes": { "timestamp": "eval: [1,2,3,4,5].include?(Time.at(value).wday) && Time.at(value).hour.between?(9,17)" } } } }
@learnpuppet
![Page 74: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/74.jpg)
Mutators
@learnpuppet
Transform event data
![Page 75: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/75.jpg)
Example: Mutator Definition$ cat /etc/sensu/conf.d/mutators/graphite_events.json
{ "mutators": { "graphite_events": { "command": "graphite_events.rb", "timeout": 10 } } }
@learnpuppet
![Page 76: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/76.jpg)
Mutator Example: graphite-event.rb
Input:
{ "id": "42ccc91a-5449-4060-b7c3-c444db177fd7", "client": { "name": "i-424242", ... }, "check": { "name": "check_haproxy", "status": 1, ... }, "occurrences": 1, "action": "create", "timestamp": 1445569640 }
Output:
sensu.i-424242.check_haproxy 1 1445569640
@learnpuppet
![Page 77: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/77.jpg)
Handlers
@learnpuppet
Take action on events
![Page 78: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/78.jpg)
Example: Handler Definition w/ Filter
$ cat /etc/sensu/conf.d/handlers/pushover.json{ "handlers": { "pushover": { "type": "pipe", "command": "handler-pushover.rb", "filters": ["only_state_change"] } } }
@learnpuppet
![Page 79: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/79.jpg)
Example: Handler Definition with Mutator$ cat /etc/sensu/conf.d/handlers/graphite_event.json{ "handlers": { "sensu_events_to_graphite": { "type": "tcp", "socket": { "host": "graphite.example.com", "port": 2003 }, "mutator": "graphite_events" } } }
@learnpuppet
![Page 80: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/80.jpg)
Sensu Primitives
@learnpuppet
![Page 81: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/81.jpg)
Sensu v2@learnpuppet
![Page 82: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/82.jpg)
Funded@learnpuppet
![Page 83: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/83.jpg)
Go@learnpuppet
![Page 84: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/84.jpg)
Self Contained
@learnpuppet
![Page 85: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/85.jpg)
Configuration shared between backends
@learnpuppet
![Page 86: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/86.jpg)
![Page 87: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/87.jpg)
# resource.json{ "type": "CheckConfig", "spec": { "name": "marketing-site", "command": "check-http.rb", "subscriptions": ["demo"], "interval": 15, "organization": "default", "environment": "default" }}
$ sensuctl create -f resource.json
![Page 88: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/88.jpg)
Recap@learnpuppet
![Page 89: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/89.jpg)
Flexible monitoring framework
@learnpuppet
![Page 90: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/90.jpg)
/sensu/sensu-puppet
@learnpuppet
![Page 91: 20181013-OLF-intro to sensu - Ohio LinuxFest 20192018/10/13 · Sensu Enterprise • Sensu Enterprise built on Sensu Core • Drop-in replacement for sensu-server & sensu-api](https://reader033.vdocuments.us/reader033/viewer/2022060416/5f139d3a1471fb0aaf6217b2/html5/thumbnails/91.jpg)
Garrett Honeycutt [email protected]
learnpuppet.com @learnpuppet
2018-10-13 Ohio LinuxFest Columbus, OH
Intro to Sensu