nagios conference 2012 - alexis le quoc - deep dive into nagios analytics
DESCRIPTION
Alexis Le Quoc's presentation on Diving into Nagios Analytics The presentation was given during the Nagios World Conference North America held Sept 25-28th, 2012 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcnaTRANSCRIPT
![Page 2: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/2.jpg)
@alqDev & OpsNagios user since 2008Datadog co-founder
![Page 3: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/3.jpg)
A little survey
![Page 4: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/4.jpg)
Top 3 failed checks
![Page 5: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/5.jpg)
Top 3 failed checks
That I responded tolast week
That woke me up
That most of my teamresponded to at least once
That impacts our businessthe most?
That I responded to5 weeks ago
![Page 6: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/6.jpg)
Top 3 failed checks
That I responded tolast week
That woke me up
That most of my teamresponded to at least once
That impacts our businessthe most?
That I responded to5 weeks ago
![Page 7: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/7.jpg)
Using memory to prioritize
remediation...
At best, finding local optimums
At worst, brownian motion
![Page 8: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/8.jpg)
Analytics
![Page 9: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/9.jpg)
Performance Metrics
Nagios TrafficOther Sources
In the “Cloud”
![Page 10: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/10.jpg)
Nagios a “chatty” source
out of 40+ Datadog supports
![Page 11: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/11.jpg)
One example
![Page 12: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/12.jpg)
![Page 13: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/13.jpg)
Almost 13000 Nagios “events”over past week
![Page 14: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/14.jpg)
Constant stream
![Page 15: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/15.jpg)
86 notifications!
![Page 16: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/16.jpg)
Pattern
![Page 17: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/17.jpg)
Pattern
![Page 18: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/18.jpg)
More data?More questions.
![Page 19: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/19.jpg)
A dialog with dataNot a scientific study
![Page 20: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/20.jpg)
Population
25% 50% 75% 100% 20 93 322 904
![Page 21: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/21.jpg)
Does size matter?
![Page 22: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/22.jpg)
Weekly Count per host split by quartile
![Page 23: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/23.jpg)
Weekly count per host split by quartile
Outliers Sick hosts,
silenced checks
![Page 24: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/24.jpg)
Notifications
![Page 25: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/25.jpg)
Notifications1-3% of alerts notify
Little difference per quartile
![Page 26: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/26.jpg)
Does time of day matter?
![Page 27: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/27.jpg)
![Page 28: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/28.jpg)
Mean about the sameacross quartiles
Time-based deviation?
![Page 29: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/29.jpg)
Does the day of week matter?
![Page 30: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/30.jpg)
![Page 31: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/31.jpg)
Not really
![Page 32: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/32.jpg)
Squeaky wheels? (checks)
![Page 33: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/33.jpg)
Outlier
![Page 34: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/34.jpg)
Outlier in more detail
![Page 35: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/35.jpg)
Long Tail
![Page 36: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/36.jpg)
Squeaky wheel?(hosts)
![Page 37: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/37.jpg)
Same outlier
![Page 38: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/38.jpg)
Similar pattern as checks
![Page 39: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/39.jpg)
Long Tail
![Page 40: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/40.jpg)
Recurring alerts
![Page 41: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/41.jpg)
Young Old
Seldom happen
s
Happens
Often
![Page 42: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/42.jpg)
Happen once in a while
Occur often, for a long time Tolerated
![Page 43: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/43.jpg)
More data?More questions.
![Page 44: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/44.jpg)
HOWTO?
![Page 45: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/45.jpg)
Find out tomorrow!Awk
Postgres
R
d3
![Page 46: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/46.jpg)
Presentation matters
![Page 47: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/47.jpg)
![Page 48: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/48.jpg)
Take-away?
![Page 49: Nagios Conference 2012 - Alexis Le Quoc - Deep Dive into Nagios Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062320/5590a2381a28abf8788b4645/html5/thumbnails/49.jpg)
Take-aways
•Don’t rely on your memory
•Your Nagios logs are a treasure trove
•Have a dialog with your data
•Presentation matters