data science the solution to #monitoringsucks
TRANSCRIPT
Data Science: The Solution to #monitoringsucks
#DevOpsDays Amsterdam 2015
Who am I?
● Operations Engineer @ STYLIGHT GmbH
Who am I?
● Operations Engineer @ STYLIGHT GmbH
● 11+ years as a System Administrator
Who am I?
● Operations Engineer @ STYLIGHT GmbH
● 11+ years as a System Administrator
● Tired of being woken up at 2am by false positives
Who am I?
● Operations Engineer @ STYLIGHT GmbH
● 11+ years as a System Administrator
● Tired of being woken up at 2am by false positives
● It all started with a 14.4Kbps modem
Your online magazine for fashion, beauty, stars, and shopping.STYLIGHT
STYLIGHT
#monitoringsucks: A Brief History
● Started in 2011
● Loosely-organized movement to address the shortcomings with monitoring tools
● Spawned an IRC channel and GitHub repo linking to available tools
Why Does Monitoring Still Suck?
● Alerts generate far too much noise to be useful
Why Does Monitoring Still Suck?
● Alerts generate far too much noise to be useful
● Dashboards aren’t actionable and require human interpretation
Why Does Monitoring Still Suck?
Why Does Monitoring Still Suck?
Why Does Monitoring Still Suck?
● Alerts generate far too much noise to be useful
● Dashboards aren’t actionable and require human interpretation
● Volume of data makes it difficult to collate, visualize, and interpret
● Finding relationships and patterns in data
● Predictive Analysis
● Anomoly Detection in large datasets
● Natural Language Processing can process and understand unstructured data
How does data science help us?
What does data science mean to me?
● Pinpoint problems before they hit a static threshold
What does data science mean to me?
● Pinpoint problems before they hit a static threshold
● Group alerts from a variety of sources into a single logical event
What does data science mean to me?
● Pinpoint problems before they hit a static threshold
● Group alerts from a variety of sources into a single logical event
● Prevent eye strain from studying hundreds of graphs
What are the tools of the future
● Kale - Etsy
● Bosun - StackExchange
● AnomalyDetection - Open source R package from Twitter
Kale
● Composed from Skyline & Oculus
Kale
● Composed of Skyline & Oculus● Skyline is an anomaly detection system
Kale
Kale
● Composed from Skyline & Oculus
● Skyline is an anomaly detection system
● Oculus is the anomaly correlation component of the Kale system
Kale
Bosun
● Monitoring and alerting system by Stack Exchange
Bosun
● Monitoring and alerting system by Stack Exchange
● Domain Specific Language for alerts and notifications
Bosun
● Monitoring and alerting system by Stack Exchange
● Domain Specific Language for alerts and notifications
● Backtest your alerts against historical data
AnomalyDetection
● Open-source R package created by Twitter
AnomalyDetection
● Open-source R package created by Twitter
● Detects anomalies in time series data and numerical vectors
AnomalyDetection
● Open-source R package created by Twitter
● Detects anomalies in time series data and numerical vectors
● Provides visualization support
AnomalyDetection
Recap
STYLIGHT.COM/JOBS
JOIN US
