prometheus data explore your in grafana - overview · in grafana david kaltschmidt @davkals promcon...
TRANSCRIPT
![Page 1: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/1.jpg)
Explore your Prometheus datain Grafana
David Kaltschmidt@davkals
PromCon 2018
![Page 2: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/2.jpg)
I’m not Carl
Carl is having a
great time on parental leave
![Page 3: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/3.jpg)
I’m David
All things UX at Grafana Labs
If you click and are stuck, reach out to [email protected]
Twitter: @davkals
![Page 4: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/4.jpg)
Grafana
From
Dashboarding solution
ToObservability platform
![Page 5: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/5.jpg)
Instant query queries only the last datapoint
best displayed in table panel
![Page 6: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/6.jpg)
Table row merge on labels
![Page 7: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/7.jpg)
Step alignment
“Jumpy” rate graphs
Align start/end parameters to step
![Page 8: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/8.jpg)
Heatmappanel
released in 5.1
Query: rate(foo_metric_bucket[10m])Legend format: {{le}}Format as: Heatmap
![Page 9: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/9.jpg)
Variable expansion in template variables
Summer intern project (Tobias, tak!!)
New built-in variable: $__range
Dashboard-range-based variables
![Page 10: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/10.jpg)
Template variable expansion
![Page 11: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/11.jpg)
Use range-based template variables in queries
![Page 12: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/12.jpg)
Provisioning API
Define data sources and
dashboards in files
Auto-reload on change
Allows version control of files
http://docs.grafana.org/administration/provisioning/
![Page 13: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/13.jpg)
Grafana is now fully CI’ed
Test new features that are in master:
docker run -d --name=grafana -p 3000:3000 grafana/grafana:latest
https://hub.docker.com/r/grafana/grafana/
![Page 14: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/14.jpg)
Prometheus adoption
![Page 15: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/15.jpg)
2.8K
2016
36K PromCon
![Page 16: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/16.jpg)
2.8K 16K
2016 2017
36K 92KPromCon
![Page 17: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/17.jpg)
2.8K 16K ?K
2016 2017 2018
36K 92K 186KPromCon
![Page 18: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/18.jpg)
2.8K 16K 54K
2016 2017 2018
36K 92K 186KPromCon
![Page 19: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/19.jpg)
New: Explore UI (Alpha)
![Page 20: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/20.jpg)
Motivation
Dashboard UI is made for
building dashboards, not query iteration.
“I just want to quickly…”
![Page 21: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/21.jpg)
Motivation
once panel is found, difficult to
interact with
overwhelming style and display options
![Page 22: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/22.jpg)
Motivation
Incident response currently happens elsewhere!
User story:As an SRE, I want to investigate a certain
aspect or incident of my infrastructure. I’m
already looking at the right dashboard panel
and have formed a hypothesis. From here I
want to quickly run variations of the panel’s
query to verify my hypothesis. The panel I
started from should stay unaffected.
![Page 23: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/23.jpg)
Explore UI wireframes
rate(http_requests_total[5m])
GRAPH TABLE BOTH Last 1 hour, Refresh: 10s
RUN
1 - rate(http_requests_total[5m])
. . .
rate(http_requests_total[5m])
1 - rate(http_requests_total[5m])
4.2s
3.2s
rate(http_requests_total[5m])
GRAPH TABLE BOTH Last 1 hour, Refresh: 10s
RUN
1 - rate(http_requests_total[5m])
. . .
rate(http_requests_total[5m])
1 - rate(http_requests_total[5m])
4.2s
3.2s
First tab Second tab 3rd tab My tab ╳
![Page 25: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/25.jpg)
Enable Explore UI
Behind feature flag. To enable, edit Grafana config ini file
[explore]
enabled = true
Set up a datasource that supports Explore, e.g., Prometheus.
Will be released in 6.0
![Page 26: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/26.jpg)
What we’re working on
![Page 27: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/27.jpg)
Explore UI needs to be refined
still behind feature flag, feedback welcome: @davkals or [email protected]
Prometheus metric metadata from HELP line in the exposition
support for completion forfoo_metric / on(|) group_left(|) bar_metric
design doc for explore components for other datasources
bring query field over to panel editor
![Page 28: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/28.jpg)
MultiStat panel https://github.com/grafana/grafana/pull/12620
![Page 29: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/29.jpg)
Dashboard management
Improve Grafana’s integration with Mixins (see Tom Wilkie’s talk)
Git integration to save changes back
Defaults for panel configs
![Page 30: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/30.jpg)
One more thing...
![Page 31: Prometheus data Explore your in Grafana - Overview · in Grafana David Kaltschmidt @davkals PromCon 2018. I’m not Carl Carl is having a great time on parental leave. I’m David](https://reader035.vdocuments.us/reader035/viewer/2022062510/613d36a9984e1626b65771fa/html5/thumbnails/31.jpg)
Logging is coming to Grafana