Un outil de monitoring applicatif

Guillaume Arnaud @guillarnaud

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Ce que fait Graphite

�2

Stockage Affichage

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Pourquoi on aime graphite

�3

• injection facile!

• rendu riche!

•compatible avec beaucoup d'outils!

•administration light

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Ce que ne fait pas Graphite

�4

Alerting Baseline

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Architecture

�5

cache relayaggregatorcache

web

collecteur:!!JMXTrans!Diamond!Metrics!…

:80

:2003!:2004

SQLLite

whisper

useruser!info

métriques

optionnel

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Round Robin DatabaseStockage RRD

�6

Taille fixe

Primary Data Point Consolidated Data Point Step

event A [instant t]!event B [instant t + 1s]!…

1 point toutes les minutes moyenne, min or max de!event A, event B…

les 10 derniers jours

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

rétentionWhisper

�7

[apache_busyWorkers] pattern = servers\.www.*\.workers\.busyWorkers$ retentions = 15s:7d,1m:21d,15m:5y

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

command lineWhisper

�8

$ whisper-dump metric.wsp

$ whisper-create 60s:10d metric.wsp

$ whisper-fetch metric.wsp

$ whisper-info metric1.wsp metric2.wsp

$ whisper-set-aggregation-method.py metric.wsp <average|sum|last|max|min>

$ whisper-resize metric.wsp 60s:10d

référence: https://github.com/graphite-project/whisper

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Alimenter Graphite

�9

echo "myproject.metric 4 `date +%s`" | nc graphite 2003

Rien de plus simple

envoi par batch (pickle) sur port 2004

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Alimenter Graphite

�10

{"servers" : [ { "port" : "8004", "host" : "mysys.mydomain", "queries" : [ { "outputWriters" : [ { "@class" : "com.g.j.m.output.GraphiteWriter", "settings" : {"port" : 2003,"host" : "myhost", "typeNames" : [ "name" ]} } ], "obj" : "Catalina:type=…,class=j.s.DataSource,name=*", "resultAlias": "datasources", "attr" : [ "numActive", "numIdle" ] } ], "numQueryThreads" : 2 } ] }

JMXTrans

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Alimenter Graphite

�11

Diamond

https://github.com/BrightcoveOS/Diamond

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Alimenter Graphite

�12

Metrics

public class QueueManager { private final Queue queue; ! public QueueManager(MetricRegistry metrics, String name) { this.queue = new Queue(); metrics.register(MetricRegistry.name(QueueManager.class, name, "size"), new Gauge<Integer>() { @Override public Integer getValue() { return queue.size(); } }); } }

http://metrics.codahale.com/

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Affichage

�13

wildcard target=server*.cpu

timeshift target=server1.cpu&from=-30min&until=-10min target=server1.cpu&target=timeShift(server1.cpu, "7d")

operations target=sumSeries(server*.cpu, "7d") target=derivative(gatling.hitcount)

format de sortie target=server1.cpu&format=json

graphique target=server*.cpu&bgcolor=blue&aeraMode=stacked

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Events

�14

curl -X POST http://localhost -d '{"what":"deploiement 3.0",

"tags": "deploy"}'

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Events

�15

/render?target=drawAsInfinite(events(deploy))

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Grafana

�16

• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC

Seyren

�17