sysdb — the system management and inventory collection service
DESCRIPTION
“System Datebase” (SysDB, https://sysdb.io/) is an Open Source daemon which aggregates system management and inventory information and stores them in a central, graph-like database. All information is (continuously) retrieved from other hardware or software systems (inventory services, monitoring services, etc. like Icinga, Puppet, collectd) and may be queried through a generic interface. SysDB focuses on meta-data like host-attributes, services and their attributes, and lists of available metrics and provides means to query data from back-end data-stores (e.g. time-series data). This way, all important information about your systems is accessible from a central location allowing for use-cases such as central dashboards, cross-link monitoring or inventory information, identify missing pieces in your system configuration, advanced monitoring configuration, and much more. This talk provides an overview over SysDB and its features as well as sample use-cases. The project is still in an early development stage but already usable. The talk also covers future directions and further integration with existing services.TRANSCRIPT
![Page 1: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/1.jpg)
SysDB – System DataBaseA system management and inventory collection service
Sebastian ‘tokkee’ Harl<[email protected]>
Icinga Camp San Francisco 2014September 25, 2014
![Page 2: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/2.jpg)
Disclaimer
WARNING:SysDB is still under heavy development
and not considered stable yet1.
Flaming, bashing or other forms of constructivefeeback are very appreciated ... also, contributions :-)
1 That is, interfaces may still change quickly.
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 2
![Page 3: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/3.jpg)
Background / Motivation
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 3
![Page 4: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/4.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 5: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/5.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 6: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/6.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 7: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/7.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 8: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/8.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld
gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 9: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/9.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 10: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/10.jpg)
Motivation
Database Server Foo
External Gateway
Zooperserver
gw1.domain.tld gw1.domain.tld
arch=amd64lsbdistid=Debianloc=BayArea
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 4
![Page 11: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/11.jpg)
The System DataBase
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 5
![Page 12: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/12.jpg)
SysDB – System information platform
SysDB
Mod1 Mod2
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 6
![Page 13: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/13.jpg)
SysDB – Overview
• SysDB collects information about arbitrary hardware and softwaresystems for a global overview of the entire infrastructure.
• Focus on meta-data and links to more specific information.• Simple examples:
Hosts and their attributes (“facts”)Services and their attributesMetrics (pointer to performance data)Monitoring information (e.g. current state)
• SysDB collects these information and merges objects frommultiple back-ends by hostname.
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 7
![Page 14: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/14.jpg)
SysDB – Core features
• https://sysdb.io/, https://github.com/sysdbCI: https://travis-ci.org/sysdb/sysdb∼ 80% code (function) unit-test coverage in the core
• BSD license• Written in C (UI written in Go WIP)• Generic store aggregating system information• Multi-threaded, event-based client/server architecture• Easy to extend (simple plugin API)• Simple network protocol and query language• Most of the code implemented as a library (reusable)
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 8
![Page 15: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/15.jpg)
SysDB – Plugins
Currently available collectors (back-ends):• collectd::unixsock – query collectd’s UNIX socket interface→ host metrics
• mk-livestatus – query Monitoring systems (Nagios, Naemon,Icinga, Shinken) using Check MK Livestatus→ hosts and services (metrics planned)
• puppet::store-configs – query Puppet→ host attributes
Planned: Passive data collection (e.g. using Gearman), Foreman,PuppetDB, $your favorite system (send patches!) :-)
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 9
![Page 16: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/16.jpg)
SysDB – Plugins (2)
Time-series fetchers:
• timeseries::rrdtool – fetch time-series from local RRD files
Misc plugins:
• cname::dns – canonicalize host-names using DNS• syslog – syslog logging
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 10
![Page 17: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/17.jpg)
SysDB – Architecture
RRD
infra
back-ends
store FE
sysdbd
sysdb
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 11
![Page 18: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/18.jpg)
The SysDB Store
• Stores generic objects (in memory atm)• Canonicalization of hosts (by their name as provided by
back-ends)• Each object stores the time-stamp of the last update and the
(automatically calculated) update interval
• Interface to query data• Generic access to time-series data• JSON is the external data representation
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 12
![Page 19: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/19.jpg)
The SysDB Store – Stored objects
• Host – any kind of physical resource• Service – any kind of service• Metric – identifier for metrics data→ time-series data is not stored in SysDB
• Attribute – attributes of hosts and services→ values support multiple types
• Common core attributes:unique nametime-stamp of last updateupdate intervallist of back-ends providing the object
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 13
![Page 20: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/20.jpg)
The SysDB Store – Example
kvm0
kvm1
kvm2
ovirt0 vhost0
vhost1
vhost2
(Full graph-support not implemented yet.)
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 14
![Page 21: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/21.jpg)
The SysDB Client
• Interactive client program for SysDB• Connects to a SysDB daemon• Interactive command shell• Receives and displays asynchronous log messages
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 15
![Page 22: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/22.jpg)
The SysDB Query Language
• Remotely similar to SQL• But it’s not SQL1
• List and query hosts• Fetch time-series information (from some back-end data-store)
1 Yes, a DSL might limit options but it was the easiest for now – believe it or not ;-)
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 16
![Page 23: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/23.jpg)
The SysDB Query Language (2)
• LIST hosts• FETCH host <hostname>
• LOOKUP hosts MATCHING <condition>
• ... FILTER <condition>
• TIMESERIES <host>.<metric>START 2014-01-01 END 2014-12-31
→ https://sysdb.io/manpages/head/sysdbql.7.html
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 17
![Page 24: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/24.jpg)
The SysDB Query Language – Example
sysdb=> LIST hosts FILTER :age < 2 * :interval;[{
"name": "monitor.lxc.tokkee.net","last_update": "2014-04-03 10:26:41 +0200","update_interval": "5m4s",
},{"name": "puppet.lxc.tokkee.net","last_update": "2014-04-05 11:04:08 +0200","update_interval": "5m2s"
}]
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 18
![Page 25: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/25.jpg)
The SysDB Query Language – Example (2)
sysdb=> LOOKUP hosts WHERE attribute.architecture = ’amd64’AND service.name =˜ ’postgres’;
{ "name": "db.lxc.tokkee.net","last_update": "...", "update_interval": "10s","attributes": [{
"name": "architecture", "value": "amd64","last_update": "...", "update_interval": "5m3s"
},{ ... }],"services": [{
"name": "PostgreSQL Master","last_update": "...", "update_interval": "5m"
},{ ... }],"metrics": [{...}]
}c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 19
![Page 26: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/26.jpg)
The SysDB Query Language – Conditionals
Search and filter conditions:
MATCHING attribute.architecture = ’amd64’AND service.name =∼ ’postgres’
MATCHING host =∼ ’\.tokkee\.net$’
FILTER :age >= 3 * :interval
FILTER :backend != ’mk-livestatus’
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 20
![Page 27: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/27.jpg)
Use Cases
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 21
![Page 28: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/28.jpg)
Use Cases
• More flexible web front-ends combining multiple back-ends→ correlate and annotate information→ central dashboard→ link to specialized front-ends
• Compare the back-ends (monitoring)→ Which hosts / services are missing in a back-end?→ Base for business-processes: What is the global status of all
Windows systems in some data-center?• Extend your CMDB with dynamic annotations or cross-checks
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 22
![Page 29: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/29.jpg)
Future Directions
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 23
![Page 30: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/30.jpg)
Future Directions
• Better and more integration with other systems• Persistent store: RDBMS, graph database (?)• Interface to query other live data (e.g., monitoring status) from
back-ends• Distributed architecture (HA and load-balancing)• Extensible Web-Interface• Extend the type system and filters• . . .
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 24
![Page 31: SysDB — The system management and inventory collection service](https://reader035.vdocuments.us/reader035/viewer/2022081404/559b79b91a28abfb248b4639/html5/thumbnails/31.jpg)
SysDB – System DataBase
Thank you for your attention!
Questions, comments, rants?
https://sysdb.io/, https://github.com/sysdbhttps://sysdb.io/+ https://sysdb.io/facebook
@SystemDatabase
c© 2014 Sebastian ‘tokkee’ Harl The System DataBase – slide 25