zabbix 3.0 training certified specialist - xiaotonghz.com.cn · zabbix 3.0 training certified...

WWW.YOURCOMPANY.COM

Zabbix 3.0 Training

Certified Specialist

The Enterprise class Monitoring Solution for Everyone

WWW.ZABBIX.COM

Advanced monitoring Visualization

Events andnotifications

Zabbixadministration

AGENDA

ZABBIX 3.0 Certified SpecialistWWW.ZABBIX.COM 2

Configuration management

Host maintenance

CONFIGURATIONMANAGEMENT

Template_MySQLQueries per second

MySQL_serverQueries per second

Average queries per second forlast 5 minutes exceeds 5000

{MySQL_server:qps.avg(300)}>5000

Average queries per second forlast 5 minutes exceeds 5000

{Template_MySQL:qps.avg(300)}>5000

LINKING TEMPLATE

Properties:

Groups

Downstream hosts

Upstream templates

Macros

What is affected

Triggers

Graphs

Applications

Templated screens

Discovery rules

Some item and trigger properties can be overridden!

TEMPLATE PROPERTIES AND CONTENTS

Change upstream, changes downstream

Multiple templates can be linked to single host

E-mail serverWeb server

Template_MySQL

Template_Apache Template_Postfix

Template_PgSQL

HOW TEMPLATES WORK

E-mail server 1

E-mail server 2

Template_E-mail server

Template_Apache

Template_PostfixTemplate_PgSQL

E-mail server 3

E-mail server 4

NESTED TEMPLATING

Visible in the list of hosts (first two levels)

Visible in the list of templates

TEMPLATE LINKAGE

Create a "Custom Templates" host group

Create a new "Template Basic" template in it

Copy all items and trigger from host to the template

Create three additional hosts for other trainee's Vms

Add them to "Training servers" host group

Link the template to them

PRACTICAL SETUP

AGENT-LESSCHECKS

Availability and performance of remote services Syntax

net.tcp.service[service,<ip>,<port>]net.tcp.service.perf[service,<ip>,<port>]

Examples

net.tcp.service[ftp]net.tcp.service[ssh,,1022]net.tcp.service.perf[http,,8080]

SERVICE CHECKS

Add an additional item in "Template Basic":"HTTP service availability"

Make sure that the item is receiving data and is shown in a human readable format

PRACTICAL SETUP

ICMPCHECKS

Uses fping Full path in server configuration file Correct suid/permission settings SELinux can prevent Zabbix from running fping

Uses fping defaults (depends on the platform)

Source IP settings

IPv6 supported by fping6 in most distributions

Supported items

icmpping[<target>,<packets>,<interval>,<size>,<timeout>]

icmppingloss[<target>,<packets>,<interval>,<size>,<timeout>]

icmppingsec[<target>,<packets>,<interval>,<size>,<timeout>,<mode>]

CONFIGURING ICMP CHECKS

Add simple item in "Template Basic":"ICMP lost packets"

Add:"Ping loss is too high on <host>" trigger

Use 5 as threshold

Make sure that the item receives data

Use the following command to simulate dropped packets and test the trigger (run it once!):

# iptables -A INPUT -m statistic --mode random --probability 0.1 -j DROP

PRACTICAL SETUP

AGGREGATECHECKS

No agent is required

Syntaxgroupfunc["Host group","Item key",itemfunc,timeperiod]Functions: grpavg, grpmax, grpmin, grpsumItem functions: avg, count, last, max, min, sum

Examples

grpsum["MySQL Servers","vfs.fs.size[/,total]",last,0]grpavg["MySQL Servers","system.cpu.load[,avg1]",last,0]grpavg["MySQL Servers",mysql.qps,avg,5m]grpavg[["Servers A","Servers B"],system.cpu.load,last,0]

AGGREGATE CHECKS

Create "Production cluster" dummy host which will represent your "production HA cluster"

Create "Template Aggregate Check" template

Add aggregate item in "Template Aggregate Check":"Average CPU load in cluster" which calculates average CPU load on all systems from "Training servers" host group

Link "Template Aggregate Check" to "Production cluster" host

PRACTICAL SETUP

CALCULATEDCHECKS

No agent is required Syntax

func(<key>|<hostname:key>,<parameter1>,<parameter2>,...) Examples

100*last("vfs.fs.size[/,free]",0)/last("vfs.fs.size[/,total]",0)

last("net.if.in[eth0,bytes]",0)+last("net.if.out[eth0,bytes]",0)

100*last("net.if.in[eth0,bytes]",0)/(last("net.if.in[eth0,bytes]",0)+last("net.if.out[eth0,bytes]",0))

CALCULATED CHECKS

Add calculated item in "Template Basic":"Total throughput on eth0" (sum of "Incoming traffic on eth0" and "Outgoing traffic on eth0")

PRACTICAL SETUP

SNMPMONITORING

Zabbix supports SNMP v 1, 2c and 3

Community, security level and port per item

User macros supported for SNMP community

SNMP polled by Zabbix pollers

Zabbix comes with number of pre-defined templates for SNMP

SNMP MONITORING

SNMP entries indexed dynamically, text index on another MIB branch Syntax

<data base OID>["index","<index base OID>","<exact match>"]

Example

IF-MIB::ifInOctets[index,IF-MIB::ifDescr,eth0]

DYNAMIC SNMP INDEXES

Configure SNMP protocol on your VM

Add SNMP interface on the host

Link “Template SNMP Device”

PRACTICAL SETUP

LOG FILEMONITORING

Examplelog["/var/log/syslog","error"]

LOG FILE ITEM

Parameter Definition

Type Zabbix agent (active)

Key log[file,<regexp>,<encoding>,<maxlines>,<mode>,<output>]

Value type Log

Update interval

usually 1 sec

MaxLinesPerSecond – configurable per agent and item

Log rotation support●Different key – logrt● "file" becomes regular expression (not path)●More resource intensive, agent has to re-read directory

contents with each check

Log time format● y, M, d, h, m, s – everything else works as a

placeholder●Numeric values only

ADDITIONAL DETAILS

Special history view Mark selected/other Hide/Show selected Add multiple log items, sorted by entry data

VIEWING LOG ITEMS

There is an error in the log:{host:log["/var/log/syslog"].str(ERROR)}=1

There are several errors in the log for last 3 minutes:{host:log["/var/log/syslog",ERROR].count(3m,ERROR,like)}>2

EXAMPLES

log[file,<regexp>,<encoding>,<maxlines>,<mode>,<output>]

Use output parameter in log and logrt items to extract the desired number

Use Numeric type of information to see graphs and write triggers easily

Example:log[/var/log/syslog,"Total processors activated: ([0-9]+)",,,,\1]

ZABBIX 3.0 Certified SpecialistWWW.ZABBIX.COM

LOGFILE CONTENT EXTRACTION

log[/path/to/the/file,large result buffer allocation.*Entries: ([0-9]+),,,,\1] → 5948

log[path,([0-9]+) message ([0-9]+),,,,\1] → 55log[path,([0-9]+) message ([0-9]+),,,,\1 and \2] → 55 and 33log[path,([0-9]+) message ([0-9]+),,,,we got \1 and \2] we →got 55 and 33

EXAMPLES

Fr Feb 07 2014 11:07:36.6690 */ Thread Id 1400 (GLEWF) large result bufferallocation - /Length: 437136/Entries: 5948/User: AUser/Form:CFG:ServiceLevelAgreement

Logging 55 message 33

Create errors.log file in /tmp

Add log monitoring item in "Template Basic":

"The errors.log monitoring"

Use "error" string as regular expression for the item

Write "error" strings to the file

PRACTICAL SETUP

INVENTORY

AUTO-COLLECTION

Item configured to put it data in inventory field

ITEM -> INVENTORY FIELD

Inventory must be set to automaticConfiguration Hosts <Host>→ →

Use "Administration General Other" to set default host → →inventory mode

HOST INVENTORY MODE

HOST INVENTORY VIEW

VISUALISATION

Multiple items in a single graph

Customised look

Multiple graph typesNormalStackedPieExploded pie

CUSTOM GRAPHS

Legend, working time, trigger line, percentile lines (left/right)

Automatic,fixed or referencedY axis min/max

Multiple itemson two axis withdifferent colors& draw styles

GRAPH PROPERTIES

"avg" is the default"max" or "min" in the legend

GRAPH ITEM FUNCTION

{{HOST.HOST}:ifAlias[1].last(0)}

Similarly to map labels, only avg(), last(), max() and min() functions with seconds as parameter are supported within this macro in graph names. Value mapping is supported as well.

MACROS IN GRAPH NAMES

MAPS, SCREENS AND SLIDE SHOWS

Configuration for the entities is available under Monitoring

All users in Zabbix (including non-admin users) can create network maps, screens and slide shows

SHARING OPTIONS

Maps, screens and slide shows can be private or public:– public ones can be accessed by all users– private ones can be accessed by its owner (creator) and all users the entity has been shared with– permissions are respected

Map properties● Name, dimensions● Background image● Icon highlighting● Element marking upon

change● Single problem expanding● Icon label type and location● Icon mapping● Trigger acknowledgment

status displaying● Min trigger severity● URLs

Map elements● Image● Trigger● Host● Host group (or elements)● Map

Different icons for different states

Custom URLs & labels

Drag and drop editing

Icon aligning, macro expanding

Map links with labels and different styles

Macros in labels for elements and links

MAP DETAILS

ACKNOWLEDGE STATUS

None Highlight Mark Don't expand All

Disabled

Problem

Recent problem

MAP OPTIONS EFFECT

Multiple triggers can be attached to a single link

Active trigger with highest severity will determine link style

MAP LINK STATUS INDICATORS

Regexps matched against specified inventory field

No match – default icon used

ICON MAPPING

Map hierarchy: submaps

Custom labels showing any useful information:{{HOST.HOST}:system.cpu.load.last(0)} or{Zabbix server:system.cpu.load.last(0)}

Make links same color as background, appear only for problem

Route links by using invisible icons

Use background images

MAP HINTS

Create a new map

Add all hosts to it

Create links between hosts

Add useful information to a host and link label (CPU load, In/Out traffic)

PRACTICAL SETUP

Works like a tableDrag and drop editingRefreshes elements in backgroundDynamic screen elementsMany different elements:

Simple, custom and graph prototypesItem valuesMapsActive problemsOther screensCustom URLsOverview page

...more

SCREENS

Slide showCycles through many screensAny orderingMultiplier refresh in monitoring

SLIDE SHOWS

EVENTS

Event sources: Triggers

On trigger status changeDepends on "Multiple PROBLEM events" option

Network discoveryIf service is discovered/lostIf device is discovered/lost

Auto-registrationWhen previously unknown active agent contacts Zabbix server

InternalItem in "not supported" stateLow-level discovery rule in "not supported" stateTrigger in "unknown" state

HOW ARE EVENTS GENERATED?

Historical viewStatus, duration, action overviewAccess to individual event detailsFilter, export to CSV

HISTORY OF EVENTS

Source and event detailsAction, command and event history

EVENT DETAILS

NOTIFICATIONS AND

ESCALATIONS

Notify usersExecute remote commandsRepeated notifications until problem is resolvedDelayed notifications and remote commandsDifferent messages for acknowledged and unacknowledged problemsEscalations to different user or user groupUnlimited number of escalation stepsProvide information with macros

FUNCTIONALITY

Ways to send out notifications

Jabber

SMSModem support

Ez-Texting

Custom "alert" scripts

MEDIA TYPES

One or more user mediaUsually e-mail, phone number or other identifierActive based on severityActive based on time period

USER MEDIA

Message

Conditions

Operations

TRIGGER ACTIONS

Flexible conditionsHost, host groupTrigger name, severity or valueApplication, time period

Type of calculation

ACTION CONDITIONS

Notifications depend on permissions

Send messageSingle user, groupSingle media, allDefault/custom messageCondition based on acknowledgment

Run remote commandScript on server or agentIPMISSH, telnetGlobal script

ACTION OPERATIONS - TRIGGERS

Operation

User group

Script

Message

Command

Zabbix Server

Remote Agent

SSH/Telnet

Global script

Remote host

Remote Agent

Remote host

OPERATIONS

Use macros to include in notifications:Technical data like last gathered valueHost inventory details

Useful for integration with ticketing systems{TRIGGER.NSEVERITY}; full list in manual

The {ESC.HISTORY} macro includes full escalation history

USING MACROS

Based on stepsDefault interval can be overridden in operationsIf intervals collide, smallest interval wins

ESCALATIONS

Immediately e-mail administrator

10 minutes send SMS to admin

15 minutes open report at helpdesk system

30 minutes e-email management

1 hour send SMS to management

2 hours restart Apache

6 hours reboot the server

24 hours power cycle whole server room

ESCALATE IT

DETECTION OF

MISCONFIGURATION

Detection of problems in configurationNot supported itemsNot supported low level discovery rulesUnknown triggers

DETECTION OF MISCONFIGURATION

CUSTOMNOTIFICATIONS

On Zabbix server side, add new MediaType having type Script

Script must be located in a directory, specified by AlertScriptsPath directive in zabbix_server.conf

Command line parameters can be defined per each script{ALERT.SENDTO} – Send to{ALERT.SUBJECT} – Subject{ALERT.MESSAGE} – Body

Executed by Zabbix server asthe "zabbix" user

CUSTOM MEDIA TYPES

HOST AND GROUP

MAINTENANCE

Suppress data collection in the maintenance period by choosing maintenance mode

Suppress notifications during the maintenance period by adding action condition

Visually identified in Status of triggers, maps and dashboard

If problem happens during maintenance, Zabbix starts new escalation after maintenance ends

Maintenance does not stop escalations

MAINTENANCE FUNCTIONALITY

Periods must match!

MAINTENANCE IN ACTION

ZABBIX

ADMINISTRATION

GUIHousekeeperIcons / Images

MacrosValue mappingWorking time

Triggers severities and displaying options

GENERAL CONFIGURATION

REGULAR EXPRESSION BUILDER

AUTHENTICATION

Supported mechanisms:InternalLDAPHTTP: anything supported by Apache

Major detailsIs online/last accessLogin blocking:

After unsuccessful loginsSuperadmins can unblock

Access details:GUI accessAPI accessDebug modeStatus

USER LIST

Details:● Alias/username, name,

surname● Password resetting● Group membership● Frontend language and

theme● Auto-login and logout● Refresh● Rows per page● URL after login● Media

Users can set some ofthese in their profiles

USER DETAILS

USER MEDIA TAB

User group listShows user countShows individual members

User group propertiesMembersAll access is group based"Special" groups

USER GROUPS

Permissions can be assigned on user group level only

Three levels:Read-writeRead onlyDeny

PERMISSION ASSIGNING

Can't revoke access for Super Admins

Read/write overrides read only

PERMISSION SYSTEM

Available in Status of Triggers Dashboard Maps Overview Events

Redirects stderr

FRONTEND SCRIPTS

Parameter Definition

NameUnique name of the script (<submenu/submenu.../>script)

Type Script or IPMI command

Execute on Script will be executed on Zabbix server or agent

CommandsAll commands to be executed within the scriptSome macros are supported

User/host group User/host group that the script is available

Required host permission

Permission level for the host group (Read or Write)

Enable confirmation

Display a confirmation message before executing the script

Confirmation text

Custom confirmation text for the confirmation popup

Filter

AUDIT LOG

Details for some operations

Verify message contents

See status and errors

AUDIT ACTIONS

Amount of items per categoryShould have received data N seconds/minutes ago, but did notPer Proxy and Details view

QUESTIONS?

The Enterprise class Monitoring Solution for Everyone

WWW.ZABBIX.COM

TIME FOR A BREAK :)

zabbix 3.0 training certified specialist - xiaotonghz.com.cn · zabbix 3.0 training certified...

Documents

enterprise-class monitoring solution for … · les types...

zabbix manual v1 manual v1.4.pdf · zabbix manual v1.4...

zabbix talk

hosted zabbix

zabbix: monitoring millions of metrics and a zabbix 3.0 ......

tutorial zabbix

welcome to zabbix conference 2014...8 of 39 zabbix...

pavel amosov - zabbix 3.0: excited for new features? design

zabbix ubuntu

zabbix conference latam 2016 - douglas esteves - zabbix at...

grafana zabbix

zabbix conference latam 2016 - jessian ferreira - wireless...

what's new in zabbix 3.0

9th of september 2016, zabbix conference, riga volker...

badoo from lld to superdiscovery - assets.zabbix.com am i?...

open source enterprise monitoring with zabbix - · pdf...

zabbix reference manual -...

zabbix manual 1.6

zabbix manual v1.4

zabbix & security · [root@node03 zabbix-4.4.6]# cat...