nagios conference 2014 - jack chu - how to think with nagios to solve monitoring problems

31
How to solve monitoring problems with Nagios Jack Chu [email protected]

Upload: nagios

Post on 18-Dec-2014

53 views

Category:

Technology


0 download

DESCRIPTION

Jack Chu's presentation on How to Think With Nagios to Solve Monitoring Problems. The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference

TRANSCRIPT

Page 1: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

How to solve monitoring problemswith Nagios

Jack [email protected]

Page 2: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Introduction & Agenda

• Different aspect of essence of monitoring"• Trick to make a plugin pass through

firewall"• Overview of NSClient++ config file"• Monitoring usage of Windows cluster

shared disk

2

Page 3: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Essence of monitoring

Page 4: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Essence of monitoring

A brief definition:""

To alert you something abnormal you care

4

Page 5: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Essence of monitoring

Let’s try to imagine:"a beautiful girl……"

5

Page 6: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Essence of monitoring

6

Connection should be built first.

The effects of different languages may differ.

More trust more information.

What do we learn from contacting this girl?

Page 7: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Essence of monitoring

7

• To build connection or even trust before monitoring."

• To evaluate different ways of connection"• To ask right questions:"

Detective Spooner: Is there something you want tell me? "Dr. Lanning: I'm sorry. My responses are limited. You must ask the right questions.

To some degree, to establish host and service monitoring is similar to contact a girl.

Page 8: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Something to remember

• Nagios Exchange - a treasury"• Try to search with different key words

combination:"Nagios ABC"Nagios monitor ABC"monitor ABC"“other tool” ABC"(anything about) ABC"……"

8

Page 9: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Make a plugin to pass through firewall

Page 10: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Make a plugin to pass through firewall

10

Page 11: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Start to think

"

Can Nagios monitor this?""

How can we monitor this with Nagios?

11

Page 12: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Start to think more

• How can we find ISP switching?

12

• Is there any plugin in Nagios exchange able to do this?

• When plugin is blocked, what can we do?

• Let’s imagine again!

Page 13: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Tweak the plugin

Original script:""

#! /bin/bash PROG="/usr/bin/traceroute -n" GREP=“/bin/egrep" HOST=$1 FILE=/tmp/check_tracert-`date +%N`.txt START="$2" LAST="$3"

13

Page 14: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Tweak the plugin

Modified script:""

#! /bin/bash PROG="sudo /usr/bin/traceroute -n -T" GREP=“/bin/egrep" HOST=$1 FILE=/tmp/check_tracert-`date +%N`.txt START="$2" LAST="$3"

14

Page 15: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Modify /etc/sudoers

To grant root privilege and avoid requiretty error:""

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/check_2hoptraceroutetcp.sh ...... #Defaults requiretty

15

Page 16: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Quote for you

"

“An idea is nothing more or less than a new combination of old elements.”""

~James Webb Young"

16

Page 17: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Overview of NSClient++ config

Page 18: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Overview of NSClient++ config

• The structure of nsclient.ini"• Enable module + finish the setting for

individual module + define command or script + restart NSClient++ daemon"

• check_nrpe -H 10.1.2.3 -p 5666 -c command (we define at nsclient.ini)

18

Page 19: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

“Indirect check” with NSClient++: to build a Windows proxy

• Enable NRPEServer and NRPEClient module"• Allow Nagios access at [/settings/NRPE/server]"• Define the command at [/settings/external scripts/scripts]"• check_abc=nscp nrpe -H [remote host IP] -P 5666 -c

get_cpu [remote command]19

Page 20: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Monitoring usage of Windows cluster shared disk

Page 21: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Monitoring usage of Windows cluster shared disk

• Scenario"• Problems of traditional monitoring way

21

Page 22: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Monitoring usage of Windows cluster shared disk - Scenario/Traditional ways/Problems

22

Page 23: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Monitoring usage of Windows cluster shared disk - Better solution

23

• What we hope our solution can do: automatic

• What are the pieces required for this puzzle?✴ NSClient++ located at proxy, nodes, and

shared disk"✴ NSClient++ daemon as a part of Windows

cluster resource

Page 24: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

24

Monitoring usage of Windows cluster shared disk

Page 25: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Config on nsclient.ini

• Config at Windows proxy"• Config at Nodes"• Config at shared disk

25

Page 26: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Make NSClient++ daemon be a part of Windows cluster resource

• At C:\Program Files of any cluster node, copy the NSClient++ folder to the shared disk."

• How to make NSClient++ daemon be a part of Windows cluster resource (step by step)

26

Page 27: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Sample command on Nagios

define command{ command_name check_nrpe_drivesize command_line /usr/local/nagios/libexec/check_nrpe -H $HOSTADDRESS$ -p 5666 -c check_shared_drive }

27

Page 28: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Sample service on Nagios

define service { use generic-service host_name ms_cluster_host_01 service_description Shared Disk Usage check_command check_nrpe_drivesize servicegroups windows }

28

Page 29: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

What do we learn?"• A different aspect to watch essence of

monitoring"• Trick to make a plugin pass through firewall"• NSClient++ config file"• How to monitor Windows cluster shared

disk via proxy"""

29

Page 30: Nagios Conference 2014 - Jack Chu - How to Think With Nagios to Solve Monitoring Problems

Conclusion• Keeping gathering and masticating

materials for your Nagios monitoring • Learn from anything to enhance your

monitoring capacity. • To solve problems is like to collect jewels

needed to make a necklace. • Nagios is the most essential jewel to make

your own necklace!

30