four monitoring solutions put to the test · four monitoring solutions put to the test reprint ......

Issue 20/2010 • October 22, 2010 funkschau

- 1 -

Issue 20/2010 October 22, 2010 www.funkschau.de

Everything at a Glance!Four Monitoring Solutions Put to the Test

REPRINT

- 2 -

funkschau Issue 20/2010 • October 22, 2010

A lot has been going on in the field of network monitoring software in the past 12 months. In particular, the majority of manufacturers have further expanded sup-port for virtual environments, overhauled user interfaces and added many new func-tions and features. But in the final analysis, not all products have actually improved: some have revealed weaknesses which render them unsuitable for many environ-ments. The products tested in this roundup were Ipswitch’s “WhatsUp Gold Premium 14.3,” ManageEngine’s “OpManager 8.7,” Paessler’s “PRTG Network Monitor 8” and SolarWind’s “Orion Network Performance Monitor 10.0”.

Paessler PRTG Network Monitor 8

Your network at a glance with software that’s “Made in Germany.” “Fast installa-tion, intuitive operation,” a statement that’s not just prominently displayed on Paessler’s website, but one that’s right on the money too. Setup on a Windows computer took less than five minutes, after which the soft-ware itself could be used without incident, even by a newcomer to network monitor-ing, without much need for familiarization or even training. So far, so good. Of course, whether the program installs five minutes sooner or later is neither here nor there, but no network administrator wants to spend long searching around in the management interface or looking things up in the manual in order to complete run of the mill—or even less usual—tasks.

PRTG Network Monitor reports on the cur-rent status of the network and connected devices, but also gathers data in order to dis-play long-term trends. Information gathered over 30 days, for example, helps network experts to identify bottlenecks and react by

increasing or redistributing capacities or im-plementing other changes. PRTG Network Monitor is particularly suited to monitoring device availability, bandwidths, and network resource loads—and not just on a single LAN. Remote probes allow centralized mon-itoring on several networks at once, and can also be used for load sharing during CPU-intensive processes such as packet sniffing or NetFlow monitoring. Remote probes open up a wide variety of applications to the PRTG user. The software can conceiv-ably be used to monitor customer networks, all branches of an organization, or separate networks within a company (e.g. LAN and DMZ). But a complex infrastructure doesn’t mean that PRTG Network Monitor is com-plicated to use, because all this monitoring requires no more than a single PRTG Core Server. The product scales up to 30,000 sen-sors per installation.

What remote probes do not do is increase the availability of the PRTG monitoring sys-tem. For that there now is the PRTG Cluster Failover Solution included with version 8. This allows for up to five instances of PRTG working together: one primary and up to four secondary servers. This type of PRTG cluster offers several additional advantages: 100% uptime even during software up-dates and server crashes, as well as auto-matic failover, means this solution can have multiple points of presence monitoring.

All servers are constantly monitoring all of the sensors. This is interesting not just from the point of view of error tolerance, but also because it allows the administrator, for ex-ample, to measure and compare response times from various locations in the network and to calculate aggregate up and down-times.

PRTG Network Monitor monitors complex, distributed networks with just one PRTG core server. One installation of the product supports up to 30,000 sensors.

Be alarmed—avoid catastrophesBy Dirk Jarzyna

Network monitoring software helps identify errors and

malfunctions in the network and on network devices

before they lead to a possible catastrophe. They are also

useful for capacity planning. funkschau tested four of

the best known examples in this area.

PRTG Network Monitor 8.0Manufacturer: Paessler

Type: Monitoring software

Price: 100 sensors, 1 core server €357, unlimited license (unlimited number of sensors, 1 core server) €4,165

Web: www.paessler.com

Pros/cons:

Complete package

Simple setup and configuration

Very good price- performance ratio

KEY FACTS


- 3 -

PRTG clustering is not expensive: all PRTG licenses immediately permit a single failover installation in which two PRTG instances work together. Additional licenses are only required for three or more nodes.

The product offers a total of four user in-terfaces: an Ajax-based web interface, a slimmed-down HTML interface for older browsers and mobile devices (IE 6/7, An-droid, Blackberry), a Windows GUI, and an iPhone app. The most frequently used inter-face, Ajax, is elegant and simple to operate. The welcome page contains a prominent button for the most important tasks to be carried out immediately after installation, including auto-discovery of networks and connected devices. Discovery in the local network through an IP address range of 0 to 255 was done at lightning speed and was complete and accurate when tested. The program can also install the necessary sensors for monitoring during the discovery process, if desired. Discovery takes more or less time depending on the number of sen-sors selected. Additional sensors are very easy to add to selected devices or groups afterwards.

The range of sensors included was already impressively large in previous versions. Sen-sors for monitoring all the usual network services (ping, HTTP, HTTPS, POP3, DNS …), for QoS, Radius Server, SLAs, Exchange and Syslog servers, as well as support for SNMP and WMI, packet sniffing, NetFlow and sFlow have long been standard. Virtualiza-tion continues to be popular. Therefore, the latest version of PRTG contains a range of new and/or extended sensors tailored espe-cially to this which allow monitoring of the hardware information from a VMware-ESX/ESXi server via WBEM, monitoring of a virtu-al machine (VM) on Xen servers, and Hyper-V storage device monitoring. The WMI sen-sors and special Linux and Mac OS sensors have also increased in scope. Paessler has tested sensors with the most current dis-tributions. But strangely, Red Hat does not appear on the list published on Paessler’s website, while Fedora does.

The major components of a PRTG installa-tion are the PRTG server for data storage as well as one or more probes. The probes carry out the actual work. They connect to the server automatically, load the sensor configurations determined for them, and carry out the monitoring functions. As the probes initiate connections to the server, a server or connection failure does not affect monitoring. The first “local” probe is al-ready created by the setup program. It runs on the PRTG server and monitors all sensors

WhatsUp Gold Premium executes an au-tomatic device discovery, carries out SNMP and WMI monitoring, produces an auto-matic graphic representation of the net-work, runs actions automatically when the status of a monitored device changes or a threshold value is exceeded, generates re-ports and notifies administrators of alarms. In the Premium Edition, this all functions on a single network which can, however, be as big as you like. In principle, What-sUp Gold can also scale over distributed networks, but for this other editions are required. Customers must carefully con-sider what functionality they require and which devices or networks they actually want to monitor, as the different WhatsUp Gold editions vary in terms of performance, functional scope and, of course, in price.

in the system. The hierarchical organization of the probes, device groups, and devices makes system administration easier. This al-lows many settings, such as login informa-tion, to be transmitted from probes via de-vice groups to individual devices by means of inheritance.

PRTG Network Monitor provides clear acoustic and visual notification of new alarms, warnings, and other messages. Upon accessing a notification or selecting an affected sensor, the program provides very good explanations of what the problem actually is and what the individual values and adjustable parameters mean. Several dashboards, diagrams and lists represent current system statuses and historical data in a meaningful way. The filter options are excellent. Report generation has meanwhile been decoupled from the web server and user interface, so that the user experience is no longer affected. Anyone who wants to can produce network maps that visually represent the monitored network or parts thereof—unfortunately only on a manual basis. To this end there are approximately 280 different icons available, representing network devices which can be connected to each other using (network) lines. The pro-gram shows alarms, warnings, etc. under-neath the icons. Creating a map of this kind is somewhat cumbersome, however. We would like to see a function that automati-cally includes the selected devices or device groups in a map.

PRTG Network Monitor 8 meets all the de-mands we make of a monitoring product. The product is complete, easy to install, flexible to use, easily scalable and provides good levels of notification and explanation. It also isn’t as hard on the pocket as many competing products. During testing it im-pressed us with its high levels of usability, reliability, and precision.

Ipswitch WhatsUp Gold Premium 14.3

WhatsUp Gold is another popular network, server, and application monitoring prod-uct that has received a lot of praise. So we were all the more disappointed by What-sUp Gold’s performance during testing. We can’t agree with manufacturer Ipswitch’s claim that the product is “simply the most intuitive, complete and cost-effective net-work management toolset available today”. Nothing was particularly intuitive, the prod-uct was only complete with the addition of extra applications or plug-ins, and in view of the underwhelming performance that WhatsUp Gold Premium gave in its basic version, the software is too expensive.

The web interface in WhatsUp Gold is quite pleasant. Less pleasant however are the many empty fields.

WhatsUp Gold 14.3Manufacturer: Ipswitch


Price: from €1,591 (Standard Edition, up to 100 devices), €6,715 (Distributed Edition, up to 500 devices). Prices do not include optional plug-ins. Prices for more than 500 devices on request.

Web: www.whatsupgold.com

Pros/cons:

Highly scalable

Variety of editions and add-ons offers flexibility, makes it difficult to choose a product

Requires the Remote Site Edition for remote sites

KEY FACTS

- 4 -


This doesn’t make choosing easy. For ex-ample, anyone who needs a product that performs the same functions as PRTG Net-work Monitor can immediately forget about the entry-level product, WhatsUp Gold Pre-mium.

The entire setup of WhatsUp Gold Pre-mium takes quite a long time. The reason for this is partly because a Microsoft SQL server must first be installed, if not already present. The setup routine installs Microsoft SQL Server 2005 Express Edition if it doesn’t find any other SQL server. Following installa-tion a Windows GUI starts up. A large “Start Here!” button in the Welcome Center shows what follows next. With one click the Quick Start Assistant starts up to configure the Administrator Notification (e-mail) and the network login information (for SNMP, Windows, ADO, Telnet, SSH and VMware). Once this has been completed, the discov-ery process starts. This is done by the as-sistant via IP range scan, SNMP smart scan, VMware scan, or host file scan. The devices found should then be added by the adminis-trator to the WhatsUp Gold database.

The results of discovery in the test network were not encouraging. While the IP range scan, for example, did discover all devices in the network, the program clearly had some difficulty identifying details such as the type of device (Windows station, Windows serv-er, Linux server, Switch, etc.). Just like PRTG Network Monitor, WhatsUp Gold can con-figure both active and passive monitors for discovered devices. To do so, however, the discovery process must identify the respec-tive device roles—and that’s exactly what it didn’t really do in our test. That meant a lot of manual work later on that no adminis-trator would want to take on, especially as almost no administrator has the necessary time for it. With previous versions of What-sUp Gold, discovery and monitor configura-tion functioned perfectly. Why it isn’t work-ing in the current version remains a mystery.

The administrator connects monitors (in PRTG these are the sensors) with standard procedures which describe a range of ac-tions that WhatsUp Gold carries out as soon as an error or change in status occurs on a device. For example, when an error is iden-tified, WhatsUp Gold sends an e-mail noti-fication, plays a particular sound, or opens a pop-up message on a selected computer.

WhatsUp Gold actively polls devices in the network in order to determine changes in status. For this the program uses the afore-mentioned preconfigured monitors or those created by the administrator. Performance

monitors observe the resources of a device, such as disks, interfaces, and memory.

Depending on the answers received, WhatsUp Gold carries out actions, such as reporting to the administrator or restarting a service. The Alert Center in the web GUI displays consolidated alert information and simplifies the management of notification procedures. Administrators are kept up-to-date on what’s happening in their network. But the Alert Center does not provide expla-nations with as much detail as the compara-ble feature in PRTG Network Monitor. Apart from that, this interface does not provide as clear an overview.

Neither the Windows GUI nor the web ap-plication is as intuitive to operate as the interfaces in PRTG. When the message “In-terface down” was received, for example, it did immediately find detailed information on the problem. Overall, there are simply not enough automated tasks in WhatsUp Gold for our liking—the program leaves too much to be done by the administrator. For example, device information such as MAC addresses and operating system names and versions must be filled in manually. However it is possible that this has something to do with the failed discovery/monitor configura-tion—we just don’t know.

New in version 14.3 are ready-made views that support tasks such as remote site man-agement. Existing monitoring settings can be easily copied to new devices, which does make the devices available somewhat more quickly. Ipswitch has made some changes to database storage and optimization and information retrieval technology, which improve the performance. In terms of da-tabases, there is also additional support for standard SQL clustering.

ManageEngine OpManager 8.7

Here we have a monitoring product that really didn’t do anything for us. ManageEn-gine is indeed a specialist in management applications, and OpManager, the com-pany’s network monitoring software, is the first choice for more than 700,000 network administrators in 93 countries, according to the manufacturer. But after repeated test-ing of the software, we still really couldn’t work out why this should be. It may well be that 25,000 IT administrators download the product each month—we’ve done it too—but whether they actually use it after initial testing, or put it back in the virtual drawer, is not as easy to find out… The product is very powerful overall and thoroughly capa-ble of monitoring networks, network de-

vices and services, identifying performance bottlenecks, notifying administrators and generating reports, but it’s quite an effort to set up the software so that it operates the way an administrator wants. There are some products that make it easier for the user.

For a start, prospective users should be aware that this is a product that only actu-ally delivers useful information when all de-vices to be monitored support SNMP. This also applies to WhatsUp Gold to a certain extent, but the effect is not as severe as in OpManager. While it’s good when a prod-uct makes active use of standards, but be-ing exclusively based on SNMP is surely not the way. For our part, we simply cannot im-agine a professional network environment that has each and every device switched to SNMP, or even supports such a thing.

The web interface of OpManager uses widgets, making it highly customizable, but the system is very sluggish.

OpManager 8.7Manufacturer: ManageEngine


Price: €1,995 for the Professional Edition (up to 100 devices), €27,995 for the Deluxe Edition (up to 2,000 devices). Prices do not include optional add-ons and plug-ins. Prices for more than 2,000 devices on request.

Web: www.manageengine.com

Pros/cons:

Highly customizable user interface

A lot of manual configuration needed

Errors in device classification

KEY FACTS


- 5 -

But this would have to be the case to enable complete management of the network and the devices used in it.

Like so many manufacturers, ManageEn-gine also offers OpManager in several edi-tions, with some additional add-ons and plug-ins. It’s not easy to determine which editions contain which expansions. This makes it difficult to calculate a final price for the product. In any case, it starts from free. However the free version only supports a maximum of ten nodes and is therefore only suitable for short product tests. For the fully-featured Distributed Edition, suitable for monitoring distributed networks, you’ll need to hand over all of $9,995. This price applies to monitoring of up to 250 nodes. What isn’t clear is whether plug-ins such as Cisco IPSCA or IPSLA monitor, the NetFlow Analyzer or the NCM plug-in are already in-cluded.

The program setup took quite a while in our test. OpManager uses a Microsoft SQL server or MySQL server as a database. ManageEngine includes MySQL, which was fortunate in our case, as we simply couldn’t get OpManager to function with our Micro-soft SQL Server 2008. We also had no luck in using OpManager or the program’s web console on a Windows 7 computer.

Here we encountered serious compatibil-ity problems. Installed on a Windows 2003 server, it eventually worked “so-so”.

After the first run of the web client, the pro-gram wants to carry out auto-discovery. Op-Manager supports automatic discovery and smart classifications with device and inter-face templates, mass imports, and process templates.

Initial discovery does require a little manual input, for example the administrator must select the services to be discovered, includ-ing DNS, web, SQL, HTTP, and POP, and then specify an IP range. The subsequent discov-ery was very time-consuming, even on just one subnetwork over a range of 0 to 255. The recognition rate was okay, the clas-sification for devices that supported SNMP was half-decent. We say “half-decent,” because we object to finding all our Win-dows 7 desktops classified as servers. These errors can be rectified, but other test can-didates have shown that there is no need for such errors to arise in the first place. An-other thing: if a program cannot manage to classify devices that lack SNMP support on the terminals, it still shouldn’t limit itself to only reporting the IP addresses of the de-vices. Why not do a short DNS query or use NetBIOS names, like other programs do?

At least then we would have some idea of the device in question.

The pleasant-looking and easy to use cli-ent web application uses standard Internet Explorer. Used with this browser, it’s not exactly fast. OpManager uses lots of Java—and between the frequent page changes, the administrator can happily pop out for a coffee without missing anything. The ap-plication makes extensive use of widgets. In previous versions of OpManager, this caused problems for some browsers which have since been rectified. Widgets are okay, because, among other things, they allow administrators to configure the interface whatever way they want.

Without intervention by an administra-tor, OpManager monitors next to nothing. Almost all monitors must first be config-ured and assigned to devices. Fortunately, templates make this process easier. Those prepared to make the effort will finally be rewarded with a system with good fault management, performance, and device monitoring. The program produces respect-able real-time graphics, historical reports on availability, utilization, response times, and inventory. The WAN monitoring is limited to monitoring the availability of WAN links, re-ports on performance analyses, and capac-ity planning. The administrator will only gain complete functionality with the addition of the optional WAN monitor add-on.

The alarm system is serviceable, if a lit-tle sluggish. But the program provides al-most no explanation of alarms, adjustable parameters, etc. What is an alarm such as “ColdStart: zero” supposed to mean to an administrator? Administrators must already be very familiar with it in order to gain much benefit from this product.

SolarWinds Orion Network Performance Monitor 10.5

Orion Network Performance Manage, Orion NPM for short, is (as its name suggests) fo-cused on monitoring network performance. If an administrator also wants to keep an eye on the performance of network applica-tions or manage the network configuration, they’ll need to use a separate product and/or module that is, of course, sold separately. NetFlow traffic analysis, IP address manage-ment, and IP SLA management are also only possible with optional extensions. Even without extensions, Orion NPM is already a very complicated product that requires a lot of patience right from the setup. There are about 380 MB to be unpacked and installed—and it takes time. One can only

hope that the setup works first time and doesn’t—as happened to us—simply trail off, due to packet errors, for example. We finally got Orion NPM to work on a Windows Server 2003 (SP2) with .NET Framework 3.5 and the Microsoft SQL 2005 Database (Express Edition) that Orion NPM installed itself.

A standard installation is designed to moni-tor approximately 1,000 nodes. While Orion NPM scales up significantly higher, it is how-ever advisable to install additional standby engines, multiple polling engines and/or ad-ditional web servers. For high availability en-vironments, purchase of the Orion Failover Engine is also recommended. The additional servers/engines and the Failover Engine are purchased in the form of so-called scalabil-ity engines, with prices starting at €5,700.

Orion NPM is a very expensive product, and administrators can only access its full functionality by purchasing optional extras.

Orion Network Performance Monitor 10.0Manufacturer: SolarWinds

Type: Network monitoring software

Price: 100 elements €2,015, unlimited elements €20,350, Prices do not include options such as Enterprise Operations Console or scalability engines.

Web: www.solarwinds.com

Pros/cons:

Flexible alert system

Tedious set-up

Prices

Additional scalability engines may be necessary in large distributed networks

KEY FACTS

- 6 -


For distributed networks with multiple in-stances of Orion NPM, the Orion Enterprise Operations Console operates as the com-mand center. Unexpectedly, the EOC is an optional component costing an additional €4,000.

The software setup took quite a while in our test. SolarWinds says that the product is “up and running” in less than an hour. It didn’t take an hour, but compared to other products, Orion NPM crawled along at a snail’s pace. Like WhatsUp Gold, Orion NPM demands a Microsoft SQL Server and if necessary installs Microsoft SQL Server 2005 Express Edition. The latter proved fortunate in our test, as we had no luck with a pre-viously-installed Microsoft SQL Server 2008 Express Edition. Orion also expects to find functioning Internet Information Services and .NET Framework. Checking installation requirements and the additional software required is nothing new and generally not a problem, but PRTG and OpManager dem-onstrate that it doesn’t have to be so.

Orion NPM offers two user interfaces, a Windows GUI and a web console. Most ad-ministrators will use the web console, which is well designed and easy to master. But this interface is not as customizable as the one from ManageEngine, although it’s just as sluggish. In addition to these user interfaces, numerous applications also appear in the Windows start menu after installation; these serve to customize the Orion NPM installation or maintain the database, for example.

The initial discovery uses SNMP and ICMP. In a local sub-network with 255 nodes, it’s rel-atively quick to run. The product functions precisely and even identifies every individual network interface and all protocols running on them. The alert system is flexible, like the discovery it operated perfectly in our test and is easy to use. Like the other products, Orion NPM generates alerts when an event occurs or a threshold value is exceeded. The program offers a range of options for react-ing to alerts, including the usual notification options, execution of an automatic script or program, and an escalation sequence. Con-figuring network alerts is not particularly difficult. The product allows administrators to define device dependencies and to con-figure alerts for contiguous events and/or SolarWind’s continuous statuses over a cer-tain period. For example, the system can be set up so that it doesn’t produce an alert immediately if CPU utilization exceeds 90 percent, but only if this utilization lasts for more than five minutes.

Monitoring Software

The test products were installed on a network on which several Windows Server 2003/2008 machines; one Exchange Server and one SQL Server are operating. The machines within the network were connected via Fast Ethernet switches and WLAN rout-ers; connection to the internet was through an ADSL router. The client machines worked with various op-erating systems, including Windows XP, Windows 7, and Linux. The serv-ices and protocols operating on the network included TCP/IP, DNS, POP3, SMTP, IMAP, SNMP, HTTP, HTTPS, and FTP.

Following initial installation and con-figuration of the monitoring pro-grams, we allowed them to search the network and gather information about the installed services and pro-tocols for a time. After that, thresh-old values were set and actions to be carried out were defined. We exam-ined whether the programs identi-fied when a threshold value was ex-ceeded, a system status changed and performance was interrupted, and if they reacted as intended. Among other things, we evaluated the price-performance ratio, user-friendliness, and the way in which the products supported monitoring of physically distributed networks.

TEST PROCEDUREConclusion

If you want comprehensive monitoring in an enterprise network, you’d better be very careful. It’s all too easy to fall into a “cost trap,” where a product that seemed cost-efficient has to be extended at great expense. The majority of manufacturers tempt purchasers with inexpensive “stand-ard editions,” which can often do nothing more than monitor a couple of nodes in an individual network segment. If expansion is required later, for example for distributed monitoring, NetFlow or SLA monitoring, the manufacturers strike and charge prices which elsewhere would get you almost a complete package covering all eventualities.

Take Paessler, for example. Paessler’s PRTG Network Monitor 8.0 is a complete, highly-scalable and very easy to operate product at a price at which experienced administra-tors would not expect to see a professional monitoring solution in the enterprise class. But as our test showed, appearances can be deceiving. Among the four products tested, PRTG Network Monitor 8.0 shone as the most fully-featured and easiest to use monitoring package at an acceptable and transparent price.

In terms of functionality, the other prod-ucts were fully on a par with PRTG Network Monitor—but often only after the installa-tion of separate products, add-ons, or plug-ins.

Ralf Ladner © 2010 WEKA FACHMEDIEN GmbH

four monitoring solutions put to the test · four monitoring solutions put to the test reprint ......

Documents