networker best practices with automation

EMC Proven Professional Knowledge Sharing 2011

Ajit Kumar RathStorage Operations SpecialistEMC Data Storage Systems (India) Pvt. Ltd.

EMC Proven Professional Knowledge Sharing 2

Table of Contents

NetWorker environment………………………………………………………..4

Why automation…………………………………………………………………5

What can be Automated with NetWorker…………………………………….6

Automation to Monitor………………………………………………………….6

Monitoring NetWorker Servers………………………………………………..7

Monitoring NetWorker Client…………………………………………………..7

Automation to Configure……………………………………………………….8

Commission/Decommission of Clients……………………………………….8

Media Management…………………………………………………………….8

Group Management…………………………………………………………….8

Conclusion……………………………………………………………………….9

Disclaimer: The views, processes, or methodologies published in this article are those of the

author. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies.

EMC Proven Professional Knowledge Sharing 3

Introduction

It has become more challenging and difficult for NetWorker® Administrators to handle a large

enterprise backup environment where the number of NetWorker servers, storage nodes, and

clients increases daily. This requires a NetWorker Administrator’s proper planning, designing,

and implementing efforts while configuring. It then becomes more difficult for NetWorker

Administrators to manage such huge environments manually using the existing features of

NetWorker. It may also require additional NetWorker Administrators to manage such huge

enterprise backup environments.

This article describes how we can use automation with NetWorker, which will save manual effort

and time of NetWorker Administrators and the best practices to manage such huge enterprise

backup environments in a much more organized way. This will also help in managing such

environments with fewer NetWorker Administrators.

For example, consider an environment consisting of different data zones having more than 50

NetWorker servers, hundreds of storage nodes, and thousands of clients. Though NetWorker

has its own features to monitor, configure, and manage for schedules, groups, clients, and

media, it becomes more difficult for a NetWorker Administrator to manually configure these

features in such an environment.

Using NetWorker Management Console (NMC) or nwadmin to manually configure groups,

clients, and schedules for more than 10 servers and 50 clients takes time and effort. So too

does ejecting, inserting, and recycling more than 50 tapes.

Using automation with the help of Perl scripting in such a backup environment not only saves

time, effort, and cost, it also improves scalability and provides an error-free environment. The

target audience for this article is NetWorker Administrators who are required to configure and

monitor such enterprise backup environments with best practices.

Overall script and program writing is difficult to gather inside this article. This is an overview of

how and what can be automated in Legato NetWorker to minimize human intervention.

The following best practices for automation with NetWorker are discussed:

Commissioning and Decommissioning Backup Clients using Automation

Media Management using Automation

EMC Proven Professional Knowledge Sharing 4

Groups, Schedules Management using Automation

Client Failures Troubleshooting using Automation

NetWorker Server Maintenance using Automation

NetWorker Environment: An Overview

NetWorker always depends upon Hosts (NetWorker Servers & Storage Nodes) and Tape

Libraries to operate. The Hosts and Tape Library configuration should be always on par with the

NetWorker Application configuration for a consistent performance of a NetWorker environment.

N/W Application = NetWorker Application

A stable backup environment is ensured when performance of Hosts, Tape Library, and

NetWorker Application is at 100% over a good network.

Host performance can be tracked using monitoring tools suitable according to the Operating

System. For Tape Library, performance can be tracked using existing configuration tabs

available from most Library vendors. NetWorker performance also can be tracked using its

monitoring and reporting features.

NetWorker Administrators can face a bottleneck when monitoring and configuring NetWorker in

an environment with more than 20 NetWorker Servers and thousands of clients. In such

environments, handling of commission, decommission, schedule, media management, client

EMC Proven Professional Knowledge Sharing 5

troubleshooting, and server maintenance needs more NetWorker Administrators. However, by

using industry best practices, manual efforts can be minimized with errorless activity.

Let us consider a few questions before thinking of why we need automation with NetWorker:

1. How do you add/remove a client in NetWorker?

2. How do you check failure of a client backup?

3. How do you collect Tape Media information in NetWorker?

4. How do you configure Group in NetWorker?

5. How do you eject/insert tapes?

6. How do you label tapes?

Answer:

1. Add/remove a client using either nsradmin, nwadmin, or NMC (NetWorker Management

Console)

2. Client failure can be checked using NMC or reading daemon.log

3. Media information can be gathered using mminfo, NMC, or nwadmin

4. Group can be configured using nwadmin, nsradmin, or NMC

5. Eject/insert of tape can be done using nwadmin, nsrjb, or NMC

6. Labeling can be done using nsrjb, nwadmin, or NMC

All of these answers are correct but the effort required to perform these activities in a big

enterprise environment is greater and takes more time. Imagine an environment with more than

20 NetWorker servers, all with DR sites and these activities; it will take hours and may lead to

human error if not streamlined. While these activities take a lot of time, they can be automated.

What can be automated with NetWorker?

Automation can be segregated into 2 parts.

Automation to Monitor

Automation to Configure

EMC Proven Professional Knowledge Sharing 6

Automation can be put in place to monitor NetWorker server configuration and load, and for

client failures. Automation for configuration can be in place for adding/removing clients, media

management, and group/schedule management.

Automation to Monitor

Usually, monitoring can be done using nwadmin or NMC. Again, monitoring server availability,

server load, and client failures in a large enterprise environment in the traditional way is difficult

for NetWorker administrators. Automating by using industry scripts will help ease the NetWorker

administrator’s tasks.

Monitoring NetWorker Servers

To maintain optimal completion time, a best practice is to check for the availability of NetWorker

server and its infrastructure resources—such as tapes, drives, and nodes—prior to a backup

cycle. However, checking for availability of these resources becomes difficult in large

enterprises, requiring more administrators. Placing a script that can automate checking for

server availability, services, tape availability, drive status, and node availability will save

NetWorker Administrators a lot of time and effort.

Using NMC and nwadmin to monitor the load in the middle of a backup cycle for more than 20

servers requires additional head count and time. Again, longer sessions may lead to a hung

application, jobs db corruption, and possibly restarting the NetWorker application mid-cycle; not

advisable in such enterprise environments. Placing a script which will pool the current running

sessions and the queued sessions from all the NetWorker master servers can avoid hung

applications and jobs db corruption.

Monitoring NetWorker Client

Checking client failures using nwadmin and NMC is time consuming in large enterprise backup

environments. Usually, we identify a client failure either from the daemon.log or using mminfo

and identifying the failed saveset. Placing a script which will check the daemon.log or the

saveset information and then send out a report on the client failed on all the servers will be

useful in minimizing manual efforts by the NetWorker administrator.

EMC Proven Professional Knowledge Sharing 7

Automation to Configure

While adding/removing clients, media management, and group/schedule management can be

done using nwadmin, nsradmin, and NMC, it is time consuming and involves more head count

in a large enterprise backup environment.

Commission/Decommission of Clients

A backup environment with more than 20 NetWorker servers with a DR site for BCP requires

that, every time the NetWorker administrators add/remove client from Servers, it must be done

at both sites. If done manually, this takes a great deal of time and concentration. This can be

remedied through automation; each time a client is added/removed, it will also be

added/removed in respective production and DR NetWorker servers.

Media Management

In larger enterprise backup environments, it becomes difficult to do media management; though

it is being done using nsrjb, nwadmin, or NMC. Continuously labeling, ejecting, and inserting

tapes in bulk becomes a tedious, time-consuming task for a NetWorker Administrator. Placing a

script to automatically eject/insert and label bulk tapes will help to administer such environments

with minimal head count.

Group Management

Groups must always load balance to avoid hung NetWorker or corrupt jobsdb. It is preferable to

give priority and run those clients with more data first and clients with less data later.

e.g.

“Client A” writes 100GB data on a full backup on the 1st week of the month and is configured at

“Group Z” which is the last group in the backup cycle. The same client writes 1TB data on the

full backup done on the last week of the month. It is now better to move “Client A” to “Group X”

which runs prior to “Group Z” so as to complete the client’s backup early and avoid backup

running into production hours of the client.

Managing all these manually is a difficult task. If we can automate to pull reports of the amount

of data clients back up in every full backup, we can load balance the groups and servers

accordingly.

EMC Proven Professional Knowledge Sharing 8

Conclusion

This article will empower NetWorker Administrators and engineers to better leverage EMC

NetWorker implementations.

EMC believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION

MAKES NO RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO

THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED

WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an

applicable software license.