optimization of vm image backups with right proxies · avvcbimage info : hotadd transport is...

27
OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES Umit Dericioglu Senior Advisor Delivery Specialist Dell EMC [email protected] Knowledge Sharing Article © 2017 Dell Inc. or its subsidiaries.

Upload: others

Post on 17-Apr-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES

Umit DericiogluSenior Advisor Delivery SpecialistDell [email protected]

Knowledge Sharing Article © 2017 Dell Inc. or its subsidiaries.

Page 2: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 2

Table of Contents

Choosing the Right Proxy for Image Backups ............................................................................................... 3

Not All Proxies Are Created Equal ................................................................................................................. 4

Finding the Efficient Proxy ............................................................................................................................ 8

An Easy Solution ............................................................................................................................................ 9

Conclusion and Some Suggestions .............................................................................................................. 12

Appendix ..................................................................................................................................................... 13

Prerequisites in order to run the scripts ..................................................................................................... 13

Disclaimer: The views, processes or methodologies published in this article are those of the author. They

do not necessarily reflect Dell EMC’s views, processes or methodologies.

Page 3: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 3

Choosing the Right Proxy for Image Backups

It all started when there was a decision to implement all VM image backups on Avamar. We, the backup

admins, were excited to explore the new technology available: VADP (VMware vStorage API for Data

Protection).

Figure 1: (taken from Dell EMC® Avamar® 7.4 and Service Packs for VMware User Guide page 18)

With VADP, the backup system asks VMware to create a snapshot of VM disk, then the proxy VM reads

from the snapshot and writes to the backup media. In order for the image backup to work, the VM to be

backed up must be snapshot capable. We quickly got to work and started importing VM’s into Avamar

as image backup clients, forming groups and selecting the proxies deployed by VMware admins, for

backup groups. Many implementation requests were coming because VMware vCenter had a huge

number of VM’s. We tested backups. They were all running OK. Everything seemed in order until…

Page 4: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 4

Not All Proxies Are Created Equal

We noticed later that a few backups were taking a very long time to complete. Actually, most of the

backups were taking a long time to complete but we did not notice because many VM’s (virtual

machines) had small amounts of data to be backed up and subsequent backups finished quickly, thanks

to CBT (Changed Block Tracking) feature. And since the data change rate was very minimal, there was

very little data being written to backup media. Nevertheless, there were still some backups that were

taking a long time.

Upon checking logs, we came across the following message:

avvcbimage Info <12006>: hotadd transport is recommended for faster backups and restores and less

exposure to network routing, firewall and SSL certificate issues.

2018-01-06 18:01:29 avvcbimage Info <12007>: One or more of these items may be preventing use of

hotadd transport with this VM:

2018-01-06 18:01:29 avvcbimage Info <12008>: Check that the backup proxy

VM(127.0.0.2,acmeproxy03) is deployed on an ESX(i) host that has a path to the storage holding the

virtual disk being backed up….

The above messages did not come to our attention as they were not error messages. However, once we

started investigating why some backups were taking so long, we noticed the messages.

When we started implementation of VM image backups we had only very basic knowledge about

VMware. We didn’t know about datastore clusters, ESX clusters, hotadd mode, etc. Likewise, VMware

admins didn’t know about backups. We provided them ova files to deploy proxies as VM’s and they did

on some VMware hosts (physical computers on which VM’s run). And we picked proxies randomly for

backups.

It turned out that some backups with a lot of data were taking a long time because the proxies used

were inefficient. They did not have a direct path to the storage as the warning messages indicated:

Page 5: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 5

Figure 2: Inefficient use of proxy during backup

The data to be backed up has to travel all the way through VLAN in order to reach the proxy.

However, if the proxy is to be used efficiently, it should have a direct path to the datastore/datastore

cluster. It would not have to go through the VLAN:

Page 6: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 6

Figure 3: Efficient use of proxy

A backup taking 110 hours was reduced to 14 hours and another came down from 40 to 5 hours when

we specified an efficient proxy!

Even if backup completes fast due to relatively small amount of data and CBT, restore job would take a

long time without using an efficient proxy if the entire disk was lost!

Page 7: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 7

Using inefficient proxy may be an obvious problem to avoid for experienced Avamar and VMware

administrators. However, an Avamar administrator may be implementing VM image backups first time

or an VMware admin may be not aware of the fact that s/he has to deploy the proxy VM as requested

by the Avamar admin, on the right VMware host having path to the datastore of the VM to be backed

up.

Inefficient use of proxies might exist for various reasons:

Avamar admin having inherited an Avamar server with these issues.

Avamar admin having mistakenly selected the wrong proxy by checking the box right under (or

above) the correct one in proxy list in Group settings.

VM being backed up through the correct proxy having been migrated to another VMware host

cluster without informing the Avamar admin. Thus, the once correct proxy no longer having path

to the VM’s datastore.

Suppose we want to implement backup for a new VM or find out whether an efficient proxy is used for

already existing backup. How are we supposed to find it?

Page 8: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 8

Finding the Efficient Proxy

To find the efficient proxy for the backup of a specific VM, one way is to use vSphere. Here are the steps:

1. Search for the VM:

2. Once the VM is found, click Summary tab

This VM’s datastore is ACMESTOR3

3. ouble click on that datastore or datastore cluster having disks of the VM. That will take you to

the datastores. Click on the “Hosts” tab:

On the right pane are the hosts (ESX) having direct path to the selected datastore

4. Double click on a host listed. It will take you that host. Then click on “VM’s” tab:

See if you can visually spot a proxy VM among the VM’s listed under that host. If not found repeat the

steps above for remaining 2 hosts. If a Proxy VM(s) is found, that is an efficient proxy because it is in the

host cluster which has direct path to the datastore (or datastore cluster)

Page 9: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 9

If no proxy is found, backup admin should ask VMware admin to deploy a new proxy on that host

cluster.

The above steps are tedious and require using vSphere. Perhaps an easier way is to keep a list of proxies

versus datastore clusters in an Excel document. This way, if we are given the name of the datastore

cluster where the VM’s disks are, we can easily search the document and find the efficient proxy.

However, the Excel document would have to be updated every time VMware admins added (or

removed) a host or datastore cluster to the vCenter. And if a VM is moved to another host/datastore

cluster, its backup would be rendered inefficient, using the proxy of the previous host cluster. Day-by-

day such a document would become out of date unless it is manually kept up to date with the

cooperation of VMware admins.

An Easy Solution

To easily determine which proxy to use, we developed 2 scripts by using VMware vSphere Perl SDK

(Software Development Kit) in order to easily determine the efficient proxies for VM image backups:

- findproxy.pl: When entering VM’s name it returns the appropriate proxy(ies) to be used for

image backups

- proxypathcheck.pl: It does a proxy health check for all VM image backups configured for a

vCenter and reports whether a proxy used for backup is a good one or not

(Scripts are provided in the appendix)

findproxy.pl does the following:

- checks to see if VM is snapshot capable. If not, it issues a warning because image backup is not

possible unless the VM is snapshot capable. The script issues a warning for VM’s, when right-

clicked on vSphere, “Take Snapshot” is greyed out, i.e., which are not snapshot capable:

- contacts Avamar server and gets the list of proxies registered in Avamar

- follows the logic of steps listed earlier in this document under the header “FINDING AN

EFFICENT PROXY” and it searches VM list for each host for the proxy names obtained from

Avamar at the step directly above

- if matches are found, they are displayed as the efficient proxy(ies) to back up the VM

Page 10: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 10

Example:

C:\>findproxy.pl –config AcmeVC.txt –vm acmevm12

Searching.. Please wait

Proxies found: acmeproxy03.com

If VM is not snapshot capable, a warning is issued:

C:\>findproxy.pl –config AcmeVC.txt –vm acmevm05

Searching.. Please wait

WARNING: VM may NOT be snapshot capable. Please check!

Proxies found: acmeproxy14.stc.com.sa

findproxy.pl makes VM image backup implementation a lot easier. It is now an essential part of our

image backup implementation procedure.

proxypaththcheck.pl does a proxy health check for all VM image backups in Avamar by

- first obtaining from the Avamar server a list of all VM image backups and their proxies used.

- then using the same logic findproxy.pl uses to find the efficient proxy for each VM image backed

up

- and comparing the proxy used for each VM’s backup to the efficient proxy found in the previous

step to see an efficient proxy is used or not

Example:

C:\>proxypathcheck.pl –config AcmeVC.txt > AcmeVMbackups.csv

The output with comma separated values can be imported to an Excel document:

“Direct Path?” column is YES if the proxy used during the last backup job is efficient. Otherwise it is NO

and efficient proxy(ies) are suggested in the next column in order for the backup admin to specify in the

backup group of the VM.

Avamar admins can use this program for an overall health check of proxy usage and easily change

settings in the Avamar server to use correct proxies by referring to the report. Since VMware

environment is dynamic with vMotions, adding/decommissioning datastores, some backups once using

Page 11: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 11

efficient proxies might end up inefficient and slow. It may be a good idea to run this program and

generate the report periodically, say every month or two, to see if all VM image backups use efficient

proxies. You may be surprised with the results!

When we first ran proxypathcheck.pl, we found that 443 VM image backups out of total 944 backups

used inefficient proxies!!

The Excel document above can also be used to analyze and see how many VM’s using the same efficient

proxy can be distributed among other efficient proxies if any. This way VM client wait time for a proxy

can be reduced or eliminated.

Page 12: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 12

Conclusion and Some Suggestions

Using efficient proxies – especially in vCenters having many VM’s and datastore clusters – is something

backup admins should be aware of in order to run backups/restores faster.

Backup admins can follow the steps given under the header “FINDING AN EFFICENT PROXY” by using

vSphere and find the efficient proxy to use for a backup. Better yet, findproxy.pl program can be used.

The program also warns if VM to be image backed up is not snapshot capable.

It is particularly important to run findproxy.pl in order to specify an efficient proxy for a VM before

restoring its disk.

An overall proxy health check can be done by using proxypathcheck.pl and if some backups using

inefficient proxies are found, they can be configured to use efficient proxies.

Here are some suggestions. They are valid for Avamar in particular, and for any backup system using

VADP in general:

Proxy selection in backup system should be client property/attribute, rather than that of group.

In other words, proxy should be specified at client level.

Since backup system has access to vCenter, it can contact vCenter and run a query similar to the

function of the program findproxy.pl and find out the appropriate proxy(ies) for a VM client.

There could be a button on the backup system GUI that, when clicked, lists the recommended

proxies to use.

Backup system can reject a VM as an image backup client by checking with vCenter and finding

out that it is not snapshot capable.

When a proxy is selected by the backup admin, the backup system can contact vCenter and

warn the admin if it was not an efficient proxy.

If an efficient proxy has more image backups than it can handle, some VM backups are queued

up. In order to optimize proxy usage and prevent queuing, backup system can distribute VM’s

among the efficient proxies, if more than one efficient proxy is available.

Page 13: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 13

Appendix

Prerequisites in order to run the scripts

Download and Install VMware vSphere Perl SDK on your windows desktop

Download and Install PostgreSQL on your desktop

Make sure that port 443 is open between your desktop and vCenter

Make sure that port 5555 is open between your desktop and Avamar utility node

Proxy VM name and Avamar proxy client name MUST be the same

Create this file: %APPDATA%\postgresql\pgpass.conf The content of the pgpass.conf:

<name or IP address of Avamar utility node>:5555:mcdb:viewuser:viewuser1

Example: avamar.acme.com: 5555:mcdb:viewuser:viewuser1

Where avamar.acme.com is the Avamar utility node name. It should be

resolvable by DNS or it should be in your desktop’s hosts file.

Create this file in, for example, C:\ProxyScripts\<vCenter name>.txt. For instance, AcmeVC.txt

with this content:

VI_SERVER = vCenter.acme.com

VI_USERNAME = avauser

VI_PASSWORD = xxxxxx

VI_AVAUTILNODE = avamar.acme.com

VI_PROTOCOL = https

VI_PORTNUMBER = 443

Where VI_SERVER (vCenter.acme.com) is the name of vCenter. It should be resolvable into IP

address through DNS or local hosts file and VI_USERNAME (avauser) is the user to access the

vCenter

Once the above steps are done, the scripts can be run.

Scripts

findproxy.pl

#!/usr/bin/perl

#

#

# This program, given a VM name, finds efficient proxy(es), if any, for the image backup, by contacting

# the Avamar server and vCenter

#

Page 14: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 14

# Usage: findproxy.pl --config <config file name in a secure folder>

# --avahost <Avamar utility node name or IP>

# --vm <vmname>

# If avahost is not specified the default below is used. If vm name is not specified, the script prompts for

it

# Example:

# proxypathcheck.pl --config C:\securedir\vcenteracme.txt

# Enter VM name or hit enter to exit:

#

# Prerequisites for this program to work:

# - download and install VMware SDK for Perl from VMware (for Windows)

# - download and install PostgreSQL (for Windows)

# - firewall port 443 must be open from your computer to vCenter

# - firewall port 5555 must be open from your computer o the Avamar utility node

# - create a text file in a secure directory for vCenter credentials. For example:

# C:\securedir\vcenter.acme.txt

# with these contents:

# VI_SERVER = vcenter.acme.com

# VI_USERNAME = avauser

# VI_PASSWORD = xxxxxxxx

# VI_AVAUTILNODE = avamar.acme.com

# VI_PROTOCOL = https

# VI_PORTNUMBER = 443

# - This program must have permission to the secure directory

# - create this file on your Windows computer for Avamar MCDB access with PostgreSQL:

# %APPDATA%\postgresql\pgpass.conf

# with these contents:

# avamar.acme.com:5555:mcdb:viewuser:viewuser1

Page 15: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 15

# - make sure that both vCenter and Avamar names are resolvable by either hosts file or

DNS

# - make sure that vCenter name in the hosts file or DNS is the same as the vCenter name

# defined in Avamar as domain

# - proxy names registered in Avamar MUST be the same as their VM names

#

use strict;

use warnings;

use VMware::VIRuntime;

#

my %opts = (

vm => {

type => "=s",

help => "VM name",

required => 0,

},

avahost => {

type => "=s",

help => "Avamar utility node host name)",

required => 0,

default => "riy1-ava-un",

},

);

Opts::add_options(%opts);

Opts::parse();

Page 16: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 16

Util::connect();

my $vmname = Opts::get_option('vm');

my $ava_hostname = Opts::get_option('avahost');

my $psql_bindir = 'C:\"Program Files"\PostgreSQL\9.5\bin';

my $psql = $psql_bindir."/psql -t -h ".$ava_hostname." -p 5555 -d mcdb -U viewuser -c ";

if ( !defined $vmname ) {

print "Enter VM name or hit enter to exit: \n\n";

$vmname = <STDIN>;

chomp($vmname);

if ($vmname eq "") {exit;}

$vmname =~ s/^\s+|\s+$//g;

}

print "\nSearching.. Please wait \n\n";

# read/validate options and connect to the server

Opts::parse();

Opts::validate();

Util::connect();

# get all proxy clients from Avamar server

my @proxylist = `$psql \"select client_name from v_clients where (client_type='VPROXY_REGULAR' and

registered)\"`;

chomp(@proxylist);

s{^\s+|\s+$}{}g foreach @proxylist;

Page 17: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 17

my $vm_view = Vim::find_entity_view(view_type => 'VirtualMachine',

filter => { 'name' => qr/$vmname/i },

properties => ['datastore','config.hardware.device'] );

die "No response from vCenter or no VM with that name" if !defined $vm_view;

my @devs = @{ $vm_view->{'config.hardware.device'} };

# check SCSI Controller

my @scsi = grep { $_->isa('ParaVirtualSCSIController') } @devs;

my %vSharing;

foreach my $controller (@scsi) {

%vSharing = %{ $controller->sharedBus };

if ( $vSharing{val} eq 'physicalSharing' ) {

print "\nWARNING: VM may NOT be snapshot capable. Please check! \n\n";

}

}

# check LSI Controller

my @lsi = grep { $_->isa('VirtualLsiLogicSASController') } @devs;

foreach my $controller (@lsi) {

%vSharing = %{ $controller->sharedBus };

if ( $vSharing{val} eq 'physicalSharing' ) {

print "\nWARNING: VM may NOT be snapshot capable. Please check! \n\n";

}

}

Page 18: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 18

# now let's go from VirtualMachine to datastore[] to host[] to VM's and see if there is a proxy among

them

my @ds_mo = @{$vm_view->datastore};

my $ds_obj = Vim::get_view(mo_ref => $ds_mo[0], properties => ['host']); # first ref is enough

die "No response from vCenter or no dataset object returned" if !defined $ds_obj;

my @host_objects = @{$ds_obj->host};

my $host_mo;

my $host_obj;

my @vm_mo;

my $vm_mo_ref;

my $host;

my $vm;

my @vms=();

foreach $host_obj (@host_objects) {

$host_mo = $host_obj->key;

$host = Vim::get_view(mo_ref => $host_mo, properties => ['vm']);

if (!defined $host) {die "No response from vCenter or no host object returned"};

next if !defined $host->vm;

@vm_mo = @{$host->vm};

foreach $vm_mo_ref (@vm_mo) {

$vm = Vim::get_view(mo_ref => $vm_mo_ref, properties => ['name']);

Page 19: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 19

die "No response from vCenter or no vm object returned" if !defined $vm;

push @vms,$vm->name;

}

}

my @GoodProxies =

grep { defined }

@{ { map { lc ,=> $_ } @proxylist } }

{ map { lc } @vms };

if (@GoodProxies) {

print "Proxies found: @GoodProxies \n";

}

else { print "No proxy found \n"; }

# disconnect from the server

Util::disconnect();

exit;

proxypathcheck.pl

#!/usr/bin/perl

#

# This program obtains VM image backup info from Avamar MCDB and checks it against proxy info

# from vCenter in order to see if the proxy used for backup has direct path to the VM datastore.

# It prints the results and suggests proxies with path if the last proxy used for backup

# is not good (has no path). The results printed are in csv format, to be imported into

# an Excel document

Page 20: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 20

#

# Usage: proxypathcheck.pl --config <config file name in a secure folder>

# --avautilnode <Avamar utility node name>

# Example:

# proxypathcheck.pl --config C:\securedir\vcenteracme.txt --avautilnode

avamar.acme.com

#

# if Avamar utility node name is specified in the config file with VI_AVAUTILNODE (see

below)

# this usage is enough:

#

# proxypathcheck.pl --config C:\securedir\vcenteracme.txt > output.csv

#

# Prerequisites for this program to work:

# - download and install VMware SDK for Perl from VMware (for Windows)

# - download and install PostgreSQL (for Windows)

# - firewall port 443 must be open to vCenter

# - firewall port 5555 must be open to the Avamar utility node

# - create a text file in a secure directory for vCenter credentials. For example:

# C:\securedir\vcenter.acme.txt

# with these contents:

# VI_SERVER = vcenter.acme.com

# VI_USERNAME = avauser

# VI_PASSWORD = PwdXyz123

# VI_AVAUTILNODE = avamar.acme.com

# VI_PROTOCOL = https

# VI_PORTNUMBER = 443

# - This program must have permission to the secure directory

# - create this file on your Windows computer for Avamar MCDB access with PostgreSQL:

Page 21: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 21

# %APPDATA%\postgresql\pgpass.conf

# with these contents:

# avamar.acme.com:5555:mcdb:viewuser:viewuser1

# - make sure that both vCenter and Avamar names are resolvable by either hosts file or

DNS

# - make sure that vCenter name in the hosts file or DNS is the same as the vCenter name

# defined in Avamar as domain

# - proxy names registered in Avamar must be the same as their VM names

#

use strict;

use warnings;

use VMware::VIRuntime;

my $psql_bindir = 'C:\"Program Files"\PostgreSQL\9.5\bin'; # change if PostgreSQL is installed

somewhere else

# in a folder other than default

# do NOT modify below this point!

# read/validate options and connect to the vCenter server

my %opts = (

avautilnode => {

type => "=s",

variable => "VI_AVAUTILNODE",

help => "Avamar utility node hostname or IP address",

required => 1,

},

);

Page 22: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 22

Opts::add_options(%opts);

Opts::parse();

Opts::validate();

Util::connect();

my $ava_hostname = Opts::get_option('avautilnode');

my $vcenter = Opts::get_option('server');

my $psql = $psql_bindir."/psql -t -h ".$ava_hostname." -p 5555 -d mcdb -U viewuser -c ";

# get all proxy clients from Avamar utility node

my @proxylist = `$psql \"select display_client_name from v_clients_2 where

(client_type='VPROXY_REGULAR' and registered = true)\"` or die "No proxy is found in Avamar server:

$ava_hostname or Avamar mcdb is not responding";

splice(@proxylist,$#proxylist,1); # remove the last element which is empty

chomp(@proxylist);

s{^\s+|\s+$}{}g foreach @proxylist;

# get all VMs backed up as image from Avamar utility node

my @VMs = `$psql \"select client_name,display_client_name from v_clients_2 where

(client_type='VMACHINE' and enabled=true and display_full_domain ~* \'$vcenter\')\"` or die "No

virtual machines found in $ava_hostname for vCenter $vcenter or Avamar mcdb is not responding";

splice(@VMs,$#VMs,1); # remove the last blank line

# create hash: client name vs display client name

my $client_name; my $display_client_name; my %displayname;

foreach (@VMs) {

Page 23: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 23

($client_name,$display_client_name)=split /\|/;

$client_name =~ s/^\s+|\s+$//g; # trim leading/trailing space

chomp($display_client_name);

$display_client_name =~ s/^\s+|\s+$//g;

$displayname{$client_name} = $display_client_name;

}

# build last used proxy and activity list

my $lastproxy; my $proxy_cid; my %lastusedproxy; my $client_activity;

my $started_ts; my $completed_ts; my $status_code; my $bytes_new; my $elapsed;

my %start_time; my %end_time; my %status; my %new_bytes; my %elapsed_time; my %MBperSec;

foreach $client_name (keys(%displayname)) {

$client_activity = `$psql \"select proxy_cid,started_ts,completed_ts,status_code,bytes_new,

extract(epoch from (completed_ts - started_ts)) from v_activities_2 where

(client_name=\'$client_name\' and proxy_cid != \' \') order by completed_ts desc limit 1"` or die "No

virtual machines found in $ava_hostname for vCenter $vcenter or Avamar mcdb is not responding";

chomp($client_activity);

($proxy_cid,$started_ts,$completed_ts,$status_code,$bytes_new,$elapsed) = split

/\|/,$client_activity;

next if !defined $proxy_cid; # skip if no backup activity for this client

chomp($elapsed);

$proxy_cid =~ s/^\s+|\s+$//g;

$lastproxy = `$psql \"select display_client_name from v_clients_2 where (cid=\'$proxy_cid\')\"`

or die "No virtual machines found in $ava_hostname for vCenter $vcenter or Avamar mcdb is not

responding";

chomp($lastproxy);

$lastproxy =~ s/^\s+|\s+$//g;

next if $lastproxy eq ""; # skip if no backup activity for this client

Page 24: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 24

$lastusedproxy{$client_name} = $lastproxy;

$start_time{$client_name} = $started_ts;

$end_time{$client_name} = $completed_ts;

$status{$client_name} = $status_code;

$new_bytes{$client_name} = $bytes_new;

$elapsed_time{$client_name} = $elapsed;

$MBperSec{$client_name} = $bytes_new / $elapsed /1024/1024;

}

print "VM name, Start Time, End Time, Elapsed Time (sec), New Bytes, MB/s, Status Code, Last Used

Proxy, Direct Path?, Suggested Proxi(es) \n";

my $vmname; my @ds_mo; my $ds_obj; my $vm_view; my @host_objects; my $host_mo; my

@vm_mo;

my $vm_mo_ref; my $host; my $vm; my @vms; my @GoodProxies; my $host_obj;

foreach $client_name (keys(%lastusedproxy)) {

$vmname = $displayname{$client_name};

$vm_view = Vim::find_entity_view(view_type => 'VirtualMachine',

filter => { 'name' => qr/$vmname/i },

properties => ['datastore'] );

next if !defined $vm_view; # maybe that VM known by Avamar no longer exists in

vCenter

@ds_mo = @{$vm_view->datastore};

$ds_obj = Vim::get_view(mo_ref => $ds_mo[0], properties => ['host']); # first ref is enough

die "No response from vCenter or no dataset object returned" if !defined $ds_obj;

@host_objects = @{$ds_obj->host};

@vms=();

Page 25: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 25

foreach $host_obj (@host_objects) {

$host_mo = $host_obj->key;

$host = Vim::get_view(mo_ref => $host_mo, properties => ['vm']);

die "No response from vCenter or no host object returned" if !defined $host;

next if !defined $host->vm; # host may exist but may be offline

@vm_mo = @{$host->vm};

foreach $vm_mo_ref (@vm_mo) {

$vm = Vim::get_view(mo_ref => $vm_mo_ref, properties => ['name']);

die "No response from vCenter or no vm object returned" if !defined $vm;

push @vms,$vm->name;

}

}

@GoodProxies =

grep { defined }

@{ { map { lc ,=> $_ } @proxylist } }

{ map { lc } @vms };

if ( $lastusedproxy{$client_name} ~~ @GoodProxies ) {

print

"$displayname{$client_name},$start_time{$client_name},$end_time{$client_name},$elapsed_time{$cli

ent_name},$new_bytes{$client_name},$MBperSec{$client_name},$status{$client_name},$lastusedprox

y{$client_name},YES \n";

}

else {

Page 26: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 26

print

"$displayname{$client_name},$start_time{$client_name},$end_time{$client_name},$elapsed_time{$cli

ent_name},$new_bytes{$client_name},$MBperSec{$client_name},$status{$client_name},$lastusedprox

y{$client_name},NO,@GoodProxies \n";

}

}

# disconnect from the server

Util::disconnect();

exit;

Page 27: OPTIMIZATION OF VM IMAGE BACKUPS WITH RIGHT PROXIES · avvcbimage Info : hotadd transport is recommended for faster backups and restores and less ... above) the correct

2018 Dell EMC Proven Professional Knowledge Sharing 27

Dell EMC believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” DELL EMC MAKES NO

RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS

PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS

FOR A PARTICULAR PURPOSE.

Use, copying and distribution of any Dell EMC software described in this publication requires an

applicable software license.

Dell, EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries.