sas visual analytics 7.3 installation and administration guide · sas visual analytics includes the...

SAS Visual Analytics 7.3 Installation and

Administration Guide

I. Pre-Installation Checklist

II. Password-less SSH for Root

III. HPCMC Installation

IV. Creating Users with Public/Private SSH

keys

V. Hadoop and TKGrid Installation

VI. SAS Visual Analytics Installation

VII. Register LASRadm to Metadata

VIII. Validation

IX. Start-up and Stopping of Services

I. Pre-installation Checklist

Prior to installation, SAS will provide a pre-installation checklist that must be validated before

starting the activity. Installation should not start without completing the checklist. Below is a

sample pre-install checklist for VA 7.3

Operating System Configuration & Patches (Initial)

Requirement: Expected Value:

Red Hat Enterprise Linux 6 (64-Bit version) RHEL 6

Python 2.6

Korn Shell ksh-20100621-6.el6.x86_64

SELINUX Permissive

PERL-NET-SSLEAY PERL-NET-SSLEAY-1.35-9.el6.x86_64 (Blade 1)

X11 Forwarding Enabled

Umask value for sasinst account 22

Ulimit for sasinst user Open Files = (-n) 350000

Max User Processes = (-u) 65536

Installed Libraries & RPMs

numactl-2.0.3-9.el6.x86_64

libXext-1.1-3.el6.x86_64

libXext-1.1-3.el6.i686

libXp-1.0.0-15.1.el6.x86_64

libXp-1.0.0-15.1.el6.i686

libXtst-1.0.99.2-3.el6.x86_64

libXtst-1.0.99.2-3.el6.i686

libgcc 32 & 64 bit versions

libXext-1.1-3.el6.i686

libXp-1.0.0-15.1.el6.i686

libXtst-1.0.99.2-3.el6.i686

Initial Operating System Accounts

root

sasinst

sassrv

sasdemo

Group sasusers

II. Password-less access for root (SSH setup)

The root user account must have password-less secure shell (SSH) access amongst all the

machines in the grid.

0) - Add /etc/hosts

- Turn off firewall

- The same root password

- The same OS version

1) Log in to the SAS node, Nimitz1 and switch to root user.

su -

<password>

2) Generate the SSH keys for root

cd ~

ssh-keygen -t rsa

Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/id_rsa

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

3) Issue the “cat” system command to create the authorized_keys file.

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

4) Verify settings.

ls -al .ssh

a. The ownership of the /root/.ssh dir should be 700 (drwx------)

b. id_rsa: root’s private key. Owned by root with a mode of 600 (-rw-------)

c. id_rsa.pub: root’s public key. Owned by root with a mode of 644 (-rw-r--r--)

You should get a listing similar to the following:

[root@Nimitz1 .ssh]# ls -al .ssh

total 24

drwx------. 2 root root 4096 Dec 30 07:14 .

dr-xr-x---. 26 root root 4096 Dec 30 07:13 ..

-rw-r--r--. 1 root root 398 Dec 30 07:14 authorized_keys

-rw-------. 1 root root 1675 Dec 30 07:14 id_rsa

-rw-r--r--. 1 root root 398 Dec 30 07:14 id_rsa.pub

-rw-r--r--. 1 root root 407 Dec 30 2011 known_hosts

5) Use the scp command to copy the folder /root/.ssh from this server to the other nodes

assigned to the cluster, while maintaining permission and ownership. You are prompted for

the password of the root account each time.

scp -o StrictHostKeyChecking=no -r ~/.ssh/* Stennis2:~/.ssh

<password>

scp -o StrictHostKeyChecking=no -r ~/.ssh/* Vinsone3:~/.ssh

<password>

scp -o StrictHostKeyChecking=no -r ~/.ssh/* Reagan4:~/.ssh

<password>

This step - and the next one - also lay down the server’s public key that is necessary for

connecting without a password.

You should get a series of messages similar to the following:

Warning: Permanently added 'Stennis2,149.173.98.54' (RSA) to the list of known hosts.

root@Stennis2's password:

authorized_keys 100% 398 0.4KB/s 00:00

6) Execute the SSH command to each node on the cluster to test the ssh password-less

connection.

ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no Nimitz1

exit

ssh -o PasswordAuthentication=no Stennis2

exit

ssh -o PasswordAuthentication=no Vinsone3

exit

ssh -o PasswordAuthentication=no Reagan4

exit

With each of these commands, you are explicitly disabling the use of a password to force

the use of the newly created public keys.

The first line in the sequence opens an SSH connection from Nimitz1 back to itself.

You see the following:

[root@Nimitz1 .ssh]# ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no

Nimitz1

Warning: Permanently added 'Nimitz1,149.173.96.126' (RSA) to the list of known hosts.

Last login: Mon Dec 12 03:19:28 2012 from Nimitz1

[root@Nimitz1 ~]# exit

logout

Connection to Nimitz1 closed.

III. SAS High-Performance Management Console

SAS Visual Analytics includes the SAS High-Performance Computing Management Console to

handle tasks that are typically performed by a Linux system administrator; customers with Linux

savvy system administrators can choose not to use the SAS HPC MC at all.

This is a completely different application from the SAS Management Console: it does not

connect to or update any metadata in the SAS Metadata Server.

We use it for creating and propagating user accounts and SSH keys across the environment.

The SAS HPC Management Console is installed on the first server and is a web application on

the URL http://Nimitz1:10020

3.1 Installation of HPC Management Console (HPC MC) 1) Log in to the SAS node, Nimitz1, as the sasinst user, and switch to root.

su -

<password>

2) Install the HPC MC with the following commands.

cd /tmp/sasdepot/depot/

cd standalone_installs/SAS_High-Performance_Management_Console/2_1/Linux_for_x64/

rpm -ivh sashpcmc-2.1.x86_64.rpm

You should get the following:

[root@Nimitz1 Linux_for_x64]# rpm -ivh sashpcmc-2.1.x86_64.rpm

Preparing... ##################################### [100%]

1:sashpcmc ##################################### [100%]

3) To configure the HPC MC run the setup script in /opt/webmin/utilbin

cd /opt/webmin/utilbin/

./setup

Provide the following details:

hpcadmin

yes

lnxsas

lnxsas

no


[root@Nimitz1 utilbin]# ./setup

Enter the username for initial login to SAS HPC MC below. This user will have rights to

everything

in the SAS HPC MC and can either be an OS account or new console user. If an OS

account exists for the user,

then system authentication will be used. If an OS account does not exist, you will be

prompted for a password.

hpcadmin

hpcadmin is not a OS account. Proceed with setting up account as console user only

(yes|no)?

yes

Enter password for user hpcadmin

Confirm password for user hpcadmin

Creating hpcadmin using password

Use SSL\HTTPS (yes|no)?

no

Finished!

4) To test if the setup is successful, start the console with the service1 command:

service sashpcmc status

service sashpcmc start


[root@Nimitz1 Linux_for_x64]# service sashpcmc status

sashpcmc is stopped

[root@Nimitz1 Linux_for_x64]# service sashpcmc start

sashpcmc started

VERIFICATION STEPS:

1) The HPC MC is installed in /opt/webmin and the whole directory is owned by root. Check it:

1 It’s worth mentioning that the service command is a RHEL feature. If you want to use a command that works on

both SUSE and RHEL, use:

/etc/init.d/sashpcmc status

/etc/init.d/sashpcmc start

ls -l /opt/webmin/


[root@Nimitz1 utilbin]# ls -l /opt/webmin/

total 20

drwxr-xr-x. 23 root root 4096 Dec 30 07:57 etc

drwxr-xr-x. 2 root root 4096 Dec 30 07:58 logs

drwxr-xr-x. 2 root root 4096 Dec 30 07:57 utilbin

drwxr-xr-x. 3 root root 4096 Dec 30 07:57 var

drwxr-xr-x. 24 root root 4096 Dec 30 07:57 webroot

2) The HPC MC is configured to start automatically when the server starts. Issue the following

command.

chkconfig --list sashpcmc


[root@Nimitz1 Linux_for_x64]# chkconfig --list sashpcmc

sashpcmc 0:off 1:off 2:on 3:on 4:on 5:on 6:off

3) On the windows client, open a browser and point to the console: http://Nimitz1:10020/

4) Log in to the HPC Management Console with the hpcadmin account, and then log off.

3.2 Creating the gridhosts file with HPCMC 1) On the Windows machine, open Notepad++ to create a .txt file that contains a list of nodes.

You can copy and paste the following list:

Nimitz1

Stennis2

Vinsone3

Reagan4

2) Save and close the file in the /tmp directory (/tmp/gridhosts.txt)

3) Log in to the HPC Management Console with the hpcadmin account, and navigate to the

Generate Gridhosts File utility.

HPC Management Gridhosts File Management Generate Gridhosts File

tab

4) Select “Upload a file” button.

5) On the “Select File to Upload” tab, browse to the local gridhost file that was created in /tmp

http://sasserver01:10020/

6) Select Upload, you get a form with server names listed

7) Press “Select all”, Submit

8) Logoff the SAS HPC MC

VERIFICATION STEP:

1) From the SAS Node, verify that the gridhosts file was created.

cat /etc/gridhosts


[root@Nimitz1 Linux_for_x64]# cat /etc/gridhosts

Nimitz1

Stennis2

Vinsone3

Reagan4

3.2.1 Simultaneous Utility Commands (simsh and simcp) The simsh and simcp commands are installed with the SAS Visual Analytics software.

The following are some key facts about them:

Typically, executed from the head node.

The /etc/gridhosts file must exist.

The user executing simsh/simcp must have password-less SSH setup properly to every

node in the environment.

Installed with SAS HPC Management Console: /opt/webmin/utilbin

The simsh command uses secure shell to invoke the specified command on every

machine that is listed in the /etc/gridhosts file.

The simsh command can also be used to simplify installation invoking the rpm or

another package installation command on each machine in the cluster.

The simcp command is used to copy a file from one machine to the other machines

listed in /etc/gridhosts file.

1) Log in to the SAS node, Nimitz1, as the sasinst user, and then switch to root.

su -

<password>

2) Test a simultaneous command.

/opt/webmin/utilbin/simsh hostname


[root@Nimitz1 ~]# /opt/webmin/utilbin/simsh hostname

Vinsone3: Vinsone3

Stennis2: Stennis2

Nimitz1: Nimitz1

Reagan4: Reagan4

Note that the order in which hosts execute the command is not guaranteed.

IV. Creating Users with Public/Private SSH keys

sasinst – SAS installation account;

lasradm – data prep admin to load/lift data

sasdemo - SAS VA end user

sassrv - SAS Spawned Servers account

hadoop - Hadoop user – Starts Hadoop services

4.1 sasinst

This account is used to install;

1) SAS Servers

2) SAS LASR Analytic Server

To install the SAS LASR Analytic Server SSH from any node to any node will be required.

Therefore, the private SSH key and authorized_keys file will be required on all nodes.

In our environment the account already exists on all servers because it is in the pre-installation

checklist; this is to mimic scenarios where consultants do not directly log in to servers with root

account, but rather use the SAS Installation account and then “su” to other ones.

1) From a browser on the windows server, log in to HPCMC as hpcadmin / SASpw123 -

http://Nimitz1:10020/

2) Select HPC Management Users and Groups

3) Select the sasinst user from the table by clicking on the sasinst name within the table

4) Select Generate and Propagate SSH Keys and select Save

4.2 lasradm

This user exists on all nodes of the cluster included the SAS node.

The public and private keys are generated for this user on the SAS node and copied to

all nodes of the environment.


This user must be able to ssh from the SAS node to all nodes on the cluster, as well as

ssh amongst all the nodes in the cluster.

An account is created in the SAS Metadata for this login.

1) From a browser on the windows server, log in to HPCMC as hpcadmin / SASpw123

(http://Nimitz1:10020/)


3) Select Create a new user and complete with values as per the following screenshot (enter

SASpw123 in the password field)

4) Select Create. You should get a confirmation page for it.

5) Verification Step

Log in to the SAS node, Nimitz1, su to lasradm and submit a remote command to each host:

su - lasradm

<password>

ssh -o StrictHostKeyChecking=no Nimitz1 -C "hostname"

ssh -o StrictHostKeyChecking=no Stennis2 -C "hostname"


ssh -o StrictHostKeyChecking=no Vinsone3 -C "hostname"

ssh -o StrictHostKeyChecking=no Reagan4 -C "hostname"

If correct you do not get a password request:

$ ssh -o StrictHostKeyChecking=no Nimitz1 -C "hostname"

Warning: Permanently added 'Nimitz1,129.8.160.202' (RSA) to the list of known hosts.

Nimitz1

$ ssh -o StrictHostKeyChecking=no Stennis2 -C "hostname"

Warning: Permanently added 'Stennis2,129.8.160.203' (RSA) to the list of known hosts.

Stennis2

$ ssh -o StrictHostKeyChecking=no Vinsone3 -C "hostname"

Warning: Permanently added 'Vinsone3,129.8.160.204' (RSA) to the list of known hosts.

Vinsone3

$ ssh -o StrictHostKeyChecking=no Reagan4 -C "hostname"

Warning: Permanently added 'Reagan4,129.8.160.205' (RSA) to the list of known hosts.

Reagan4.

4.3 sasdemo

Used to log in to the SAS Visual Analytics Web applications.

Only needed on the SAS node, is not needed on the cluster nodes.


No ssh keys are created on any of the nodes.


checklist;

1) Log in to the SAS node, Nimitz1, and su to root user

su –

<password>

2) Use the simsh command to simultaneously check the uid:

/opt/webmin/utilbin/simsh cat /etc/passwd | grep sasdemo

You should get a listing similar to the following :

[root@Nimitz1 ~]# /opt/webmin/utilbin/simsh cat /etc/passwd | grep sasdemo

Nimitz1: sasdemo:x:503:504::/home/sasdemo:/bin/bash

Vinsone3: sasdemo:x:503:504::/home/sasdemo:/bin/bash

Stennis2: sasdemo:x:503:504::/home/sasdemo:/bin/bash

Reagan4: sasdemo:x:503:504::/home/sasdemo:/bin/bash

3) Use the simsh command to simultaneously verify that sasdemo has no ssh keys set up.

/opt/webmin/utilbin/simsh ls -la /home/sasdemo/.ssh


[root@Nimitz1 ~]# /opt/webmin/utilbin/simsh ls -la /home/sasdemo/.ssh

ls: cannot access /home/sasdemo/.ssh: No such file or directory




4.4 sassrv

Shared login used by standard BI servers.

Only needed on the SAS node, is not needed on the cluster nodes.


No ssh keys are created on any of the nodes.


checklist; this is to mimic scenarios where consultants do not directly log in to servers with root

account, but rather use the SAS Installation account and.

1) Log in to the SAS node, Nimitz1, and su to root user

su –

<password>

2) Use the simsh command to simultaneously check the uid:

/opt/webmin/utilbin/simsh cat /etc/passwd | grep sassrv


[root@Nimitz1 ~]# /opt/webmin/utilbin/simsh cat /etc/passwd | grep sassrv

Nimitz1: sassrv:x:505:504::/home/sassrv:/bin/bash

Stennis2: sassrv:x:505:504::/home/sassrv:/bin/bash

Reagan4: sassrv:x:505:504::/home/sassrv:/bin/bash

Vinsone3: sassrv:x:505:504::/home/sassrv:/bin/bash

3) Use the simsh command to simultaneously verify that sassrv has no ssh keys set up.

/opt/webmin/utilbin/simsh ls -la /home/sassrv/.ssh


[root@Nimitz1 ~]# /opt/webmin/utilbin/simsh ls -la /home/sassrv/.ssh

ls: cannot access /home/sassrv/.ssh: No such file or directory




4.5 hadoop

The account used to run the Hadoop Distributed File System (HDFS) will be hadoop. The

software will be both installed and run using this user account. The Hadoop user requires SSH

from any node to any node to complete the installation. The installer does not create this

account automatically and it must be created manually using the HPCMC.

1) From a browser on the windows server, log in to HPCMC as hpcadmin / SASpw123 -

http://Nimitz1:10020/


3) Select Create a new user and complete with values as per the following screenshot (enter

SASpw123 in the password field)


4) Select Create. You should get a confirmation page.

VERIFICATION STEPS

[root@Nimitz1 ~]# su – Hadoop

ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no Nimitz1

exit

ssh -o PasswordAuthentication=no Stennis2

exit

ssh -o PasswordAuthentication=no Reagan4

exit

ssh -o PasswordAuthentication=no Vinsone3

exit

V. Hadoop and TKGrid Installation

5.1 Create the Installation Directory

Login to the Hadoop Name node, Nimitz1, as the sasinst user and change to root

su -

<password>

Use the simsh script to create the hadoop installation directory on all nodes

Note : /Hadoop should be in the large drive

Example : /data/Hadoop

Default Hadoop data is : /Hadoop/Hadoop-data

/opt/webmin/utilbin/simsh mkdir /hadoop

/opt/webmin/utilbin/simsh chown hadoop:hadoop /hadoop

5.2 Install the SAS Visual Analytics Hadoop Software

Login to the Hadoop Name node, Nimitz1, as the root user, then change to the hadoop

user, use the following command as root

su - hadoop

cd ~

Copy the SAS High-Performance Deployment for Hadoop installer to the hadoop user’s

home directory

cp /tmp/sasdepot/depot/standalone_installs/SAS_High-

Performance_Deployment_for_Hadoop/2_1/Linux_for_x64/sashadoop.tar.gz .

Extract the installer

tar -xvf sashadoop.tar.gz

Run the installation script

cd sashadoop

./hadoopInstall

Use the values in the following table to install SAS High-Performance Deployment for

Hadoop

Question Response

Do you wish to use an existing Hadoop installation (y/N) N

Enter path to install Hadooop. The directory ‘hadoop-0.23.0’ will be created in

the path specified

/hadoop

Enter replication factor. Default 2 2

Enter port number for fs.defaultFS. Default 54310 <press enter>

Enter port number for mapred.job.tracker. Default 54311 <press enter>

Enter port number for dfs.namenode.https-address. Default 50470 <press enter>

Enter port number for dfs.datanode.https.address. Default 50475 <press enter>

Enter port number for dfs.datanode.address. Default 50010 <press enter>

Enter port number for dfs.datanode.ipc.address. Default 50020 <press enter>

Enter port number for dfs.namenode.http-address. Default 50070 <press enter>

Enter port number for dfs.datanode.http.address. Default 50075 <press enter>

Enter port number for dfs.secondary.http.address. Default 50090 <press enter>

Enter port number for dfs.namenode.backup.address. Default 50100 <press enter>

Enter port number for dfs.namenode.backup.http-address. Default 50105 <press enter>

Enter port number for mapred.job.tracker.http.address. Default 50030 <press enter>

Enter port number for mapred.task.tracker.http.address. Default 50060 <press enter>

Enter port number for com.sas.lasr.hadoop.service.namenode.port. Default

15452

<press enter>

Enter port number for com.sas.lasr.hadoop.service.datanode.port. Default

15453

<press enter>

Enter user that will be running the HDFS server process. hadoop

Enter path for JAVA_HOME directory. (Default: /usr/lib/jvm/jre) <press enter>

Enter path for Hadoop data directory. This should be on a large drive.

Default is '/hadoop/hadoop-data'.

<press enter>

Enter path for Hadoop system directory. Default is '/hadoop/hadoop-system'. <press enter>

Enter path for Hadoop local directory. Default is '/hadoop/hadoop-local'. <press enter>

Enter path for Hadoop name directory. Default is '/hadoop/hadoop-name'. <press enter>

Enter full path to machine list. The NameNode 'Nimitz1' should be listed first. /etc/gridhosts

You should see the following:

Creating 'hadoop-0.23.1' in '/hadoop'.

When the TKGrid installer prompts for 'HADOOP_HOME', enter '/hadoop/hadoop-0.23.1'.

Copying SAS jars to '/hadoop/hadoop-0.23.1/share/hadoop/hdfs/lib'.

Creating Hadoop tmp dir in '/hadoop/hadoop-0.23.1/tmp'.

Creating '/hadoop/hadoop-0.23.1/etc/hadoop/masters' file.

Creating '/hadoop/hadoop-0.23.1/etc/hadoop/slaves' file.

Adding SAS settings to '/hadoop/hadoop-0.23.1/etc/hadoop/hdfs-site.xml'.

mkdir -m 755 /hadoop/hadoop-local /hadoop/hadoop-data /hadoop/hadoop-name

/hadoop/hadoop-system

chown hadoop /hadoop/hadoop-local /hadoop/hadoop-data /hadoop/hadoop-name


mkdir -m 777 /hadoop/hadoop-0.23.1/logs

touch /hadoop/hadoop-0.23.1/logs/SecurityAuth.audit

chmod 777 /hadoop/hadoop-0.23.1/logs/SecurityAuth.audit

The installer can now copy '/hadoop/hadoop-0.23.1' to all the slave machines

using scp, skipping the first entry. Perform copy? (YES/no)

Answer “YES” and you should see the following as the files are copied to all nodes:

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 mkdir -p -m

755 /hadoop/hadoop-0.23.1

scp -o StrictHostKeyChecking=no -o PasswordAuthentication=no -q -r /hadoop/hadoop-

0.23.1/* Stennis2:/hadoop/hadoop-0.23.1/

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 chmod -R 755

/hadoop/hadoop-0.23.1


/hadoop/hadoop-0.23.1/logs /hadoop/hadoop-0.23.1/tmp

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 mkdir -m 755

/hadoop/hadoop-local /hadoop/hadoop-data /hadoop/hadoop-name /hadoop/hadoop-system

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 chown hadoop


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 mkdir -p -m


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 mkdir -p -m


scp -o StrictHostKeyChecking=no -o PasswordAuthentication=no -q -r /hadoop/hadoop-

0.23.1/* Reagan4:/hadoop/hadoop-0.23.1/

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 scp -o

StrictHostKeyChecking=no -o PasswordAuthentication=no -q -r /hadoop/hadoop-0.23.1/*

Vinsone3:/hadoop/hadoop-0.23.1/

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 chmod -R


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 chmod -R 755



777 /hadoop/hadoop-0.23.1/logs /hadoop/hadoop-0.23.1/tmp


/hadoop/hadoop-0.23.1/logs /hadoop/hadoop-0.23.1/tmp

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 mkdir -m 755


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 mkdir -m 755


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 chown

hadoop /hadoop/hadoop-local /hadoop/hadoop-data /hadoop/hadoop-name


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 chown

hadoop /hadoop/hadoop-local /hadoop/hadoop-data /hadoop/hadoop-name


Installation complete.

Please add the following line to /etc/sudoers on all nodes:

hadoop ALL=NOPASSWD:/bin/mkdir,/bin/chmod,/bin/chown,/bin/ln,/bin/rm

Start your HDFS as userid 'hadoop' with the following commands:

export HADOOP_HOME=/hadoop/hadoop-0.23.1

Run this command only if you want to format a new hadoop directory. It will erase existing

data.

$HADOOP_HOME/bin/hadoop namenode -format

$HADOOP_HOME/sbin/start-dfs.sh

VERIFICATION STEP:

Verify hadoop directory creation on all nodes:

/opt/webmin/utilbin/simsh ls -la /hadoop


[hadoop@Nimitz1 sashadoop]$ /opt/webmin/utilbin/simsh ls -la /hadoop

Nimitz1: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:24 hadoop-system

Nimitz1: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:24 hadoop-name

Nimitz1: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:24 hadoop-local

Nimitz1: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:24 hadoop-data

Nimitz1: drwxr-xr-x. 11 hadoop hadoop 4096 Oct 30 11:24 hadoop-0.23.1

Nimitz1: dr-xr-xr-x. 29 root root 4096 Oct 30 11:07 ..

Nimitz1: drwxr-xr-x. 7 hadoop hadoop 4096 Oct 30 11:24 .

Nimitz1: total 28

Nimitz1: Warning: Permanently added 'Nimitz1,10.96.3.64' (RSA) to the list of known hosts.

Stennis2: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:25 hadoop-system

Stennis2: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:25 hadoop-name

Stennis2: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:25 hadoop-local

Stennis2: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:25 hadoop-data

Stennis2: drwxr-xr-x. 11 hadoop hadoop 4096 Oct 30 11:25 hadoop-0.23.1

Stennis2: dr-xr-xr-x. 29 root root 4096 Oct 30 11:07 ..

Stennis2: drwxr-xr-x. 7 hadoop hadoop 4096 Oct 30 11:25 .

Stennis2: total 28

Vinsone3: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-system

Vinsone3: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-name

Vinsone3: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-local

Vinsone3: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-data

Vinsone3: drwxr-xr-x. 11 hadoop hadoop 4096 Oct 30 11:26 hadoop-0.23.1

Vinsone3: dr-xr-xr-x. 29 root root 4096 Oct 30 11:07 ..

Vinsone3: drwxr-xr-x. 7 hadoop hadoop 4096 Oct 30 11:26 .

Vinsone3: total 28

Reagan4: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-system

Reagan4: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-name

Reagan4: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-local

Reagan4: drwxr-xr-x. 2 hadoop hadoop 4096 Oct 30 11:26 hadoop-data

Reagan4: drwxr-xr-x. 11 hadoop hadoop 4096 Oct 30 11:26 hadoop-0.23.1

Reagan4: dr-xr-xr-x. 29 root root 4096 Oct 30 11:07 ..

Reagan4: drwxr-xr-x. 7 hadoop hadoop 4096 Oct 30 11:26 .

Reagan4: total 28

Change to the root user

su -

<password>

Update the /etc/sudoers file on each node, edit the file using the following command:

visudo

Add the following two lines to the bottom of the file:

Defaults:hadoop !requiretty

hadoop ALL=NOPASSWD:/bin/mkdir,/bin/chmod,/bin/chown,/bin/ln,/bin/rm

Save and exit the file. Use the following command to copy the file to all nodes:

/opt/webmin/utilbin/simcp /etc/sudoers /etc/sudoers

Change to the hadoop user

su - hadoop

Update the .bash_profile for the hadoop user to set the environment variable

$HADOOP_HOME

echo "export HADOOP_HOME=/hadoop/hadoop-2.4.0" >> ~/.bash_profile

Process the bash_profile to make the environment variable available

. ~/.bash_profile

Test :

echo $HADOOP_HOME


Format the Hadoop file system using the following command:

$HADOOP_HOME/bin/hdfs namenode -format

A large amount of output will be sent to the screen following the format command.

Make sure you see the message: " INFO namenode.NNStorage: Storage directory

/hadoop/hadoop-name has been successfully formatted", a few lines up from the

bottom.

Start the Hadoop file system


You should get some “starting” messages that report the startup of daemons and the

creation of log files. The actual number depends on the number of nodes in the cluster, so at

customer sites it may vary.

Also you will be prompted to continue connecting (yes/no) during the startup of the

processes. Always answer “yes”.

In this workshop environment there should be four “starting” lines. You can also verify that

the correct number of processes (nine) has started on all hosts with the following command:

/opt/webmin/utilbin/simsh ps -ef | egrep "(Node|Tracker)" | egrep -v grep | wc -l


[hadoop@Nimitz1 ~]$ simsh ps -ef | egrep "(Node|Tracker)" | egrep -v grep | wc -l

5

Still using the hadoop user account, create a directory in HDFS that permits Read and

Write access for all users:

$HADOOP_HOME/bin/hadoop fs -mkdir /hps

$HADOOP_HOME/bin/hadoop fs -chmod 777 /hps

VERIFICATION STEPS:

Validate the creation of your HDFS directory using:

$HADOOP_HOME/bin/hadoop fs -ls -R /


drwxrwxrwx - hadoop supergroup 0 2012-05-24 20:04 /hps

You can confirm that Hadoop is running successfully by opening a browser to

http:/Nimitz1:50070/dfshealth.jsp. Review the information in the cluster summary section of the

page. Confirm that the number of live nodes equals the number of DataNodes and that the

number of dead nodes is zero.

http://sasserver01:50070/dfshealth.jsp

*In some cases, you might see 0 live and dead nodes. This is due to the firewall settings.

Turn off the firewall/iptables and refresh the page to confirm the live nodes are seen.

Close the browser and exit from the hadoop user session.

exit

5.3 Install the SAS High-Performance Analytics components

Login to the LASR Head node, Nimitz1, as the sasinst user, then switch to root

su -

<password>

Create the installation directory /opt/TKGrid_2.1, on all nodes

/opt/webmin/utilbin/simsh mkdir /opt/TKGrid_2.1

/opt/webmin/utilbin/simsh chown sasinst:sas /opt/TKGrid_2.1

Change to the sasinst user

su - sasinst

Copy the installation file from the depot to the installation directory

cd /opt/TKGrid_2.1/

cp /tmp/sasdepot/depot/standalone_installs/SAS_High-

Performance_Node_Installation/2_1/Linux_for_x64/TKGrid_Linux_x86_64.sh .

Install SAS High-Performance Analytics components, by running the command:

./TKGrid_Linux_x86_64.sh

You will see the following output

Verifying archive integrity... All good.

Uncompressing

TKGrid.......................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

...................................................................................................................................................

.............................................................

running TKGrid/bin/tkgridconfig TKGrid /opt/TKGrid_2.1

TKGrid Configuration Utility.

Running on 'Nimitz1'

Using stdin for options.

Shared install or replicate to each node? (Y=SHARED/n=replicated

Respond to the prompts from the “TKGrid Configuration Utility”:

Parameter Description

Shared install or replicate to

each node?

(Y=SHARED/n=replicated)

n

Enter additional paths to

include in

LD_LIBRARY_PATH,

separated by colons (:)

Press ENTER. Do not add additional options.

Enter additional options to

mpirun.

Press ENTER. Do not add additional options.

Enter path to use for Utility

files. (default is /tmp).

SAS High-Performance Analytics applications might write scratch

files. By default, these files are created in the /tmp directory.

Press ENTER to accept the default.

Enter path to Hadoop. (default

is Hadoop not installed).

Enter /hadoop/hadoop-0.23.1 unless you installed Hadoop in a

different location.

Force Root Rank to run on

headnode? (y/N)

N

Enter full path to machine list... Enter the name of the file that you created earlier, /etc/gridhosts .

Enter maximum run time for

grid jobs (in seconds). Default

7200 (2 hours).

If a SAS High-Performance Analytics application executes for more

than the maximum allowable run time, it is automatically terminated.

31560000 means 1 year, or 604800 is 1 week.

Keep the default

Enter value for UMASK.

(default is unset.)

Type 022 for the umask value and press ENTER.

You will see the following prompt after answering the prompts above:

The install can now copy this directory to all the machines

listed in '/opt/TKGrid_2.1/TKGrid/grid.hosts' using scp, skipping the first entry. Perform

copy? (YES/no)

Answer YES

The following will be displayed as the installation is copied to each node:

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 mkdir -p -m

755 /opt/TKGrid_2.1

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 mkdir -m 755

/opt/TKGrid_2.1/TKGrid

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 ln -s

bin/tkmpinodelib.sh /opt/TKGrid_2.1/TKGrid/tkmpinodelib.sh

scp -o StrictHostKeyChecking=no -o PasswordAuthentication=no -q -r

/opt/TKGrid_2.1/TKGrid/* Stennis2:/opt/TKGrid_2.1/TKGrid/



ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 mkdir -p -m

755 /opt/TKGrid_2.1

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 mkdir -p -m

755 /opt/TKGrid_2.1

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 mkdir -m 755


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 mkdir -m 755


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Reagan4 ln -s


ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Vinsone3 ln -s


scp -o StrictHostKeyChecking=no -o PasswordAuthentication=no -q -r

/opt/TKGrid_2.1/TKGrid/* Reagan4:/opt/TKGrid_2.1/TKGrid/

ssh -q -o StrictHostKeyChecking=no -o PasswordAuthentication=no Stennis2 scp -o

StrictHostKeyChecking=no -o PasswordAuthentication=no -q -r /opt/TKGrid_2.1/TKGrid/*

Vinsone3:/opt/TKGrid_2.1/TKGrid/




755 /opt/TKGrid_2.1/TKGrid

Create a symbolic link /opt/TKGrid pointing to the new installation, first change to root

and then run the command:

su -

<password>

/opt/webmin/utilbin/simsh ln -s /opt/TKGrid_2.1/TKGrid /opt/TKGrid

5.4 Test the HPA Server Cluster Installation

Login to the LASR Head node, Nimitz1, as the sasinst user

Execute the Grid Monitor shell script

/opt/TKGrid/bin/gridmon.sh

As a result, you should have the following window showing up on the Windows machine

VI. SAS Visual Analytics Installation

The following steps show the installation steps required to install SAS Visual Analytics. The plan

file will be provided by SAS. All of the steps above must be completed prior to this installation

step. Most of the values required are default, so there will be minimal inputs required. Please be

guided by the screenshots below.

VII. Register LASRadm to Metadata

Execute management console and register LASRadm by following the steps below:

Create a New User by right-clicking User Manager and select create a new user

Type lasradm as the account name

From the Groups and Roles tab, select Visual Analytics Data and Visual Data Builder Administrators. Assign lasradm to that group.

Assign lasradm account as the DefaultAuth login. Click ok to finish.

VIII. Validation

You can perform validation by following the steps noted in the Intructions.html file, which is the

result of the SAS Visual Analytics Installation. Below are screenshots of some the steps:

Validation of the SAS Workspace, Pooled Workspace, and Stored Process Servers;

Acessing the VA Hub:

Loading to Hadoop:

Starting the SAS LASR Analytic Server:

Loading a Table to LASR:

IX. Start-up and Stopping SAS Services

To start SAS services on Nimitz1, you must follow the correct sequence as noted below:

1) Start Hadoop (Must be logged in as hadoop user)


2) Start the SAS services (all commands below must be executed as sasinst user)

/sas93/config/Lev1/sas.servers start

3) Start the Information Retrieval Studio

/sas93/config/Lev1/Applications/SASInformationRetrievalStudio*/IRStudio.sh start

4) Start the LASR Monitor Server

/sas93/config/Lev1/Applications/SASVisual*/H*/LASRMonitor.sh start

To stop SAS services on Nimitz1, you must follow the correct sequence as noted below:

1) Stop the LASR Monitor Server (You must use sasinst account for steps 1 – 3)

/sas93/config/Lev1/Applications/SASVisual*/H*/LASRMonitor.sh stop

2) Stop the Information Retrieval Studio

/sas93/config/Lev1/Applications/SASInformationRetrievalStudio*/IRStudio.sh stop

3) Stop the SAS services

/sas93/config/Lev1/sas.servers stop

4) Stop Hadoop Nodes(Must be logged in as hadoop user)

$HADOOP_HOME/sbin/stop-dfs.sh

sas visual analytics 7.3 installation and administration guide · sas visual analytics includes the...

Documents