How To Use TheVampir Toolchain At IU

Robert Henschelhenschel@indiana.edu

April 2010

Robert Henschel

Contents• Quarry• BigRed

Robert Henschel

General• Trace creation and visualization does not have to

happen on the same platform!– You can look at traces created on Quarry (or your

own desktop) on BigRed

Robert Henschel

Quarry – SoftEnv Keys• Vampir Version 5.2

+vampir-5.2-32 or +vampir-5.2-64

• VampirServer Version 2+vampirserver-2-openmpi-64

• VampirTrace Version 5.8@intel-11.1.059-64+openmpi-1.4.1-intel-11.1.059-64+vampirtrace-5.8.0-intel-openmpi-64or @intel-11.1.059-32+openmpi-1.4.1-intel-11.1.059-32+vampirtrace-5.8.0-intel-openmpi-32

Robert Henschel

Quarry – Running Vampir• Vampir Version 5.2

– Interactive job in PBS– On interactive nodes b005-b008

Robert Henschel

Quarry – Running Vampir• Vampir Version 5.2

– Interactive job in PBSqsub -I -Xsoft add +vampir-5.2-64vampir

– On interactive nodes b005-b008• You share the node with other people, this is

especially a problem for opening large trace files!ssh -X b005soft add +vampir-5.2-64vampir

Robert Henschel

Quarry – Running VampirServer• Vampir Server Version 2

– Server in PBS, viewer on interactive node– Server and viewer in interactive job in PBS– Server and viewer on interactive nodes b005-b008

Robert Henschel

Quarry – Running VampirServer• Vampir Server Version 2

– Server in PBS, viewer on interactive node–Make sure your “.soft” file contains@intel-11.1.059-64+openmpi-1.4.1-intel-11.1.059-64+vampirserver-2-openmpi-64

–Logout/Login to active changes in “.soft”–PBS script

Start jobqsub vampir.pbs

–Wait for the job to start

#PBS -l nodes=2:ppn=8,walltime=5:00:00#PBS -m ae#PBS -N VampirServer#PBSvampirserver --mode=mpi-mode --nproc=16 1>$HOME/vampir.out 2>&1

Robert Henschel

Quarry – Running VampirServer• Lookup host and port information

» It may take a minute or two after the job started for this file to get updated!!!

cat $HOME/vampir.outRunning...Server listens on: b052:30000

• Viewer on interactive nodes b005-b008ssh -X b005soft add +openmpi-1.4.1-intel-11.1.059-64soft add +vampirserver-2-openmpi-64vampir //b052:30000/path/to/*.otf

Robert Henschel

Quarry – Running VampirServer• Interactive job in PBS• Only one nodeqsub -I -Xsoft add +openmpi-1.4.1-intel-11.1.059-64soft add +vampirserver-2-openmpi-64vampirserver -m mpi-mode -n 8 1>output 2>&1 &cat output

Running...Server listens on: b048:30000

vampir //b048:30000/path/to/*.otf

Robert Henschel

Quarry – Running VampirServerOn interactive nodes b005-b008

• Only one node• Runtime needs to be less than 24 hours• You share the node with other peoplessh -X b005soft add +openmpi-1.4.1-intel-11.1.059-64soft add +vampirserver-2-openmpi-64vampirserver -m mpi-mode -n 4 1>output 2>&1 &cat output

Running...Server listens on: b005:30000

vampir //b005:30000/path/to/*.otf

Robert Henschel

Quarry – Tracing with VampirTrace 5.8• VampirTrace is configured to work with the Intel

compilers version 11.1 and OpenMPI 1.4.1

• Include the following keys in your .soft file@intel-11.1.059-64+openmpi-1.4.1-intel-11.1.059-64+vampirtrace-5.8.0-intel-openmpi-64

• The VampirTrace compiler wrappers are now available• It is possible to use them also with GCC, if you need to

do that, please send a mail to hpahelp@indiana.edu• We can also provide a version for the PGI compilers.

Robert Henschel

BigRed – SoftEnv Keys• Vampir Version 5.1

+vampir-5.1-64

• VampirServer Version 1.10+vampirserver-1.10-64

• VampirTrace Version 5.6.3+openmpi-1.3.1-ibm-64+vampirtrace-5.6.3-ibm-64or +openmpi-1.3.1-ibm-32+vampirtrace-5.6.3-ibm-32