Software OverviewEnvironment, libraries, debuggers, programming tools and applications

Jonathan CarterNUG Training3 Oct 2005

2

Overview

• Environment• Libraries• Debuggers • Programming Tools• Applications

3

Environment

• Suse Linux• Linux Networx HPC software stack• Pathscale compilers• PBS Pro Batch System• GPFS HOME and SCRATCH file

systems

4

Environment (cont)

• Passwords– NIM password, use NIM to change

• Shells– Default shell is tcsh, to change use NIM

interface• Startup files

– Read-only, use <name>.ext if you want to add customizations

– fixdots command will repair files

5

Environment (cont)

• Access to libraries, tools and applications is controlled via modules

• Number of modules reduced compared with seaborg, since we only have 64-bit ABI and fewer software versions

6

Libraries

• Math– ACML, fftw, NAG, parpack, scalapack,

superlu, (imsl, petsc)• I/O

– hdf, hdf5, netcdf– MPI I/O available via MVAPICH

• Visualization– ncar

7

Libraries (cont)

• ACML– Optimized for AMD processors– BLAS 1-3, FFT and LAPACK– “fast” math functions (log is ~90 cycles

instead of ~120)– “vector” math functions, e.g. vrda_log(n, x, y) is roughly twice as fast as log at n>20.

8

Libraries (cont)

• Fortran I/O– Intel/AMD hardware is little-endian,

Power is big-endian, so binary data is incompatible

– pathf90 offers several features to write binary data in big-endian format

9

Libraries (cont)

• pathf90 options– -byteswapio writes all data in format opposite

to that of native processor– -conversion [native, little_endian, big_endian]– assign commandFILENV=.assignexport FILENVassign –N mips u:10

10

Libraries (cont)

• Many libraries are linked at runtime, so LD_LIBRARY_PATH must include a path to each library linked against

• Modules environment takes care of this provided you load at compile time and at run time

11

Debuggers

• Totalview and gdb available for serial applications

module load totalviewtotalview progname [corefile]

gdb progname [corefile]

12

Debuggers (cont)

• Totalview and gdb are coming for parallel applications

• gdb opens xterm per processor

module load totalviewmpirun –tv –np procs progname

mpirun –debug –np procs progname

13

Debuggers (cont)

• Generating core files– Serial applicationsulimit -S -c unlimited– Parallel applications, make shell script

progname.sh and run this via mpirun (this problem will be resolved by PBS bugfix)

#!/bin/bashulimit -S -c unlimitedexec progname

14

Debuggers (cont)

• Useful Pathscale compiler options– Use –g flag to help debugging– Use –trapuv to set uninitialized (local,

automatic, alloca(); not Fortran allocatable) variables to NAN

15

Debuggers (cont)

• Further information on Totalview– Etnus tutorial

http://www.etnus.com/TotalView/started/getting_started.html

– LLNL tutorial – http://www.llnl.gov/computing/tutorials/totalview/

16

Programming Tools

• AMD CodeAnalyst – suite of tools to optimize code for Opteron: coming soon

• ipm – mpi overhead and performance report

• papi – interface to hardware performance counters

• tau – suite of tools enabling performance analysis of application codes

• valgrind – multiple tools to check memory use, profile cache use, detect data race conditions in threaded applications

17

Applications

• Chemistry & Materials Science– GAMESS, Gaussian, Gromacs,

MOLPRO, NWChem, NAMD, VASP• Math

– maple, matlab, mathematica • Visualization

– idl, vmd• Other

– Subversion (client)