Cactus Visualisation

greentexture

Outline

I. Cactus Data I/O Infrastructure● Cactus I/O Methods and Thorns● How to obtain periodic output● Parallel I/O● The HDF5 file format● Hyperslabbing

II. Cactus Data Visualization● Simple 1D/2D visualization tools● Advanced 3D visualization packages

greentexture

The Cactus I/O Subsystem

● I/O Thorns● Provide I/O methods which they register with the flesh at STARTUP● Include functions for the flesh to call for I/O● Decide themselves how, when, and where to output● If applicable they also provide checkpointing / recovery functionality

● Most users perform (periodic) output simply by setting appropriate I/O parameters in a parameter file for a run

● see wavetoy example parameter file

● (Unconditional) output can also be triggered directly from application thorns via the flesh I/O API (useful for non-periodic output or debugging)

greentexture

Cactus I/O Infrastructure

IOUtil

Registration & Scheduling       of I/O methods

Flesh

Set of basic I/O parametersRegistration & Scheduling   of Recovery Methods

IOBasic

IOFlexIO IOHDF5

IOJpeg

IOASCII

greentexture

I/O Methods in Cactus

I/O MethodInfo

Scalar

IOASCII_1D

IOASCII_2D

IOASCII_3D

IOHDF5

IOFlexIO

IOJpeg

IsoSurfacer

CarpetIOASCII

CarpetIOHDF5

DescriptionScreen output of scalars and

grid array reductions

Scalars and grid array reductions

1D line output of grid arrays

2D slice output of grid arrays

3D grid arrays

grid arrays and hyperslabs in HDF5 format

grid arrays and hyperslabs in FlexIO format

2D slice output as Jpeg images

Isosurface geometries for 3D grid arrays

Carpet ASCII output

Carpet HDF5 output

Providing ThornCactusBase/IOBasic

CactusBase/IOBasic

CactusBase/IOASCII

CactusBase/IOASCII

CactusBase/IOASCII

CactusPUGHIO/IOHDF5

CactusPUGHIO/IOFlexIO

CactusIO/IOJpeg

CactusPUGHIO/IsoSurfacer

Carpet/CarpetIOASCII

Carpet/IOHDF5

greentexture

HPC Issues for I/O

● human-readable ASCII data is nice but● it eats a lot of disk space● it involves conversions into/from strings which potentially

loose floating-point precision information a compact data representation is necessary

● datafiles should be interchangeable in a heterogenous computing environment

● data format should be standardized and supported by existing postprocessing and visualization tools

● large-scale parallel applications may generate data in the terabytes range which cannot be efficiently managed by traditional serial I/O schemes (I/O bottleneck) design your applications with parallel I/O in mind !

greentexture

HDF5Hierachical Data Format - Version 5: http://hdf.ncsa.uiuc.edu/hdf5

● provides a general Scientific Data Format specification and a supporting I/O library implementation

● simple but powerful data model: datasets + groups● self-describing data format: implicit / user metadata● predefined standard + user-defined datatypes● transparent datatype conversion during I/O● efficient partial reads/writes of datasets (hyperslab selections)● modular, extensible library design: Virtual File Driver layer

to plug in different low-level drivers

● Support and active development by NCSA's HDF5 group● More widely accepted by the HPC community as a standard

greentexture

Hyperslabbing● hyperslab selection

● identify only portionsof a 3D dataset for output

● hyperslab parameters● dimensionality● origin● extent● downsampling● (output precision)

Hyperslab write

greentexture

Cactus and HDF5● Cactus uses HDF5 as the standard data format for

(full) output of nD grid variables and hyperslabs thereof● through its Virtual File Driver layer, the HDF5 functionality

was enhanced for remote visualization and steering● online data streaming using the Stream Driver● efficient remote data access using the GridFtp Driver

● HDF5 utilities/readers are available for● Amira● DataVault● OpenDX

greentexture

Serial vs. Parallel I/O

P3 P2

P1 P0

One proc gathers datafrom all other processors ...

Serial I/O

... and writes it intoa single unchunked file

Serial I/O via dedicated I/O processor

I/O processor becomes bottleneck

greentexture

Serial vs. Parallel I/O

P3 P2

P1 P0

All processors write concurrently ...

Parallel I/O

... into multiple chunked files (possibly distributed)

Single unchunked file

Recombination

Parallel chunked I/O

● system limit for number of open file● chunked files might need recombination

greentexture

Cactus Visualization Tools

http://www.cactuscode.org/VizTools

OpenDX

IsoView

gnuplotxgraph

AmiraDataVault

xvs

greentexture

Simple Visualization: [xy]graph

● freely available, lightweight visualization tool

● generates 1D lineplots from ASCII output data

● mouse-controlled zooming● animations of time-dependent data● x-derivatives, re-scaling, time

integration of input data

● more info available on the ygraph homepage: http://www.aei.mpg.de/~pollney/ygraph/

greentexture

Simple Visualization: xvs

● freely available visualization package written by M. Choptuik

● based on Visualization Toolkit (VTK)● OpenGL functionality provides fast

and intuitive, mouse-based rotation and zooming of rendered images

● SDF datasets are streamed from a client to a server

● more info available on the xvs homepage: http://laplace.physics.ubc.ca/~matt/410/Doc/xvs/

● HDF5­to­SDF utility client available from   http://www.cactuscode.org/VizTools/DataVaultXVSutils.html

greentexture

Simple Visualization: gnuplot

● open source software● command-driven interactive

function plotting program● 2D and 3D surface plots● various terminal drivers

for different output devicesand formats

● variety of data manipulationand postprocessing features

● more info available on the xvs homepage: http://www.gnuplot.info/

greentexture

DataVault (DV)

● freely available visualization package written by M. Choptuik

● light-weight visualization packagebased on Visualization Toolkit (VTK)

● OpenGL functionality provides fast and intuitive, mouse-based rotation and zooming of rendered images

● hdf5todv client streams datasets in SDF format to a DV server

● more info available on the DV homepage: http://laplace.physics.ubc.ca/Doc/DV/

greentexture

Amira

● Commercial Visualization package developed at ZIB

● Advanced high-end 3D visualization andvolume modeling methods

● OpenGL hardware graphicssupport

● available for Windows, IRIX, HP-UX, SunOS, Linux

● more info on Amira home page http://amira.zib.dedemo version can be downloaded from http://amira.zib.de/download.html

greentexture

OpenDX

● powerful open-source software package for the visualization of arbitrary scientific datasets

● flexible data model● data-driven program flow● variety of already built-in

functional modules ● OpenDXutils package provides

reader modules to import HDF5 (Cactus and Carpet) data

● OpenDX home page: http://www.opendx.org/● OpenDXUtils: http://www.cactuscode.org/VizTools/OpenDX.html