National Center for Supercomputing Applications University of Illinois at Urbana–Champaign

Bridging the Gap Between High Quality and High Performance for HPC Visualization Rob Sisneros

Outline

•  Why am I here? •  NVIDIA/NCSA collaboration •  Current efforts address need in HPC visualization

•  This talk •  Some context •  Some opinions on HPC Visualization

•  A gap in the state of the art •  Why it is critical to address this gap

•  Introduction to IndeX + ParaView

CONTEXT

Visualization:

Analysis (e.g. statistics)

Graphics (e.g. plots)

Data

Understanding

Pretty Pictures

Operation

Operation

Operation

. . .

. . .

NCSA Data Analysis and Visualization Team

•  Recently formalized •  3 Members and 2 students (and we’re hiring!) •  Various visualization expertise

•  Large scale visualization, algorithms, and I/O •  Data analysis •  Visual analytics •  Infovis •  Production visualization

•  Our mission •  Support large scale science by making visualization available to

teams utilizing NSF computing resources (primarily Blue Waters) •  Further the science of visualization through cutting edge

research

Blue Waters (AKA The World’s Best Cray)

Sonexion: 26 usable PB

>1 TB/sec

100 GB/sec

10/40/100 Gb Ethernet Switch

Spectra Logic: 300 usable PB

120+ Gb/sec

100-300 Gbps WAN

IB Switch External Servers

Total Cabinets: 288 Compute Nodes (XE + XK): 26,864 Aggregate Memory: 1.5 PB

SCUBA Subsystem - Storage Configuration for User Best Access

Talking Points

•  Blue Waters •  Compelling use cases of GPUs for visualization •  Data staging on XK nodes

•  In situ analysis/visualization •  Full nodes or even just GPUs

•  Drivers and methods for remote visualization •  And also…

•  We’re hiring

^ to at least talk to

Visualization Software for HPC Centers

•  Scalable •  General •  Widely used

•  Accessibility •  Utility

•  Ongoing development •  Active community •  Open source

Scalable Scaled > 100K cores

Offer interactive client/server mode Can operate in batch mode In situ support Rich set of data operators Native support for many file formats

VisIt Paraview

HPC VISUALIZATION RESEARCH

Simulation Data Generation

Data Structures I/O Disk

In Situ Frameworks

Data Structures I/O Disk

In Situ Frameworks

•  This area •  Where the data we visualize and analyze lives •  The data that drives science and therefore support

•  How does support affect our research?

Data Structures I/O Disk

In Situ Frameworks

How Visualization Fits in HPC Research

Visualization research w/

HPC application

HPC research w/

Visualization application

How Visualization Research Fits in HPC Research

•  Theoretical and applied •  Value often judged via

application to specific science domain

•  Novelty is often at odds with adoption

•  Not investment priority at HPC centers

•  Applied •  Value often judged via

•  Performance •  Performance •  Performance

•  Makes bosses happy

Visualization research w/

HPC application

HPC research w/

Visualization application

WHAT’S MISSING?

Mind the Gap

•  Should we (at least sometimes) offer capability in lieu of performance?

•  Slow codes on a supercomputer simply aren’t publishable •  The cutting edge of visualization research is providing data

models and algorithmic developments to address ~95% of typical data analysis/visualization tasks for expected future hardware

•  Those I know using GPUs for visualization are in the 5% •  Real Q+A session:

•  Q: “How can I use GPUs for vis?” •  A: “Why would you want to do that? Are you sure you want to?”

(Yay!)

^ Widening

The Problem

•  Sometimes •  I have huge data, and •  I want an insanely high quality rendering

•  Or… •  I have huge data, and •  I want interactivity

The Good News

Future HPC Visualization will finally begin to focus on interactivity Reason 1: The 5% will bring it with them Reason 2: The data is too big not to use interactivity

CONVERGING TO INTERACTIVITY HPC Data

Data Reductions for Visualization

•  Typical large-scale visualization •  Begins with data •  Which goes through an analysis pipeline •  Ends with (interactive) graphics

•  The pipeline •  Calculations, derived variables •  Direct data reductions: thresholds, isosurfaces •  Finding the region of interest

•  Graphics •  Interactions: rotations, panning, zooming •  Indirect data reductions •  Setting up the camera

The Problem with Current Reduction Split

•  Direct data reductions are all we care about •  Finding a region of interest is the “hard” problem, the “big

data” problem •  The assumption: the region of interest is trivially viewable •  Not true anymore even for “small-scale” features

The New Research Question

•  How do we view at resolution?

•  Direct Solutions

•  Throw on big, expensive display

•  Resolve yourself to look at several resolution chunks

Resolution Chunks

•  VR Headset! •  Rendering in 360 degrees

•  Looking around determines which chunk to look at (and all indirect reduction is removed from rendering)

The Point + Some Bad News

•  Interactivity may make a comeback, but •  Was never distanced from visualization, just not expected •  For HPC visualization 1 frame per second is hopeful (10+ FPS is

happy dance time)

•  Top quality rendering for HPC data doesn’t have as natural an alignment with future HPC visualization

•  Is nonetheless critical

ON VALUE OF HIGH QUALITY DAV Team’s SC Showcase of Tornado Data

Non- Photorealistic

vs.

Photorealistic Renderings

Unsurprisingly

•  For data with naturally physical representations, high quality and/or photorealistic rendering is preferred

•  This represents a large portion of generated simulation data for which we could be doing better, and

•  Is completely underrepresented in the literature

NVIDIA INDEX + PARAVIEW Moving in the Right Direction

NVIDIA IndeX: In Situ & At Scale Rendering

•  Leverages GPU-clusters for scalable large-scale data visualization

•  Is a GPU-cluster aware solution for interactive visual computing

•  Is a commercial software solution available and already deployed by customers for In-Situ visualization for large-scale data

http://www.nvidia-arc.com/products/nvidia-index.html

Features

•  Supports 8-bit, 16-bit, float & RGBA data types •  Supports unstructured tetrahedral cells •  Depth correct transparent geometry such as heightfields

and triangle meshes and volumetric data •  In Situ operation •  Asynchronous streaming of time varying data •  Over 1 TB of data rendered at 20+ fps

Integration with ParaView

•  Domain decomposition is done by ParaView

•  Affinity information is supplied to NVIDIA IndeX

•  NVIDIA IndeX adapts to ParaView’s domain decomposition

[..] [..]

Cluster

Implementation

•  Interface with vtkVolumeMapper & vtkImageVolumeRepresentation

•  MPI infrastructure is implemented in the plugin

ParaView

•  Varietyofreaders,maturecomputa5onalpipeline

NVIDIAIndeX

•  Scalable,highqualityrendering

ParaViewPlugin

AND NOW FOR SOMETHING COMPLETELY RELATED