parallel volume rendering on the ibm blue gene/ptpeterka/talks/peterka-egpgv08... · 2010-10-05 ·...

Tom Peterka

tpeterka@mcs.anl.gov

Parallel Volume Rendering on

the IBM Blue Gene/P

www.ultravis.org

Radix Laboratory for Scalable Parallel System Software

Mathematics and Computer Science Division

Argonne National Laboratory

2Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.org

Rob Ross, ANL

Intro: Wanted - 50 Reward

Hongfeng Yu, UCD Tom Peterka, ANL

Kwan-Liu Ma, UCD

3Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.org

Bigger graphics clusters?

• Cost of scalability

• Power, cooling, space

• Not a rendering bottleneck

Intro: Supercomputer software rendering

- Déjà vu?

Availability

• Short runs available ad hoc

• Longer runs scheduled

• Dedicated resources

Cost effectiveness

• Short runs easy to schedule

• Backfill available cycles

• Policy decisions

4Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgBackground: Big iron

BUT

NO GRAPHICS!

5Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgBackground: Data

• Supernovae shock wave

• Structured grid

• 8643

• 5 variables in netCDF

• Preprocess to extract single variable

• Time-varying, 200 time steps

• Each time step 2.5 GB

• Courtesy John Blondin, NC State,

and Tony Mezzacappa, ORNL

6Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgBackground: Parallel volume rendering

tframe = tio + trender + tcomposite

• MPI programming model

• Distributed memory

7Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgImplementation: Data distribution & I/O

MPI_File_open(…)

MPI_Type_create_subarrray(…)

MPI_Type_commit(…)

MPI_File_set_view(…)

MPI_File_read_all(…)

File view:

bytes

Dataset view:

logical blocks

Process view:

processes…

P0 P1 Pn

8Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgImplementation: Ray casting

• Trilinear interpolation

• Front-to-back color, opacity

accumulation

• Early ray termination

• Static data partition

• Each process completes

subimage of its subvolume data

i = ( 1.0 – old) * inew + iold

= ( 1.0 – old) * new + old

where i = intensity, = opacity

P0 data P1 data

P3 dataP2 data

P0 image P1 image

P2 image P3 image

9Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgImplementation: Direct-send compositing

• Load balanced

• Non-scheduled

• O (n4/3 + n) where

n = number of

cores

• O(m) where m =

number of pixels

10Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgResults: Strong scaling

• End-to-end results, including file I/O

• Vis-only time .8 s

11Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgResults: Absolute time distribution

12Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgResults: Relative time distribution

13Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgResults: Efficiency

14Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgResults: Improved efficiency

15Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgConclusions

Successes

• Visualization on leadership machines at ultrascale

• End-to-end performance is a delicate balance

• I/O matters (E2E time, not just rendering time)

• Combination of systems and visualization solutions

Current work

• Load balance

• Hiding I/O cost

• Improved image quality

• Larger data

Future work

• Less structured data

• Interaction

• Novel display environments

• Exploit multi-cores

• Other architectures

16Argonne National

LaboratoryEurographics Parallel Graphics and Visualization Symposium ‘08 April 15, 2008 Tom Peterka

www.ultravis.orgQuestions and challenges

Technical

• Compositing

• Interactivity

• In situ visualization:

implementation

Nontechnical

• Machine availability

• Machine utilization

• In situ visualization:

collaboration

Tom Peterka

tpeterka@mcs.anl.gov

Parallel Volume Rendering on

the IBM Blue Gene/P

www.ultravis.org

Radix Laboratory for Scalable Parallel System Software

Mathematics and Computer Science Division

Argonne National Laboratory