ascr scientific data management analysis & visualization pi meeting exploration of exascale in...

ASCR Scientific Data Management Analysis & Visualization PI Meeting

Exploration of Exascale In Situ Visualization and Analysis Approaches LANL: James Ahrens, Jon Woodring, Joanne Wendelberger, Francesca Samsel

. We explore two in situ approaches at the extreme ends of a spectrum between flexibility and accuracy. We will strive to understand the advantages and disadvantages of both approaches and evaluate their effectiveness. Using the results of this evaluation, we will merge the best of both approaches to produce an optimize exascale

visualization and analysis approach.

Statistics and Sampling of Simulation Data with BitmapsChallenges

Locating the data that a scientist needs is daunting due to the scale of the data and lack of information

Solution: Sample BitmapsBitmap indices provide summary information for a large-scale data set

They also provide distributional data that can be used for samplingStatistics can be extracted from this summary to be able to drill down and extract

information of interestBitmaps accelerate statistics and sampling for faster turn-around in exploration with

lower sample errorPapers

Y. Su, G. Agrawal, J. Woodring, K. Myers, J. Wendelberger and J. Ahrens, "Effective and Efficient Data Sampling Using Bitmap Indices", Cluster Computing, March 2014.

Y. Su, G. Agrawal, J. Woodring, A. Biswas and H.-W. Shen, "Supporting Correlation Analysis on Scientific Datasets in Parallel and Distributed Settings", in Proceedings of the International ACM Symposium on High-

Performance Parallel and Distribued Computing (HPDC'14), June 2014, Vancouver, Canada.Y. Su, G. Agrawal, J. Woodring, K. Myers, J. Wendelberger and J. Ahrens. “Taming Massive Distributed Datasets: Data Sampling Using Bitmap Indices.” In Proceedings of the International ACM Symposium on

High-Performance Parallel and Distributed Computing (HPDC’13), New York, NY, USA, June 2013.Y. Su, G. Agrawal, and J. Woodring, “Indexing and Parallel Query Processing Support for Visualizing Climate

Datasets”, Proceedings of the 41st International Conference on Parallel Processing, Pittsburgh, PA, Sept. 2012.

Bitmaps are used for sampling and statistics for large-scale

data analysisContact: James Ahrens <ahrens@lanl.gov>

Adaptive refinement based on analysis metric

highlighting areas of interest

Reduced Simulation Data Approach Significantly reducing simulation data by storing

sampled and compressed data representations

Adaptive Sampling of Simulation DataChallenges

Simulations and experiments generate more data that can be feasibly stored by the scientist

Solution: Adaptive Sample Data based on Analysis Metrics Treat the exascale data deluge as a sampling problem

Use a variety of metrics to automatically select and triage the important data

Analysis Driven Refinement is a framework that prioritizes and samples using these metrics

Papers B. Nouanesengsy, J. Woodring, K. Myers, J. Patchett, and J. Ahrens, “ADR Visualization: A Generalized Framework for Ranking Large-Scale Scientific Data using Analysis-Driven Refinement”, LDAV 2014, November 2014, Paris, France.

K. Myers, E. Lawrence, M. Fugate, J. Woodring, J. Wendelberger, and J. Ahrens, “An In Situ Approach for Approximating Complex Computer Simulations and Identifying

Important Time Steps”, in submission, arXiv:1409.0909. A. Biswas, S. Dutta, H.-W. Shen, J. Woodring. “An Information-Aware Framework for

Exploring Multivariate Data Sets.” IEEE Visualization 2013, Atlanta, GA, November, 2013.

Image Database ApproachSignificantly reducing simulation data by storing rendered visualization and analysis images into an image database

Sampling in “Visualization and Analysis” SpaceChallenges

Simulations and experiments generate massive datasets that are difficult to store and analysis in a post processing manner

Solution: Generate In Situ Image DatabaseEnables many different interaction modes including: 1)

animation and selection, 2) camera and 3) timeCreates an responsive interactive visualization solution,

rivaling modern post-processing approaches, based on producing constant time retrieval and assembly

Encourages the use of both computationally intensive analysis and temporal exploration typically avoided in post-processing

approaches

Supports Metadata SearchingBy leveraging an image database, our approach allows the

analyst to execute meta-data queries or browse analysis results to produce a prioritized sequence of matching results

Creation of New Visualizations and Content Querying Supports composing of individually imaged operators

Provides access to the underlying data to enable advanced rendering during post-processing (e.g. new lookup tables,

lighting, ...) Makes it possible to perform queries that search on the

content of the image in the database. Using image-based visual queries, the analyst can ask simple scientific questions

and get the expected results

Papers J. Ahrens, S. Jourdain, P. O'Leary, J. Patchett, D. H. Rogers, M. Petersen, “An Image-

based Approach to Extreme Scale In Situ Visualization and Analysis”, Supercomputing 2014, New Orleans.

Interactive visualization and compositing using images from the image database

Using lighting and color mapping, render passes and compositing enable more capable visualization pipelines

such as changing color scale mapping for objects

Queries based on the image content can be used to search for qualitative results like “best view”

LA-UR-15-20106