SAEED Al-ZAHRANI

HPC at Saudi Aramco’s EXPEC Computer Center

Agenda

Introduction to Saudi Aramco and ECC

Why do we use HPC?

ECC HPC Platforms

ECC HPC Strategy

ECC HPC Challenges

Saudi Aramco

Manages proven crude oil reserves of

260.1 Billion Barrels

4th largest gas reserves of

279 Trillion Cubic ft

Worldwide refining

capacity of

over 4 million BPD

Owns the world’s 2nd largest fleet of tankers

Fully integrated global petroleum enterprise

– Exploration,

– Producing,

– Refining,

– Petrochemicals,

– Distribution,

– International Shipping , and

– Marketing

http://www.saudiaramco.com

ECC Vision World-class

High Performance Computing

Data Center in Oil & Gas

ECC Mission • Support Upstream

• Scientific Computing

• Cost Effective

• Certified Professionals

• World-class Practices

EXPEC Computer Center (ECC)

EXPEC Computer Center

ECC Supporting Services

Why do we use HPC?

Main Functions – Seismic Processing

– Reservoir Simulation

Seismic Processing

• Conventional Processing

• Pre-Stack Time Migration (PSTM)

• Pre-Stack Depth Migration (PSDM)

Seismic Processing

Reservoir Simulation

An area of reservoir engineering in which computer models are used to describe fluids (oil/water/gas) and its flow dynamics in the subsurface geologic formations.

Gas

Oil

Water

Reservoir Simulation

• Understand fluid flow behavior

• Develop depletion strategies & forecast future

• Software

POWERS (Parallel Oil WatER Simulator)

GigaPOWERS • Can simulate billions of cells

• Giga models for super-giant fields

Low Resolution modeling

High Resolution modeling

Technology Evolution

1

100

10,000

1,000,000

19951996

19971998

19992000

20012002

20032004

20052006

20072008

2009

2010

2011

IBM Main Frame, CRAY IBM SP PC CLUSTERS

ECC HPC platforms

Driving Force: Do more with less!

Cray T90

IBM SP

PC Cluster (2002)

PC Cluster (2011)

12 Processors

512 Processors

1900 Processors

65,000 Processors

Impact of affordable HPC

• Dramatic improvements in price/performance bring out more computationally intensive applications that were previously thought to be cost prohibitive

• IBM SP: Enabled PSTM, POWERS

• Clusters: Full PSTM, PSDM, GigaPOWERS

• Heterogeneous (CPU + Accelerator/GPU)

– for RTM, full wave equation

Current HPC Environment

• 500 Teraflops total compute capacity

– > 6800 nodes, 65,000 cores, 97 TB RAM

• 43 HPC Clusters

– Different sizes & configurations

– 2 systems in the latest Top500 list

• Several Petabytes of high performance disk

HPC System Design

Clustered File System

(CSSP) I/O

Nodes

Disk Storage

Cluster A

Cluster B

Cluster Z

Co

mp

ute

N

od

es

MPI

CSSP (Conventional Serial Seismic Processing)

• Small number of nodes connected to High Performance Storage pool

• Clustered File System

PSTM (Pre-Stack Time Migration)

• Embarrassingly Parallel

• Master node on CSSP cluster

• Data fed via MPI to Grid of compute nodes

• GigE interconnect sufficient

HPC System Design

POWERS/GigaPOWERS

• Memory bandwidth is key

• Cannot use all cores in CPU due to this

• Needs low latency, high bandwidth interconnect

• Quadrics Myrinet Infiniband

• Typical clusters are 512 nodes

• 2 CPUs (12 cores) per node

HPC Strategy

• Meet current and future growth Scalable

• Expand and provide computing capacity as needed, avoiding unnecessary pre-investment. Modular

• Exploit the utilization of inexpensive computing resources that work together, to address complex problems that would otherwise require very expensive systems to solve.

Parallel

Key Factors

Meet Capability requirements

Reliability

Maturity of Technology

Ease of Management

Price / Performance

Sustained improvements in Roadmap

Trends

• Higher Resolution Seismic surveys

• Intelligent Fields (i-field) Huge increase in data

• Interactive visualization Giga-size models for higher resolution

• Minimize the cycle time Optimize decisions

• Increased complexity computing power Better Algorithms

HPC Challenges

•GPUs, Intel MIC (Many Integrated Core)

•Coding & Optimization is non-trivial

Heterogeneous Computing

•Memory bandwidth

•Interconnect bandwidth and latency I/O bottlenecks

•Data Management

•Data Backup/Archival Big Data

•Security

•Solutions Maturity

•Large Data transfer HPC in the Cloud

•Watching and Learning Design Challenges for Exascale

Summary

• Saudi Aramco leverages innovative technologies to deliver the best price/performance in HPC

• Constant efficiency Improvements in hardware, software and workflow

• Continual evaluation of alternate technologies