operational cosmo demonstrator opcode

Federal Department of Home Affairs FDHAFederal Office of Meteorology and Climatology MeteoSwiss

Operational COSMO Demonstrator OPCODE

André Walser and Oliver Fuhrer

MeteoSwiss

COSMO-GM, Rome, 5-9 September 2011

2 OPCODE | COSMO-GM 2011André Walser ([email protected])

Project overview

• Additional proposal to the Swiss HP2C initiative to build an “OPerational COSMO DEmonstrator (OPCODE)”

• Project proposal accepted by end of May• Start of project 1 June 2011 until end of 2012• Project resources:

• second contract with IT company SCS to continue collaboration until end of 2012

• 2 new positions at MeteoSwiss for about 1 year• Swiss HPC center CSCS• C2SM (collaboration with ETH Zurich and others)


Main goals• Leverage the research results of the ongoing HP2C COSMO

project• Prototyp implementation of the COSMO production suite of

MeteoSwiss making aggressive use of GPU technology• MeteoSwiss ready to buy a GPU based hardware for the 2015

production machine• Same time-to-solution on substantially cheaper hardware:

Cray XT4 (3 cabinets)

GPU based hardware(a few rack units)


GPU perspectives

GFLOPS per Watt is expected to increase strongly in the next years


Workflow on demonstrator


COSMO-7 / COSMO-2 suite:

0 34

3h

ass

imila

tion

(2

1 U

TC

)

0-2

4h

fo

reca

st (

00

UT

C)

an

d T

C p

rod

uct

s

Elapsed timein min

3h

ass

imila

tion

(2

1 U

TC

)

0-2

4h

fo

reca

st (

00

UT

C)

an

d T

C p

rod

uct

s

25

-72

h f

ore

cast

(0

0 U

TC

)a

nd

TC

pro

du

cts

1 7 11 46 49

COSMO-2 forecast

COSMO-7 assimilation

COSMO-7 forecast

COSMO-2 assimilation

COSMO-2 TC products

COSMO-7 TC products

61

Current production scheme

• Time-critical post-processing takes about 15 minutes longer than forecasts for both COSMO-2 and COSMO-7

• current bottleneck is post-processing tool fieldextraentire suite has to be optimized for demonstrator


Two workpages

• Workpage A: Porting remainig parts of opr COSMO code @ MeteoSwiss to demonstrator

• Workpage B: Porting suite to demonstrator, optimize it, and operate it


To use full speed-up, data has to remain on GPU within a time step; sent to CPU for I/O only

Work package A

COSMO workflow:

Input

Physics

Dynamics

Assimilation

Boundary Conditions

Diagnostics

Output

What’s still missing for a full GPU implementation?


To use full speed-up, data has to remain on GPU within a time step; sent to CPU for I/O only

Work package A

COSMO workflow:

Input

Physics HPC2

Dynamics HPC2

Assimilation

Boundary Conditions

Diagnostics

Output

What’s still missing for a full GPU implementation?


Tasks Work Package A

Task

A1. Dynamical Core: complete/update HP2C code SCS


Task A2: Inter-/intra-GPU parallelization

• COSMO requires a communication library with halo-update as well as several other communications (e.g. global reduce, gather, scatter)

• e.g. peer-to-peer:


Work Package A

Task


A2. Inter-GPU parallelization: library for halo-updates, global reductions, scatters, gathers CSCS


Work Package A

Task



A3. Interoperability C++/CUDA/Fortran: common compile system, Unified Virtual Addressing SCS


A4. Data Assimilation: Porting to GPU

Assimilation part is a huge code!


Work Package A

Task




A4. Data Assimilation: Porting to GPU MeteoSwiss


Work Package A

Task





A5. I/O: Software layer controling copying of fields from CPU to GPU and vice versa for I/O C2SM?


Work Package A

Task





A5. I/O: Software layer controling copying of fields from CPU to GPU and vice versa for I/O C2SM?

A6. Porting other code parts (BC, diagnostics) to GPU SCS


Work Package B

Task

B1. Hardware CSCS

B2. System Software CSCS


Work Package B

Task

B1. Hardware CSCS


B3. COSMO-Package: Porting and optimization of steering scripts MeteoSwiss


Work Package B

Task

B1. Hardware CSCS



B4. Post-processing: Parallelization of post-processing tools, additional work in fieldextra (partly paid by “COSMO license money”) MeteoSwiss


Work Package B

Task

B1. Hardware CSCS



B4. Post-processing: Parallelization of post-processing tools, additional work in fieldextra (partly paid by “COSMO license money”) MeteoSwiss

B5. Setup and Testing MeteoSwiss


Organization

1.7 FTESCS, CSCS, C2SM

0.9 FTEnew position @MeteoSwiss1 yearstill open

1.9 FTE new collaborator @MeteoSwiss 15 months, CSCS


Schedule


Thank you !

operational cosmo demonstrator opcode

Documents