- how does the coupled modeling system work? and - setting up a coupled application
DESCRIPTION
- How does the coupled modeling system work? and - Setting up a coupled application. C oupled Modeling System. Model Coupling Toolkit Mathematics and Computer Science Division Argonne National Laboratory http://www-unix.mcs.anl.gov/mct/. - PowerPoint PPT PresentationTRANSCRIPT
- How does the coupled modeling system work?
and
- Setting up a coupled application
Model Coupling ToolkitMathematics and Computer Science Division Argonne National Laboratoryhttp://www-unix.mcs.anl.gov/mct/
MCT is an open-source package that provides MPI based communications between all nodes of a distributed memory modeling component system.
Download and compile as libraries that are linked to.
Warner, J.C., Perlin, N., and Skyllingstad, E. (2008). Using the Model Coupling Toolkit to couple earth system models. Environmental Modeling and Software
Coupled Modeling System
Model A running on M nodes.
Model B running on N nodes.
Model C ………
(it also works here)
MCT providescommunications
between all models.………
Libraries
MCT - v2.60 or higher (distributed)
1) cd to the MCT dir
2) ./configureThis makes Makefile.conf. you can edit this file.
3) make
4) make install
5) set environment varssetenv MCT_INCDIR COAWST/Lib/MCT/includesetenv MCT_LIBDIR COAWST/Lib/MCT/lib(or where ever you installed them, see last slide)
Compilers dir (side note)
Model organization
master.F
mpi_init
initrun
finalize
initrun
finalize
{{
SWAN
ROMS
init_file (# procs/model)
init, run, and finalizeROMS SWAN
init(grid decomp)
run(sync. point)
finalize
init_paraminit_parallelinit_scaclarsinit_couplingro
ms_
init
rom
s_ru
n
main3d .....
waves_coupling...
mpi_finalizeclose_io
rom
s_fi
nali
ze
MPI_INIT
SW
INIT
MP
IS
WM
AIN
SW
EX
ITM
PI
SWINITSWREAD (grid)
init_coupling
swanmain .....
ocean_coupling...
mpi_finalizeclose_io
Grid decomposition (during initialization)
ROMS SWAN
- Each tile is on a separate processor.
- Each tile registerswith MCT.
init_coupling
ROMS- init_coupling SWAN- init_coupling
1
1
22
3
3
processed by each ROMS tileprocessed by each SWAN tile
Synchronization (run phase)
ROMS- ocean_coupling SWAN- waves_coupling
MCT
MCT
processed by each ROMS tile processed by each SWAN tile
ATM to OCN data fields
ATM
Uwind, V
wind, P
atm
, RH, T
air,
cloud
, rain
, eva
p, S
Wra
d, L
wrad
LH, H
FX, Ustr
ess,
Vstres
s
OCN
Integration and Application Network (ian.umces.edu/symbols),University of Maryland Center for Environmental Science.
Uwind, VwindSwrad, Lwrad,RH, Tair, cloud
Ustress, Vstress,Swrad, LwradLH, HFX
Use momentum + heat fluxes computed
in WRF for both ROMS+WRF
Use wrf vars in COARE algorithm
rain, evap
Salt flux
or#define BULK_FLUXES#define ATM2OCN_FLUXES
#define ATM_PRESS - Patm
#define EMINUSP
stflx_temp = Swrad+Lwrad +LH+HFX
stflx_salt = evap - rain
LH + HFX computed in bulk_fluxes
OCN to ATM data fields
OCN
ATM
WAV
SST
Hwave , L
pwave ,
Tpsurf ,
Sur
face
flu
xes
Momentum
Heat
Moisture
= f ( Hwave, Lpwave, Tpsurf )
SST OCN
WAV
How to create coupled application 1) Create all input, BC, init, forcing, etc files
for each model as if running separately. I recommend that you run each model separately first.
2) modify cppdefs in your header file. 3) SCRIP (if different grids or grid refinement) 4) coupling.in 5) coawst.bash 6) run it as coawstM
- Handout ends here- More in the online ppt
- Classroom tutorial will now follow:Projects/Sandy/create_sandy_application.m
1) Use each model separatelyWRF
27 vertical levels dt 36 s Physics
Lin microphysics RRTM longwave, Dudhia
shortwave Mellor-Yamada-Janjic (MYJ) PBL Kain-Fritsch (KF) cumulus scheme
6 km grid
ROMS 16 vertical levels dt 240, 48 Physics
GLS turbulence closure COARE bulk fluxes BC's from HYCOM Timestep = 240s
5km and 1 km grid(s)
These models are on different grids.
2) sandy.h
3) SCRIP - grid interpolation
Atmosphere model provides heat flux to cover entire ocean grid. SCRIP interpolations weights needed to remap data fields.
Ocean model provides higher resolution and coupled response of SST to atmosphere. But the ocean grid is limited in spatial coverage so atmosphere model must combine data from different sources, which can create a discontinuity in the forcing.
http://climate.lanl.gov/Software/SCRIP/
Flux conservativeremapping scheme
Ocean grid5 km
Atm Grid6 km
HFLX10 GFS data
SST
Libraries SCRIP - v 1.6 (distributed)Used when 2 or more models are not on the same grid.
1) cd to COAWST/Lib/SCRIP/source dir2) edit makefile3) make
3) SCRIP Need to create SCRIP weights using
COAWST\Tools\mfiles\mtools\ create_scrip_weights_master.m
3) SCRIP There will be created a weight file between each grid.So if you have 1 WRF grid, and 2 ROMS grids, it will produceatm1_to_ocn1_weights.ncatm1_to_ocn2_weights.ncocn1_to_atm1_weights.ncocn2_to_atm1_weights.nc
WRF grid 1
1000 m
5000 m
ROMS grid 1
ROMS grid 2
4) coupling.in (this is a ROMS+WRF app)
set # procs for each model
set coupling interval. Can be different for each direction.
input file names. only 1 for WRF, 1 for ROMS, multiple for SWAN
SCRIP weights are listed here
4) namelist.input
set # procs for atm model
need dt of 30 to divide evenly into coupling interval
of 1200 sec.
6) run it as coawstM
use total number of procs from coupling.in only 1 executable
Processor allocation
stdout reports processor allocation
This looks like from a different run,but you get the idea
Processor allocation
"Timing for …." = WRF
"1 179 52974 02:59:00 " = ROMS
Here is where the model coupling synchronization occurs.
so probably could re-allocate more nodes to WRF
JOE_TC - test case examples
JOE_TC test cases are distributed applications for testing ROMS+WRF coupling JOE_TCw = wrf only JOE_TCs = same grid, roms + wrf coupled JOE_TCd = different grids for roms and wrf,
needs scrip weights