- how does the coupled modeling system work? and - setting up a coupled application
TRANSCRIPT
![Page 1: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/1.jpg)
- How does the coupled modeling system work?
and
- Setting up a coupled application
![Page 2: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/2.jpg)
Model Coupling ToolkitMathematics and Computer Science Division Argonne National Laboratoryhttp://www-unix.mcs.anl.gov/mct/
MCT is an open-source package that provides MPI based communications between all nodes of a distributed memory modeling component system.
Download and compile as libraries that are linked to.
Warner, J.C., Perlin, N., and Skyllingstad, E. (2008). Using the Model Coupling Toolkit to couple earth system models. Environmental Modeling and Software
Coupled Modeling System
Model A running on M nodes.
Model B running on N nodes.
Model C ………
(it also works here)
MCT providescommunications
between all models.………
![Page 3: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/3.jpg)
Libraries
MCT - v2.60 or higher (distributed)
1) cd to the MCT dir
2) ./configureThis makes Makefile.conf. you can edit this file.
3) make
4) make install
5) set environment varssetenv MCT_INCDIR COAWST/Lib/MCT/includesetenv MCT_LIBDIR COAWST/Lib/MCT/lib(or where ever you installed them, see last slide)
![Page 4: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/4.jpg)
Compilers dir (side note)
![Page 5: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/5.jpg)
Model organization
master.F
mpi_init
initrun
finalize
initrun
finalize
{{
SWAN
ROMS
init_file (# procs/model)
![Page 6: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/6.jpg)
init, run, and finalizeROMS SWAN
init(grid decomp)
run(sync. point)
finalize
init_paraminit_parallelinit_scaclarsinit_couplingro
ms_
init
rom
s_ru
n
main3d .....
waves_coupling...
mpi_finalizeclose_io
rom
s_fi
nali
ze
MPI_INIT
SW
INIT
MP
IS
WM
AIN
SW
EX
ITM
PI
SWINITSWREAD (grid)
init_coupling
swanmain .....
ocean_coupling...
mpi_finalizeclose_io
![Page 7: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/7.jpg)
Grid decomposition (during initialization)
ROMS SWAN
- Each tile is on a separate processor.
- Each tile registerswith MCT.
![Page 8: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/8.jpg)
init_coupling
ROMS- init_coupling SWAN- init_coupling
1
1
22
3
3
processed by each ROMS tileprocessed by each SWAN tile
![Page 9: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/9.jpg)
Synchronization (run phase)
ROMS- ocean_coupling SWAN- waves_coupling
MCT
MCT
processed by each ROMS tile processed by each SWAN tile
![Page 10: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/10.jpg)
ATM to OCN data fields
ATM
Uwind, V
wind, P
atm
, RH, T
air,
cloud
, rain
, eva
p, S
Wra
d, L
wrad
LH, H
FX, Ustr
ess,
Vstres
s
OCN
Integration and Application Network (ian.umces.edu/symbols),University of Maryland Center for Environmental Science.
Uwind, VwindSwrad, Lwrad,RH, Tair, cloud
Ustress, Vstress,Swrad, LwradLH, HFX
Use momentum + heat fluxes computed
in WRF for both ROMS+WRF
Use wrf vars in COARE algorithm
rain, evap
Salt flux
or#define BULK_FLUXES#define ATM2OCN_FLUXES
#define ATM_PRESS - Patm
#define EMINUSP
stflx_temp = Swrad+Lwrad +LH+HFX
stflx_salt = evap - rain
LH + HFX computed in bulk_fluxes
![Page 11: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/11.jpg)
OCN to ATM data fields
OCN
ATM
WAV
SST
Hwave , L
pwave ,
Tpsurf ,
Sur
face
flu
xes
Momentum
Heat
Moisture
= f ( Hwave, Lpwave, Tpsurf )
SST OCN
WAV
![Page 12: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/12.jpg)
How to create coupled application 1) Create all input, BC, init, forcing, etc files
for each model as if running separately. I recommend that you run each model separately first.
2) modify cppdefs in your header file. 3) SCRIP (if different grids or grid refinement) 4) coupling.in 5) coawst.bash 6) run it as coawstM
- Handout ends here- More in the online ppt
- Classroom tutorial will now follow:Projects/Sandy/create_sandy_application.m
![Page 13: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/13.jpg)
1) Use each model separatelyWRF
27 vertical levels dt 36 s Physics
Lin microphysics RRTM longwave, Dudhia
shortwave Mellor-Yamada-Janjic (MYJ) PBL Kain-Fritsch (KF) cumulus scheme
6 km grid
ROMS 16 vertical levels dt 240, 48 Physics
GLS turbulence closure COARE bulk fluxes BC's from HYCOM Timestep = 240s
5km and 1 km grid(s)
These models are on different grids.
![Page 14: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/14.jpg)
2) sandy.h
![Page 15: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/15.jpg)
3) SCRIP - grid interpolation
Atmosphere model provides heat flux to cover entire ocean grid. SCRIP interpolations weights needed to remap data fields.
Ocean model provides higher resolution and coupled response of SST to atmosphere. But the ocean grid is limited in spatial coverage so atmosphere model must combine data from different sources, which can create a discontinuity in the forcing.
http://climate.lanl.gov/Software/SCRIP/
Flux conservativeremapping scheme
Ocean grid5 km
Atm Grid6 km
HFLX10 GFS data
SST
![Page 16: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/16.jpg)
Libraries SCRIP - v 1.6 (distributed)Used when 2 or more models are not on the same grid.
1) cd to COAWST/Lib/SCRIP/source dir2) edit makefile3) make
![Page 17: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/17.jpg)
3) SCRIP Need to create SCRIP weights using
COAWST\Tools\mfiles\mtools\ create_scrip_weights_master.m
![Page 18: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/18.jpg)
3) SCRIP There will be created a weight file between each grid.So if you have 1 WRF grid, and 2 ROMS grids, it will produceatm1_to_ocn1_weights.ncatm1_to_ocn2_weights.ncocn1_to_atm1_weights.ncocn2_to_atm1_weights.nc
WRF grid 1
1000 m
5000 m
ROMS grid 1
ROMS grid 2
![Page 19: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/19.jpg)
4) coupling.in (this is a ROMS+WRF app)
set # procs for each model
set coupling interval. Can be different for each direction.
input file names. only 1 for WRF, 1 for ROMS, multiple for SWAN
SCRIP weights are listed here
![Page 20: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/20.jpg)
4) namelist.input
set # procs for atm model
need dt of 30 to divide evenly into coupling interval
of 1200 sec.
![Page 21: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/21.jpg)
6) run it as coawstM
use total number of procs from coupling.in only 1 executable
![Page 22: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/22.jpg)
Processor allocation
stdout reports processor allocation
This looks like from a different run,but you get the idea
![Page 23: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/23.jpg)
Processor allocation
"Timing for …." = WRF
"1 179 52974 02:59:00 " = ROMS
Here is where the model coupling synchronization occurs.
so probably could re-allocate more nodes to WRF
![Page 24: - How does the coupled modeling system work? and - Setting up a coupled application](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649d1f5503460f949f2833/html5/thumbnails/24.jpg)
JOE_TC - test case examples
JOE_TC test cases are distributed applications for testing ROMS+WRF coupling JOE_TCw = wrf only JOE_TCs = same grid, roms + wrf coupled JOE_TCd = different grids for roms and wrf,
needs scrip weights