![Page 1: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/1.jpg)
TOPAZoperations, products, ongoing
developments
Laurent Bertino, Knut Arild Lisæter,Goran Zangana, NERSC
OPNet meeting, Geilo, 6th Nov. 2007
![Page 2: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/2.jpg)
The TOPAZ model system TOPAZ3: Atlantic and Arctic
HYCOM EVP ice model coupled 11- 16 km resolution 22 hybrid layers
EnKF 100 members Sea Level Anomalies (CLS) Sea Surface Temperatures Sea Ice Concentrations (SSM/I) Sea ice drift (CERSAT)
Runs weekly since Jan 2003 ECMWF forcing (T799)
![Page 3: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/3.jpg)
Principle of the EnKF
To assimilate one observation one needs to know the error statistics: Which model variables to update? Over which area and depth?
We ignore most of that, but We assume we know the sources of errors
We set arbitrary error statistics for them Ensemble representation (emulation) of the errors
The error statistics and the impact of assimilation depend on our prior assumption on the error sources.
![Page 4: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/4.jpg)
Sequential data assimilationRecursive Monte Carlo method
Forecast Analysis
Observations
1. Initial uncertainty
2. Model uncertainty
3. Measurement uncertainty
12
3
Member1
Member2
Member99
Member100
10d Forecast
![Page 5: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/5.jpg)
Operator’s week
Tuesday Get ocean data + QC Start assimilation + QC Log status to text file
Wednesday Start 10d forecast + QC
Generate products Start ensemble forecast
Friday Generate “best guess”
products Cleanup
Automatic (cron deamon) Download and convert
ECMWF files (daily) Transfer files to
OPeNDAP Backup files to archive
E-mails standard output to the “calldesk” [email protected]
o
![Page 6: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/6.jpg)
The State Space
2D variables (800 x 880 grid cells) Barotropic pressure, u/v velocity, ice concentration, ice
thickness
3D variables (800 x 880 x 22 grid cells) Temperature, salinity, u/v current, layer thickness
TOTAL: n = 81.000.000 state variables 100 members in double precision = 60 Gb
![Page 7: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/7.jpg)
The observations
Sea level anomalies – SLA (satellite, radar altimeters): CLS Non linear function of state variables 100.000 observations every week
Sea-surface temperature – SST (satellite, optical): NOAA 8.000 observations every week
Sea-ice concentrations (satellite, microwave): NSIDC 40.000 observations every week
Sea-ice drift (AMSR-E, QuickSCAT): CERSAT/Ifremer 80.000 observations every week
TOTAL: m=228.000 obs Coming up: in-situ profiles (10.000 obs.), HR SST (120.000 obs.), HR
ice conc. (160.000 obs.) …
![Page 8: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/8.jpg)
Computations2 recursive steps
Propagation (HYCOM) Embarrassingly parallel 1 job per member
13 Gb 1 node x 1h 100x 16 CPU (4x8
OpenMP/MPI) SMT is used
1600 CPU hours / week
Analysis (EnKF) Sequential, 3 datasets MPI parallelization needed
on Njord (<13Gb constraint) 8 CPUs, 1:30 each dataset
MPI-parallel output Post-processing jobs to
assemble the EnKF output 16 CPUs, 30 min each dataset
72 CPU hours / week
![Page 9: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/9.jpg)
Incoming data volume
Ocean observations SSM/I
NSIDC: 7 Mb per week Merged SLA maps
Aviso: 3.8 Mb per week SST
NOAA: 0.5 Mb per week OSTIA: 10 Mb
In-situ Coriolis: 5-10 Mb per week
Negligible download / processing time Less than a minute
Atmospheric fields ECMWF T799 (1/4th deg)
1d analysis + 10d forecast 5 variables 1.2 Gb per day About 1h for download &
pre-processing Potential issue for daily
runs
![Page 10: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/10.jpg)
The Products
MERSEA standard productsForecast for the Tara expedition
![Page 11: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/11.jpg)
MERSEA Product generation
Conversion to NetCDF 1.2 Gb per week Past forecast overwritten by new best guess
File server External to NERSC (Parallab) 90 Gb produced every year Limited by disk space
![Page 12: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/12.jpg)
MERSEA productsAll are derived from HYCOM daily averages
Class1 3D daily fields (U,V, T, S) 15 fixed depths 2D Surface fields (SSH, sea
ice …) 25 km output grid
Class2 2D daily sections Moorings Same variables as Class1 33 fixed depths
Class3 Time series of integrated
variables Water mass transports
(Atlantic water …) Sea Ice transports Other: MOSF, ice area, volume
Class4 Model-minus observations Coriolis T/S profiles SSM/I ice concentrations Next: Tide gauge, SST, ...
Documented in MERSEA WP5
![Page 13: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/13.jpg)
Examples of output
Sea-ice minimum 2007
![Page 14: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/14.jpg)
![Page 15: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/15.jpg)
RMS errors - Example for the Barents Sea Ice
Analysis better than forecast Forecast better than
persistence See http://topaz.nersc.no
![Page 16: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/16.jpg)
A non-MERSEA product
TOPAZ successive forecasts in red Actual positions of Tara from DAMOCLES in black Updated on Google Earth [ K. A. Lisæter]
![Page 17: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/17.jpg)
Arctic sea-ice area minimum
![Page 18: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/18.jpg)
Forecasting the ice minimum in TOPAZ
![Page 19: TOPAZ operations, products, ongoing developments Laurent Bertino, Knut Arild Lisæter, Goran Zangana, NERSC OPNet meeting, Geilo, 6 th Nov. 2007](https://reader036.vdocuments.us/reader036/viewer/2022062409/56649ee55503460f94bf40e0/html5/thumbnails/19.jpg)
Conclusion
First operational application based on the EnKF inside Europe (so far only in USA and Canada) Installed on Met.no’s facilities (Njord) Code and restart files provided to met.no Scheduling shell scripts
Self-documented, but documentation not up-to-date.
Some tasks can be done in parallel Setting up the OPeNDAP/THREDDS
View the differences with NERSC TOPAZ on a LAS Scheduling the jobs in the operational suite
Start from the main script and descend into dependencies