cams ga data access by razinger
TRANSCRIPT
![Page 1: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/1.jpg)
Copernicus
Atmosphere
Monitoring
Service
CAMS General Assembly, Athens, 14-16 June 2016
Miha Razinger (ECMWF)
Accessing Data and
Products
![Page 2: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/2.jpg)
CAMS Datasets in Numbers
Combined volume of datasets 300 TB
180 individual products
33 distinct atmospheric parameters
9 data servers
2
![Page 3: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/3.jpg)
3
The challenge
Diversity of data producers, variety of data formats and
dataset volumes, existing data serving methods
![Page 4: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/4.jpg)
4
global surface ozone data June 2010
![Page 5: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/5.jpg)
What products and
services are available
from CAMS?
What is the quality
of a product?
How can I access the
data?
Is a product suitable
for operational use?
Can I use a product
in our commercial
service?
How can I read and
interpret the data?
it depends
it depends it depends
it depends
© Copyright Showeet.com
single entry point
single data licence
Questions from a potential user
![Page 6: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/6.jpg)
6
CAMS product catalogue
• Comprehensive and up-to-date inventory of
CAMS products
• 180 individual products and services
accompanied with description, details about
geographical and temporal coverage, data
access links, previews, documentation,
validation reports, contact details ...
• Interoperable: INSPIRE and WMO Core Profile
compliant meta data
• Can be harvested through OGC Catalogue
Service for the Web (CSW) (implemented using
pycsw library)
http://atmosphere.copernicus.eu/catalogue
![Page 7: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/7.jpg)
New catalogue BETA
![Page 8: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/8.jpg)
Product catalogue
FTP servers
WCS
WMS
Regionaldata
servers
Solar radiationservices
ECMWF data
servers
ECPDS
service
Distributed data access
![Page 9: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/9.jpg)
Accessing global data
1. New map application
2. ECMWF data servers
3. ECPDS data delivery
![Page 10: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/10.jpg)
New interactive map application
• OGC WMS and Leaflet library
• Fast development and publishing of maps
• Maps can be composed by various WMS sources
• Global or regional views, panning, zoom,
fullscreen view
• Point forecast chart generation from the maps
• Regional forecasts, observations
• Take it for a spin and send us your feedback at
10
BETA
http://atmosphere.copernicus.eu/maps/global-surface-forecasts
http://atmosphere.copernicus.eu/maps/global-total-column-forecasts
http://atmosphere.copernicus.eu/maps/aerosol-forecasts
http://atmosphere.copernicus.eu/maps/fire-activity
![Page 11: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/11.jpg)
![Page 12: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/12.jpg)
12
Iztočnice
• - catalogue as main entry point, distributed data servers
• - find, preview, access
• - use cases
• - compo announcement: service changes, timing, schedule, data
volume
• - data usage numbers and trends
• - DOIs
• - direct feedback but another survey might be useful
• - data volumes and disk write speed limitations: subsetting, slicing,
extracting - mogoče za na konec
![Page 13: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/13.jpg)
3 March 2015
13
![Page 14: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/14.jpg)
Delivering global model results
• Hosting the global real-time analyses and forecasts, MACC
reanalysis, GFAS and GHG flux inversions datasets
• Very capable of serving big volumes of data
• Services build on top of existing ECMWF systems
• Most of the data is archived in MARS, some on disk
• Not best suitable for data browsing and occasional / light-
weight data usage
![Page 15: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/15.jpg)
ECPDS FTP data service
• Serves global model, boundary
conditions and GFAS data
streams
• FTP-push and –pull service
• Suitable for large data transfers
• 24/7 monitored and supported
• Highly available, load balanced,
scalable service
• User Account Management
![Page 16: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/16.jpg)
16
Global model upgrade
• On 21 June we are going to upgrade the model version
• New horizontal resolution (from about 80 km to about
40 km)
• Two forecast runs per day (00 run available at 10 UTC
and 12 run at 22 UTC)
• 24/7 monitoring
• Test data available
![Page 17: CAMS GA Data Access by Razinger](https://reader031.vdocuments.us/reader031/viewer/2022021506/5889120a1a28ab4a5c8b5c43/html5/thumbnails/17.jpg)
Future plans
• Better data subsetting and slicing services (custom plots,
subareas, time series, vertical profiles ...)
• Exposing catalogue to web crawlers
• Dataset DOIs – improved data citation, better usage
tracking
• Follow CCCS Climate Data Store development