bde esd tool - big data met norway rasmus benestad
TRANSCRIPT
![Page 1: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/1.jpg)
Utskifting av bakgrunnsbilde:
- Høyreklikk på lysbildet og velg «Slide» -> «Set Background Picture for Slide...»
- Velg ønsket bilde og klikk «Open»
Brussels 2015-06-15
Climate analysis & Big data
Rasmus E. Benestad
Abdelkader Mezghani, & Kajsa M. Parding
esd for retrieving, processing, dissecting, analysing, and visualisation
![Page 2: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/2.jpg)
How do I find useful climate informationin large volumes of data?
![Page 3: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/3.jpg)
Information & Data & Knowledge
● Information "...in its most restricted technical sense, is a sequence of symbols that can be interpreted as a message".(wikipedia)
● Data are measurements, observations, calculations
● Knowledge is expectation about causalities, dependencies, and why?
![Page 4: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/4.jpg)
Norwegian Meteorological Institute
The ultimate objective is to find the answers.
The data is the source and the means finding answers.
Analytical tools designed for data analysis/statistics.
What is the question?
![Page 5: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/5.jpg)
Big data - simple algorithmsFast “distillation” of information
![Page 6: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/6.jpg)
Open source tool ‘esd’http://github.com/metno/esd
![Page 7: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/7.jpg)
Norwegian Meteorological Institute
esd: open-source and free
![Page 8: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/8.jpg)
Data accessSource of information
![Page 9: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/9.jpg)
Temperature
ECA&D:~1.1 Gb-temperature-precipitation
![Page 10: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/10.jpg)
Temperature
ECA&D +GHCN +MET Norway-temperature
![Page 11: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/11.jpg)
Precipitation
![Page 12: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/12.jpg)
Reanalyses and satellite data
![Page 13: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/13.jpg)
Climate model results
![Page 14: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/14.jpg)
Norwegian Meteorological Institute
Make use of information hidden in vast archives of climate model results and observations
![Page 15: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/15.jpg)
ExampleExtracting information embedded in global climate models and observations
![Page 16: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/16.jpg)
Climate model results
![Page 17: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/17.jpg)
Norwegian Meteorological Institute
![Page 18: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/18.jpg)
Norwegian Meteorological Institute
![Page 19: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/19.jpg)
Norwegian Meteorological Institute
What is hidden behind the results?
![Page 20: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/20.jpg)
Norwegian Meteorological Institute
Building block
Empirical-statistical downscaling
Dependencies & connections
Redundancy
![Page 21: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/21.jpg)
Norwegian Meteorological Institute
Extension of the results to regions
![Page 22: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/22.jpg)
Norwegian Meteorological Institute
High one-in-twenty mean summer temperature in 2100
![Page 23: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/23.jpg)
InfographicsExposing different aspects
![Page 24: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/24.jpg)
Different sides to information
![Page 25: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/25.jpg)
Changes in rainfall statistics?
![Page 26: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/26.jpg)
Quick look at temperatures
![Page 27: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/27.jpg)
Maps
Correlation
![Page 28: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/28.jpg)
AnalysisPatterns of behaviour
![Page 29: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/29.jpg)
Anomaly with respect to latitude
![Page 30: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/30.jpg)
How similar are the reanalyses? Correlation maps
![Page 31: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/31.jpg)
How good are the models?
![Page 32: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/32.jpg)
Using mathematics to make sense of the data
Is there a change in the precipitation?
![Page 33: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/33.jpg)
Norwegian Meteorological Institute
How much does it rain?
![Page 34: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/34.jpg)
Norwegian Meteorological Institute
Global rain gauge data
= huge volume
35,000 rain gauges with daily data
Best way to mine hidden information?
![Page 35: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/35.jpg)
Norwegian Meteorological Institute
Understand the data
Exponential distribution?
Vast number of data points on top of each other
![Page 36: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/36.jpg)
Norwegian Meteorological Institute
Extract the essence (cleverly)
Principal component analysis: two main characteristics
![Page 37: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/37.jpg)
Norwegian Meteorological Institute
E.g. The wet-day 95-percentile for 24-hr precipitation
Calculated Observed
![Page 38: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/38.jpg)
Norwegian Meteorological Institute
Test results
![Page 39: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/39.jpg)
Norwegian Meteorological Institute
Changing rainfall patterns
![Page 40: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/40.jpg)
Norwegian Meteorological Institute
Changing rainfall patterns
![Page 41: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/41.jpg)
Norwegian Meteorological Institute
Aggregate
Principal Component Analysis (PCA)
Regression analysis
Quick search
Combination of sources
Mapping/gridding
Statistical distributions
Main instruments
![Page 42: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/42.jpg)
Norwegian Meteorological Institute
Tricks for efficiency
Structured and standardised metadataCommon information model (CIM)Data reference syntax (DRS)classes and ‘S3’ methodsFast algorithms making use knowns
![Page 43: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/43.jpg)
Norwegian Meteorological Institute
• Facilitate intercomparisons
• Sharing of generic methods
• Traceability and replicability
• Promotes community building & discussions
Benefits of common standards & structures
![Page 44: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/44.jpg)
Norwegian Meteorological Institute
Summary
●esd - “easy and simple data” or empirical-statistical downscaling
●Statistics - information from the data
●Aim to address specific questions
![Page 45: BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad](https://reader031.vdocuments.us/reader031/viewer/2022022414/5872d1991a28ab74188b528f/html5/thumbnails/45.jpg)
Meteorologisk institutt
Thanks for your attention!