gems global e arth-system m onitoring using s pace and in-situ data
Post on 31-Jan-2016
43 Views
Preview:
DESCRIPTION
TRANSCRIPT
GEMSGEMS
Global Global EEarth-system arth-system MMonitoring onitoring
using using SSpace and in-situ datapace and in-situ data
GEMS– OverviewGEMS– Overview
Atmospheric Composition and Dynamics Build an operational thoroughly-validated
assimilation system for atmospheric composition and dynamics, by 2008. Daily global monitoring of dynamics &
composition Improvements in daily regional air quality
forecasts Monthly / seasonal estimates of surface fluxes
for CO2 and other species Extended reanalyses of composition & dynamics
Integrated Project co-funded by European Commission, 6th FP GMES (EC&ESA) Atmosphere theme
31 consortium members 4 years (started in March 2005)
Goals of GEMS:Goals of GEMS:Global Global EEarth-system arth-system MMonitoring onitoring
using using SSpace and in-situ datapace and in-situ data
Coordinator A.Hollingsworth (ECMWF)
Greenhouse Gases P.Rayner (LSCE)
Reactive Gases M.Schultz (Juelich)
Aerosols O.Boucher (MetOff)
Regional Air Quality V-H.Peuch (Meteo.Fr)
Validation H.Eskes (KNMI)
Production System A.Simmons (ECMWF)
Link between main elements of Link between main elements of GEMSGEMS
Model inter-comparisons in Model inter-comparisons in GEMSGEMS
GHGGHG: 2 models: 2 modelsIFS (ECMWF), LMDzT (LSCE)IFS (ECMWF), LMDzT (LSCE)
GRGGRG: : 3 models 3 models MOZART-3 (MPI-M), TM5 (KNMI), MOZART-3 (MPI-M), TM5 (KNMI), MOCAGE (MeteoFr)MOCAGE (MeteoFr)
AERAER: 1 model: 1 modelIFS (-> AeroCom)IFS (-> AeroCom)
RAQRAQ: 10 models: 10 modelsMOCAGE (MeteoFr), BOLCHEM (CNR-MOCAGE (MeteoFr), BOLCHEM (CNR-ISAC), EURAD (FRIUUK), CHIMERE ISAC), EURAD (FRIUUK), CHIMERE (CNRS), SILAM (FMI), MATCH (FMI), CAC (CNRS), SILAM (FMI), MATCH (FMI), CAC (DMI), MM5-UAM-V (NKUA), EMEP (DMI), MM5-UAM-V (NKUA), EMEP (MetNo), REMO (MPI-M), UMAQ-UKCA (MetNo), REMO (MPI-M), UMAQ-UKCA (UKMO)(UKMO)
RAQ: Ensemble forecastsRAQ: Ensemble forecasts
Analysis: Analysis: Centralized vs DecentralizedCentralized vs Decentralized
SameSame analysis analysis applied to all applied to all modelsmodels
Communication Communication platformplatform
A lot of work for A lot of work for analyzing teamanalyzing team
Progress of work Progress of work guaranteedguaranteed
Large storage Large storage facilities neededfacilities needed
Increased Increased implication of implication of individual groupsindividual groups
Duplication of Duplication of workwork
Distribution by Distribution by topictopic
Progress depends Progress depends on many peopleon many people
Large data Large data transfertransfer
Observational data setsObservational data sets
MethodologiesMethodologies
Which method to use?Which method to use? ““Eyeball” methods Eyeball” methods Basis statistical evaluationBasis statistical evaluation Sophisticated skill scoresSophisticated skill scores
Subject of Comparison?Subject of Comparison? Fields / Fluxes / Processes Fields / Fluxes / Processes
What to compare?What to compare? Continuous behaviorContinuous behavior Categorical behavior (Threshold Categorical behavior (Threshold
exceedance)exceedance) Averaging in time & spaceAveraging in time & space
Limited area / time verificationLimited area / time verification
Topics to think aboutTopics to think about
Influence of model resolutionInfluence of model resolution Interpolation techniquesInterpolation techniques Reference stateReference state (e.g. Observation, (e.g. Observation,
Climatology, Persistence, Median)Climatology, Persistence, Median) Errors of reference state / Errors of reference state /
observationsobservations Representativity of stationsRepresentativity of stations
Mixing of model skillsMixing of model skills Maintenance of data baseMaintenance of data base
Eyeball MethodEyeball Methodss
Comparison of time series at a given locationComparison of time series at a given location
P. AgnewP. Agnew
((EducatedEducated)) Eyeball Method Eyeball Methodss
Plots taken from talk of Adrian Simmons at the GEMS Annual Assembly, Feb. 2006
Comparison of fields at a given time (period)Comparison of fields at a given time (period)
Basis Statistic Evaluation Basis Statistic Evaluation HERBS HERBS ((M. ChinM. Chin))
How well does the distribution of model results How well does the distribution of model results
corresponds to the distribution of observed quantities?corresponds to the distribution of observed quantities? Histogram Histogram HH
What is the average error of the model compared to the What is the average error of the model compared to the
observations?observations? Mean error Mean error EE
How well do the model calculated values correspond to How well do the model calculated values correspond to
the observed values?the observed values? CorrCorrelationelation Coef Coefficientficient RR
What is the model bias?What is the model bias? Mean bias Mean bias BB
What is the overall model skill?What is the overall model skill? Skill score Skill score SS M. ChinM. Chin
Basic statistical evaluationBasic statistical evaluation
(Rank) Correlation coefficient between (Rank) Correlation coefficient between observations and reference stateobservations and reference state
Slope and offset Slope and offset in scatter plotsin scatter plots (Normalized) Root-mean square errors (Normalized) Root-mean square errors Bias (absolute and relative to reference values)Bias (absolute and relative to reference values) RMSE (absolute and relative to reference RMSE (absolute and relative to reference
values)values) Variability ratio (i.e. standard deviation of Variability ratio (i.e. standard deviation of
modelled values versus standard deviation of modelled values versus standard deviation of refecence values)refecence values)
Contingency tables defined with respect to Contingency tables defined with respect to thresholdsthresholds
Histograms of - absolute and relative - errorsHistograms of - absolute and relative - errors ……
P. AgnewP. Agnew
Basic statistical evaluation (RAQ)Basic statistical evaluation (RAQ)(continous behaviour)(continous behaviour)
measure of overall forecast errormeasure of overall forecast error
fractional gross errorfractional gross error
2' i in
i i i
f oB
N f o
2' i in
i i i
f oB
N f o
2' i in
i i i
f oB
N f o
normalized RMSE not used:normalized RMSE not used:
• errors not symmetric, errors not symmetric, • overweighting larger errors due to squaringoverweighting larger errors due to squaring
P. AgnewP. Agnew
Basic statistical evaluation (RAQ)Basic statistical evaluation (RAQ)(continous behaviour)(continous behaviour)
extent of over/under prediction extent of over/under prediction Modified mean biasModified mean bias: symmetric around 0, -1 : symmetric around 0, -1 -> 1,-> 1,
degree of pattern match:degree of pattern match: Correlation Correlation CoefficientCoefficient
2' i in
i i i
f oB
N f o
2' i in
i i i
f oB
N f o
2' i in
i i i
f oB
N f o
no offset no offset P. AgnewP. Agnew
correlation
Reference
stan
dard
dev
iation rm
s deviation
Taylor Diagramme condense info of spatio-temporal varying fields Use geometric relation between RMS – STDDEV –
CORRELATION Graphic display of model skill (RMS or others)
M. SchulzM. Schulz
Taylor skill scoresTaylor skill scores Skill score shouldSkill score should
increase monotonically with correlationincrease monotonically with correlation increase with match of modeled and observed varianceincrease with match of modeled and observed variance vary between 0-1
SS11 = 4(1+R) / [( = 4(1+R) / [(ff +1/ +1/ ff))22(1+R(1+R00)])] SS22 = 4(1+R) = 4(1+R)44 / [( / [(ff +1/ +1/ ff))22(1+R(1+R00))44] (+] (+ penalty for low corr.)penalty for low corr.)
Where RWhere R00=max attainable R, =max attainable R, ff =std_dev (model)/std_dev =std_dev (model)/std_dev (data)(data)
Categorical Skill ScoresCategorical Skill Scores
Definition of an event or a thresholdDefinition of an event or a threshold Number of a certain event (‘hit’)Number of a certain event (‘hit’) Basis: 2x2 contingency tableBasis: 2x2 contingency table
O NoO NoO YesO Yes
F NoF No
F YesF Yes
a+b+c+d=na+b+c+d=nb+db+da+ca+c
c+dc+d
a+ba+b
Correct Correct Rejections Rejections
d d
Misses Misses c c
False AlarmsFalse Alarms b b
Hits Hits a a
mo
do
bs
P. AgnewP. Agnew
Radar Model forecast
Source: Marion Mittermaier, derived from Casati (2004)
Radar > 1 mm Forecast > 1 mm Binary error image
X < uX < uX > uX > u
Y < uY < u
Y > uY > u
a+b+c+d=na+b+c+d=nb+db+da+ca+c
c+dc+d
a+ba+b
Correct Correct Rejections Rejections
d d
Misses Misses c c
False AlarmsFalse Alarms b b
Hits Hits a a
mo
do
bs
P. AgnewP. Agnew
Categorial Skill Scores: Categorial Skill Scores: Odds Ratio (Stephensen, 2000)Odds Ratio (Stephensen, 2000)
‘‘Odds Ratio’ defined asOdds Ratio’ defined as
ratio of probability that event ratio of probability that event occurs to probability that event occurs to probability that event does not occurdoes not occur
Easily calculated from contingency Easily calculated from contingency tabletable
Significance testing possibleSignificance testing possible
P. AgnewP. Agnew
How to compare ?How to compare ?
M. SofievM. Sofiev
Evaluation tools used/discussed Evaluation tools used/discussed within GEMSwithin GEMS
MetPy (ECMWF) MetPy (ECMWF) MMAS (FMI)MMAS (FMI) AeroCom (LSCE)AeroCom (LSCE) several other tools at partner several other tools at partner
institutesinstitutes
CDO, MetView?, CDAT, nco,…CDO, MetView?, CDAT, nco,…
MetPyMetPy gridded data – gridded datagridded data – gridded data
(gridded data – station data)(gridded data – station data)
(station data – stat ion data)(station data – stat ion data) Python-based scriptsPython-based scripts user-friendly front end (“Verify”)user-friendly front end (“Verify”) ““all” formats which Python supportsall” formats which Python supports to be run in batch modeto be run in batch mode designed for operational usedesigned for operational use additional visualization tool requiredadditional visualization tool required
C. Gibert et al., ECMWFC. Gibert et al., ECMWF
MetPyMetPycompute(
param = Z,levtype = pl,levelist = (1000,500,100),score = (ancf,ref),steps = StepSequence(12,240,12),area = (‘europe’, ‘north hemisphere’),forecast = forecast ()
persistence = persistence( )
analysis = analysis (expver = ‘0001’,date = DateSequence(20040101,20040131),
))
C. GibertC. Gibert
Model and Measurement Model and Measurement Analysis Software (MMAS)Analysis Software (MMAS)
Point data sets, NO MAPS Point data sets, NO MAPS station data – station data (ASCII)station data – station data (ASCII)
easy menu-driven for individual easy menu-driven for individual useuse
to be run in Microsoft Windows to be run in Microsoft Windows environmentsenvironments
output: ASCII & GraDS binoutput: ASCII & GraDS bin additional visualization tool additional visualization tool
neededneeded
M.SofievM.Sofiev
Finnish Meteorological InstituteFinnish Meteorological InstituteM. SofievM. Sofiev
MMAS MMAS strategystrategy
Measurements
Model
Merged Model/Measurements
Statistical characteristics
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Binary files with mapped
statistics
Input data sets Output data set
merges two arbitrary time-dependent data setsmerges two arbitrary time-dependent data sets computes statistics/skill scores for the merged setscomputes statistics/skill scores for the merged sets presents the results in numerical and graphic-ready presents the results in numerical and graphic-ready
formatformat M. SofievM. Sofiev
MANY THANKS TO MANY THANKS TO
Paul AgnewPaul AgnewOlivier BoucherOlivier BoucherMian ChinMian ChinFadoua Eddounia Fadoua Eddounia Hendrik Elbern Hendrik Elbern Claude GibertClaude GibertKathy LawKathy LawDimitris MelasDimitris MelasMartin SchultzMartin SchultzMichael SchulzMichael SchulzMikhael SofievMikhael SofievLeonor TarrasonLeonor Tarrason
Odds Ratio Skill ScoreOdds Ratio Skill Score
A skill score can be derived by a A skill score can be derived by a simple transformation:simple transformation:ORSS=(OR-1)/(OR+1)ORSS=(OR-1)/(OR+1)This mapping produces a skill This mapping produces a skill score in the range -1 to +1 score in the range -1 to +1
When ORSS=-1 forecasts and When ORSS=-1 forecasts and observations are independentobservations are independent
Providing number of forecasts is Providing number of forecasts is statistically significantstatistically significant, ORSS , ORSS approaching +1 indicates a skillful approaching +1 indicates a skillful forecastforecast
- different approaches around to do the - different approaches around to do the data handling data handling - software tools - software tools -regridding -regridding -visualisation -visualisation -maximizing the use of 'ensemble' data -maximizing the use of 'ensemble' data versus individual models versus individual models -involvement of participants. -involvement of participants. -dissemination of data -dissemination of data -typical problems encountered during -typical problems encountered during intercomparison and how to avoid them. intercomparison and how to avoid them.
- whatever you think is important to - whatever you think is important to share with your collegues along this share with your collegues along this concept. concept.
GEMS Research and Operational GEMS Research and Operational GoalsGoals
Delivering Daily global monitoring of dynamics & composition Improvements in daily regional air quality forecasts Monthly / seasonal estimates of surface fluxes
for CO2 and other species Extended reanalyses of composition & dynamics for
validation, and in support of GCOS Using
Best available models, assimilation systems Best available in-situ data Best available satellite data and algorithms
Collaborating with EU-IPs MERSEA & GEOLAND to implement IGOS_P Themes on
Carbon Cycle Atmospheric Chemistry
Build an operational thoroughly-validated assimilation system for atmospheric composition and dynamics, by 2008.
T. HollingsworthT. HollingsworthT. HollingsworthT. Hollingsworth
GEMS– OverviewGEMS– Overview
Atmospheric Composition and Dynamics Build an operational thoroughly-validated
assimilation system for atmospheric composition and dynamics, by 2008.
Integrated Project co-funded by European Commission, 6th FP GMES (EC&ESA) Atmosphere theme
17 M€ budget, 12.5 M€ EC-contribution 31 consortium members 4 years (started in March 2005)
T. HollingsworthT. Hollingsworth
top related