displaying the verification results of terminal aerodrome ... · • international civil aviation...
Post on 23-May-2020
11 Views
Preview:
TRANSCRIPT
Displaying the Verification Results of Terminal Aerodrome Forecasts for Thunderstorms and Visibility
Jadran Jurković 1
Zoran Pasarić2 and Igor Kos 1 1 Croatia Control ltd 2 University of Zagreb, Dept. Of Geophys.
7th International Verification Methods Workshop | Berlin 2017
1
Outline
• Introduction
• Diagnostic purpose
• Displaying Results
– Thunderstorms
– Visibility
• Conclusion
7th International Verification Methods Workshop | Berlin 2017
2
Introduction: TAF
• Terminal aerodrome forecast
• Standard ICAO product
• but each country has its own style (Sharpe 2016) – „…as agreed by the meteorological authority with
the ATS authority and operators concerned”
– duration 24h (9-32h)
– Other: standards and recommendations , criteria for groups of changes (and/or AMD)
7th International Verification Methods Workshop | Berlin 2017
3
Introduction: TAF verification
• ICAO Annex 3 attachment B – Operationally desirable accuracy of forecasts – The same in EASA documents (without changes)
• Several basic approaches – Harris (2000), Mahringer (2008), Sharpe (2016)
• Other features for TAF verifications: – TAF statements are deterministic, probabilistic and temporal – Results: scores, contingency tables – Rare events
• forecast for a given point • the verification time period is just one hour • aviation requirements usually refer to high impact weather
– Climatological difference
7th International Verification Methods Workshop | Berlin 2017
4
7th International Verification Methods Workshop | Berlin 2017
Diagnostic verification
• Basic principles – G. Mahringer (2008)
– Best (FC and OBS) and worse (FC and OBS) conditions are verified
• Diagnostic verification
– Weakness and strength of forecast
– Verification of special problems for aviation in Croatia
• Convection (TS)
• Fog (reduced visibility)
• Wind (especially bora events)
5
7th International Verification Methods Workshop | Berlin 2017
Verification
• Verification scores – p0, Bias, TCC/PCC (tetrachoric/polychoric correlation
coefficient) – Juras and Pasarić (2006): Application of tetrachoric
and polychoric correlation coefficients to forecast verification (link) • Pearson (1900) • measure of association in contingency tables • TCC=-1, 1, TCC=0 for random • TCC/PCC do not depend on bias nor on marginal frequencies
=> independent information between them • TCC/PCC are particularly good for rare events (fig.2)
6
7th International Verification Methods Workshop | Berlin 2017
Verification
• Verification of hours with TS • FC: „worst conditions”=> all group of changes
• OBS: METAR 0:00 or 0:30
– Contingency table 2x2
• Frequency of observed hours with TS at airports in Croatia => 0.8-1.5% => rare event
• Relatively large false alarms 7
7th International Verification Methods Workshop | Berlin 2017
TS verification: hit a)
• Analysis of hit event – Months
– Hours
– Issue time
– Lead time
– Chronological
– Contingency table
– Forecast expression of TS in TAFs
Challenges in meteorology 3 21-22 November 2013.,Zagreb
ANALIZA PROGNOZE GRMLJAVINE (TS)Termini prognozirani: a = Točno (pogodak)
(Prognozirano: DA Motreno: DA) (interno)
Zračna Luka: LDZA Razdoblje: 2009-2013
Da Ne Ukupno
a= 1062 b= 8059 9121
c= 820 d= 164308 165128
1882 172367 174249
Načini i i izvještavanje za događaje "a" LDZA
Ne
... ...
Ukupno
METAR TAF70 TS
41 TSRETS
41 TS TS
25 TS
22 -TSRARETS -RA
20 -TSRA -TSRA
19 -TSRA TS
15 TSRA TSRA
15 TSRETS
15 VCTS
14 -TSRA TSRA
12 -TSRA -RA
12 TS TSRETS
11 -TSRA
9 -TSRARETS
8 RARETS RA
8 RETS
8 -TSRARETSRA
8 TS -TSRA
6 TSRARERA
6 TSRARETS RA
6 -TSRA -TSRARETS
6 TSRA -TSRARETS
5 -TSRARERA -
TSRARERA
5 TSRARETS
5 -TSRA TSRARERA
5 TSRETS RETS
4 +TSRARESHRA -TSRA
4 -RA -RARETS
4 -RA TS
183 NSW TE SHRA P40 TSRA
99 NSW TE TSRA
86 NSW P40 TSRA
78 NSW TE SHRA P30 TSRA
57 NSW P30 TSRA
27 NSW TE TSRA P30 TSGR
12 RA TE TSRA
10 NSW TE SHRA TE TSRA
9 NSW P40 SHRA P30 TSRA
6 NSW TE RA P30 TSRA
6 NSW TE TSRA BE RA
5 NSW TE TSRA P40 TSGR
4 NSW TE SHRA P40 TSRA BE
RA BR
4 NSW TE RA P40 TSRA
4 RA P30 TSRA
4 BR P40 BCFG P30 TSRA
3 NSW BE TSRA
3 NSW P40 TS
3 NSW TE SHRA P40 TSRA BE
RA
3 NSW TE TSRA P30 TSRAGR
3 NSW TE SHRA TE TSRA P30
TSGR
2 NSW FM RA TE SHRA P30
TSRA
2 TSRA
2 NSW TE TSRA P30 TSRA
2 NSW BE RA P30 TSRA
2 NSW P40 SHRA TE TSRA
2 RA P40 TSRA
Motreno
Prognozirano
Da
0 1 21 56
279 311265
17394
37 0 13
0
20
40
0
500
1 2 3 4 5 6 7 8 9 10 11 12 Pr.
br.
Kl.
TS
term
ina
M
ETA
R (
po
lusa
tni)
Br.
"Pro
pu
st"
ili
mo
tren
ih T
S sa
ti
Raspodjela prognoziranih sati "Točno (pogodak)" po mjesecima
KLIMA pros. br. TS termina Uk. verif. "Točno (pogodak)" sati 2009-2013 Uk. Motreni TS termini 2009-2013
29 25 22 8 13 12 23 13 9 1032 35
78101 95 86
117148
102 9851 36 50 57
0
50
100
150
200
0
100
200
300
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Pr.
br.
Kl.
TS
term
ina
M
ETA
R (
po
lusa
tni)
Br.
"Pro
pu
st" i
li
mo
tre
nih
TS
sati
Sat u danu (UTC)
Raspodjela prognoziranih sati "Točno (pogodak)" po satu (UTC) u danu
KLIMA pros. Br. TS termina Uk. verif. "Točno (pogodak)" sati 2009-2013 Uk. Motreni TS termini 2009-2013
74 7756
4467 65 55 52
37 3254
6552 54
39 3147
6851 54 45
3344 54
0
50
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Sat nastupa TAF-a
Raspodjela prognoziranih sati "Točno (pogodak)" po vremenu nastupa(1-24) TAF-a
5 11 17 23 Ukupno Točno (pogodak)" sati
294332 336
288
200
300
400
5 11 17 23
Raspodjela prognoziranih sati "Točno (pogodak)" po vremenu izdanja TAF-a
0 0 02210
6993
353 5 0 4 0 0 9 0
19243257
1 0 0 0 0 0 0 8
75104
360 12 9 0 9 0 0 2 11
643930
9 2310 0 0 0 1 8 4
47364463
323 0 0
0
100
200
2009-1 2009-4 2009-7 2009-10 2010-1 2010-4 2010-7 2010-10 2011-1 2011-4 2011-7 2011-10 2012-1 2012-4 2012-7 2012-10 2013-1 2013-4 2013-7 2013-10
Br.
"Pro
pu
st" i
li
mo
tre
nih
TS
sati
Raspodjela prognoziranih sati "Točno (pogodak)" po mjesecima kronološki 2009-2011
Uk. verif. "Točno (pogodak)" sati 2009-2013 Uk. Motreni TS termini 2009-2013
8
7th International Verification Methods Workshop | Berlin 2017
Bias / TCC display
Example of contingeny tables for different bias and TCC
Example: frequency (a+c) of hours with TS je 1.3%
7 0,65% 9,10% 0,90% 9,10% 1,15% 9,10%
0,65% 89,60% 0,40% 89,60% 0,15% 89,60%
1,30% 98,70% 1,30% 98,70% 1,30% 98,70%
5 0,55% 6,50% 0,79% 6,50% 1,09% 6,50%
0,75% 92,20% 0,51% 92,20% 0,21% 92,20%
1,30% 98,70% 1,30% 98,70% 1,30% 98,70%
3 0,41% 3,90% 0,64% 3,90% 0,90% 3,90%
0,89% 94,80% 0,66% 94,80% 0,40% 94,80%
1,3% 98,7% 1,3% 98,7% 1,3% 98,7%
0,5 0,65 0,8
TCC
bia
s
a b
c d
9
7th International Verification Methods Workshop | Berlin 2017
Bias / TCC display example 2
Dispersive results are expected
10
Tablica kontningencije Tablica kontningencije Verifikacijski indeksi Ostali pomoćni pokazatelji
Razdoblje: 9.2.2012-31.12.2014. Aspolutne čestine [sat] Relativne čestine [%] Točnost Pristranost Pa+Pb Pa+Pc a/(a+c) a/(a+b) a/(a+b+c) (PSS+HSS)/2
Ime_EERS a b c d Pa Pb Pc Pd TCC Bias a-c/(a+c) Pfc Pobs POD pTSfc CSI KPI
Puhalo Dragović, Nataša(729) 480 93 521 66 10756 0,8% 4,6% 0,6% 94,1% 0,70 3,9 159 0,17 5,4% 1,4% 0,58 0,15 0,14 0,26SMB 4111 703 4684 549 92366 0,7% 4,8% 0,6% 94,0% 0,67 4,3 0,12 5,6% 1,3% 0,56 0,13 0,12 0,25
Raspodjela događaja a (pogodak), b (krivi alarm), c(propust)
Broj
TAFovaMotreni
TS sati
1,0
2,0
3,0
4,0
5,0
6,0
7,0
0,4 0,5 0,6 0,7 0,8 0,9 1,0
Pri
stra
no
st (
Bia
s)
Točnost (TCC)
Indeksi verifikacije TS prognoze u TAF-u po prognostičaru za razdoblje 9.2.2012-31.12.2014.
FC
Ostali
SMB
PO godinama
0
20
40
60
0 2 4 6 8 10 12 14 16 18 20 22
SAT U DANUb
c
a
0
20
40
60
1 3 5 7 9 11 13 15 17 19 21 23
NASTUPNI SATb
c
a
0
50
100
150
200
5 11 17 23
IZDANJE
0
50
100
150
200
1 2 3 4 5 6 7 8 9 10 11 12
MJESEC
XY
7th International Verification Methods Workshop | Berlin 2017
TS verification: forecaster LDZA
Challenges in meteorology 3 21-22 November 2013.,Zagreb
TCC
BIA
S
11
7th International Verification Methods Workshop | Berlin 2017
Bias / TCC display example 3
• Verification of visibility
– According to categories with limits 150, 350, 600, 800, 1500, 3000 and 5000m
– Best (FC and OBS) and worse (FC and OBS) conditions are verified
– 2 contingency tables NxN: polyhoric correlation coefficient is used
12
7th International Verification Methods Workshop | Berlin 2017
Bias / TCC display example 4
Better results when p0 is higher
Bias is larger - p0 smaller
13
7th International Verification Methods Workshop | Berlin 2017
Conclusions
• Diagnostic verification – Based on approach by Mahringer (2008)
• Verified elements in TAFs are rather rare • Displayed results are used with triplet
– marginal frequency, bias and tetrachoric coefficient (Juras and Pasarić 2006)
– Bias and TCC represents forecast – 4 examples for thunderstorms and visibility
• Additional conclusion : verification results depend on – climatology (frequency of an event) – forecaster
14
7th International Verification Methods Workshop | Berlin 2017
Literature
• Harris GR. 2000. Comparison of different scoring methods for TAFs and other
probabilistic forecasts. 15th AMS Conference on Probability and Statistics in the Atmospheric Sciences, 8–12 May 2000, Asheville, NC.
• International Civil Aviation Organization (ICAO). 2010. Meteorological Services for International Air Navigation. Annex 3 to the Convention on International Civil Aviation, 15th edn. ICAO: Montreal.
• Mahringer G. 2008. Terminal aerodrome forecast verification in Austro control using time windows and ranges of forecast conditions. Meterol. Appl. 15: 113–123.
• Pearson, K. 1900: Mathematical contributions to the theory of evolution. VII. On the correlation of characters not quantitatively measurable, Philos. Tr. R. Soc. S.–A, 195, 1–47.
• Juras J., Pasarić Z. 2006, Application of tetrachoric and polychoric correlation coefficients to forecast verification, Geofizika 23, 59-82
• A. Sharpe M., E. Bysouth C. , Trueman M. 2016, Towards an improved analysis of Terminal Aerodrome Forecasts, Meterol. Appl. 23, 698–704.
15
top related