1 nextor monitoring and modeling nas performance at the daily level mark hansen performance metrics...
TRANSCRIPT
1
NEXTOR
Monitoring and Modeling NAS Monitoring and Modeling NAS Performance at the Daily Performance at the Daily
LevelLevelMark Hansen
Performance Metrics TIM
May 2002
2
NEXTORNormalizationNormalization
Translate before/after performance comparisons to with/without comparisons
Enable effects “Beyond FAA Control” to be removed while examining metrics of direct economic relevance
Enable comparison of observed and predicted (by models) impacts of technology deployments
3
NEXTORNEXTOR NormalizationNEXTOR Normalization
Effects of FFP1 on Terminal Area and En Route Performance
Focus on Delays and Time-in-System Metrics
Presented HereEffect of TTMA at LAXEffect of URET
4
NEXTORModeling CycleModeling Cycle
Models
R&D and Deployment Decisions Introduction
of New Systems
Benefits and Impacts of New Systems
Normalization
5
NEXTORConceptual FrameworkConceptual Framework
Demand
Weather
Conditionsat otherAirports
Time at Origin
AirborneTime
Taxi-In Time
Total Flight Time
System Performance
•En Route
•Terminal Area
•ATM
•Aircraft
6
NEXTORDaily Flight Time Index (DFTI)Daily Flight Time Index (DFTI)
Daily weighted average of flight times to a given airport from a set of origins
Flight time=Actual Arrival Time - Scheduled Departure TimeScheduled Flight Time+Departure Delay+Flight Time DelayTime-at-Origin+Airborne Time+Taxi-In Time
Origins have at least one completed flight in each day of sample
Weights reflect origin share of flights to study airport over study period
7
NEXTOR
Figure 2.1 Daily DFTI Values for LAX
100
120
140
160
180
200
220
240
1/1/
00
2/1/
00
3/1/
00
4/1/
00
5/1/
00
6/1/
00
7/1/
00
8/1/
00
9/1/
00
10/1
/00
11/1
/00
12/1
/00
1/1/
01
2/1/
01
3/1/
01
4/1/
01
5/1/
01
6/1/
01
Date
DFT
I (in
min
ute
s)
8
NEXTORFigure 2.2 Daily DFTI Moving Average, Three Components
0
20
40
60
80
100
120
140
160
180
200
Jan
-00
Fe
b-0
0
Ma
r-0
0
Ap
r-0
0
Ma
y-0
0
Jun
-00
Jul-
00
Au
g-0
0
Se
p-0
0
Oct
-00
No
v-0
0
De
c-0
0
Jan
-01
Fe
b-0
1
Ma
r-0
1
Ap
r-0
1
Ma
y-0
1
Jun
-01
Date
Av
era
ge
Tim
e P
er
Flig
ht
(Min
ute
s)
Taxi_inAir_BorneT_Origin
9
NEXTORWeather NormalizationWeather Normalization
Based on CODAS hourly weather observations for LAX
Factor analysis of weather dataCreate small number of factors that capture
variation in large number of variablesFactors are linear combinations of original
variablesFactors correspond to principal axes of N-
dimensional data elipse
10
NEXTORFactor Analysis with Two Factor Analysis with Two
VariablesVariables
X1
X2
First Factor
11
NEXTOR
Table 5. LAX Weather Factor Interpretations, Rotated Factors
Factor Interpretation 1 High temperature throughout the day
2 High cloud ceiling and VFR conditions in the afternoon and evening
3 High cloud ceiling and VFR conditions in the morning
4 High visibility throughout the day
5 High wind speed in the afternoon and evening
6 Medium cloud ceiling in the evening
7 Medium cloud ceiling in the morning hours
12
NEXTORDemand NormalizationDemand Normalization
Deterministic Queuing Analysis Arrival Curve from ASPM OAG Departure Curve Based on Either
Called AARs (Demand and Weather Normalization)
Hypothetical Rates
13
NEXTORDeterministic Queuing Deterministic Queuing
DiagramDiagram
14
NEXTOR
Figure 2.6 Hypothetical Queuing Diagram at for LAX Arrivals on June 8, 2001
0
200
400
600
800
1000
1200
0:00 2:24 4:48 7:12 9:36 12:00 14:24 16:48 19:12 21:36 0:00
Time of Day
Cu
mu
lati
ve
Nu
mb
er
Completed, Capacity=60
Completed, Capacity=80
Completed, Capacity=40
Scheduled
15
NEXTORTrends in HDD Parameters for the Study Period (7-Day Moving Average)
0
0.5
1
1.5
2
2.5
3
3.5
4
1/7/00 4/16/00 7/25/00 11/2/00 2/10/01 5/21/01
Date
HD
D R
ela
tive
to
Fir
st W
ee
k o
f 20
00
HDD50
HDD70
HDD80
OAG_Arrivals
16
NEXTORNormalization for Conditions Normalization for Conditions
at other Airportsat other Airports Consider airports included in DFTI
average For each compute daily average
departure delay for flights not bound to LAX region
Average airport departure delays using DFTI weights
17
NEXTOR
Figure 2.4 Origin Airport Congestion at LAX
0
5
10
15
20
25
30
35
40
45
1/1/
00
2/1/
00
3/1/
00
4/1/
00
5/1/
00
6/1/
00
7/1/
00
8/1/
00
9/1/
00
10/1
/00
11/1
/00
12/1
/00
1/1/
01
2/1/
01
3/1/
01
4/1/
01
5/1/
01
6/1/
01
Date
Ori
gin
Air
port
Del
ay (
min
)
18
NEXTORPerformance ModelsPerformance Models
ttttt ODELDMDWXfY ),,(
Where:Yt is DFTI or DFTI component for day t;WXt is vector of weather factors for day t;DMDt is vector of demand factors for day t;ODELt is average origin departure delay for day t;t is stochastic error term.
19
NEXTORFunctional Forms ConsideredFunctional Forms Considered
ParametricLinear (with 3, 6, 9, and 12 weather
factors)Quadratic response surfaceNon-linear
Non-parametric9 clusters based on 3 weather factors12 clusters based on 9 weather factors
20
NEXTORLinear Model Estimation Results Linear Model Estimation Results
(Pre-TTMA)(Pre-TTMA)
Variable Description Estimate T - statistic P - value
INTERCEPT Intercept 138.055 567.065 0.0001 OAC Origin airport congestion 1.128 44.351 0.0001 WX1 Warm daily temperatures -1.357 -12.101 0.0001 WX2 VFR ops, no low cloud ceiling in the morning -0.988 -7.116 0.0001 WX3 VFR ops, no low cloud ceiling in the afternoon -1.123 -7.583 0.0001 WX4 High visibility throughout day -0.449 -3.575 0.0004 WX5 Medium cloud ceiling throughout day 1.440 10.555 0.0001 WX6 High winds throughout the day 0.512 4.531 0.0001 WX7 High cloud ceiling throughout day 0.911 4.172 0.0001 WX8 Precipitation in late morning and afternoon 1.871 8.324 0.0001 WX9 Precipitation in early morning -0.379 -2.614 0.0091 DMD1 Peak demand 0.075 0.725 0.4685 DMD2 Base demand 0.440 4.574 0.0001
ADJUSTED R2 0.743
21
NEXTORPredicted vs Actual ValuesPredicted vs Actual Values
130.00
140.00
150.00
160.00
170.00
180.00
190.00
200.00
210.00
130.00 140.00 150.00 160.00 170.00 180.00 190.00 200.00
Observed Values (min)
Pre
dic
ted
Val
ues
(min
)
22
NEXTOROutliersOutliers
Used TMU logs to investigate days for which predictions have large errors
Reasons for higher than predicted DFTIEast flowRadar outagesAir Force OneOver-stringent ground delay program
No clear explanations for lower-than-average DFTI: “No news is good news”
23
NEXTOR Variable Parameter Estimates DFTI At Origin Airborne Taxi-in intercept 139.29 15.23 115.9 8.11 pFAST -1.99 -1.71 -0.19 -0.07 OAC 1.39 1.29 0.03 0.06 Peak Demand -0.35 -0.1 -0.24 -0.01 Base Demand 0.97 0.74 0.04 0.19 Weather Factor1 -3.37 -0.89 -2.63 0.14 Weather Factor2 -2.66 -1.8 -0.75 -0.12
Weather Factor3 -1.88 -1.36 -0.48 -0.04 Weather Factor4 0.19 -0.24 0.49 -0.07 Weather Factor5 1.48 0.73 0.79 -0.03 Weather Factor6 0.46 0.12 0.31 0.02 Weather Factor7 0.64 0.27 0.29 0.81 Adjusted R-Square 0.79 0.83 0.55 0.39 Significant at 5% level Significant at 10% level
DFTI Model ResultsDFTI Model Results
24
NEXTOREffect of TTMA at LAXEffect of TTMA at LAX
Used same methodology but included TTMA (PFAST) dummy variable for Post 2/9/01
Used AARs in computing deterministic delay variable
Considered both DFTI and Average Delay as Dependent Variables
25
NEXTOR
Model Model1 2
Variable Estimate EstimateIntercept 4.03 5.49pfast -1.74 -3.76oac 0.58 0.54ahdd 0.59 0.22pct EDCT 0.18wx_fact1 -0.79 -1.50wx_fact2 -1.30 -1.15wx_fact3 -0.98 -0.27wx_fact4 0.01 0.15wx_fact5 0.87 0.72wx_fact6 0.25 -0.02wx_fact7 0.24 -0.10
Adj R2 0.77 0.87
Delay Model Results Delay Model Results
26
NEXTORURET ImpactURET Impact
Performed Flight Level Analysis Observed Airborne Time for ASQP Flights for
Corresponding Months Before and After URET (about 800,000 per analysis)
Effects EstimatedBefore vs After URETUse URET Sectors vs Don’t Use Use URET Sectors After URET
Control for Distance, Direction, and Originating Airport and Destination Airport
27
NEXTORURET Model ResultsURET Model Results
Variable Estimate
Intercept 10.14distance 0-200 0.16distance 200-500 0.14distance 500-1000 0.14distance 1000+ Miles 0.13change in latitude 5.00change in longitude 39.77after URET 0.42Use URET sector 0.45Use URET sector after URET -0.24
Adj R2 0.98
28
NEXTORConclusionsConclusions
A Priori Modeling without Ex Post Analysis is One Hand Clapping
Normalization Required to Compare Observed with Predicted Outcomes and Account for Factors Beyond FAA Control
We can Do This