Tom G. Reynolds, Michael McPartland, Tom Teller & Seth Troxel Sponsor: Gary Pokodner, FAA Weather Technology in the Cockpit

June 24, 2015 11th USA/Europe Air Traffic Management Research and Development Seminar (ATM2015)

Exploring Wind Information Requirements for Four Dimensional

Trajectory-Based Operations*

*This work was sponsored by the Federal Aviation Administration (FAA) under Air Force Contract FA8721-05-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the United States Government

Winds/4D-TBO - 2 06/24/15

Meter Fix

Time target @ Meter Fix; Wind updates

Actual winds

Wind Needs for NextGen Applications

RTA

Aircraft Winds

ATC/Airline

Ground Winds

Objective: Establish relationship of wind information accuracy to 4D-TBO and IM performance and hence identify wind needs to support them

• Many NextGen applications depend on access to high accuracy wind data – Required Time of Arrival (RTA) at a

meter fix under 4D-Trajectory Based Ops – Assigned Spacing Goal (ASG) between

aircraft under Interval Management (IM)

Winds/4D-TBO - 3 06/24/15

Performance Assessment

Wind Information Analysis Framework

Allows analysis of wind information impacts on range of NextGen applications Designed to be scalable wrt scope/fidelity & flexible wrt application

Wind forecast assumpt

ions

Wind Scenario Truth Forecast

Airc

raft/

Aut

omat

ion

Sim

ulat

ion

Stakeholder needs

ATC Scenario 4D-TBO IM

Wind Requirements

Identify combinations of wind data forecast model & look-ahead time which meet required wind info

quality goal

Perf

Met

ric

Wind Information Quality

Low

Med

High

Sampletargetperformance

Aver

age

fore

cast

err

or

Look-ahead time

Benign, Moderate,

Severe, Extreme

Model A B C

Req’d wind info

quality

Winds/4D-TBO - 4 06/24/15

Wind Information Implications Flow Diagram

1. Define scenarios of interest

2. Identify performance trade-space

3. Select target performance level

4. Do feasible combinations of performance drivers meet target level of performance?

Yes (wind error limit)

No

5. Do feasible combinations of wind

forecast model and look-ahead time meet error

limit?

Yes

No

Identified need for enhanced

automation and/or wind model

“or”

6. Define procedure/ConOps/supporting technology needs to get required wind info accuracy to aircraft

Perf

Met

ric

Wind Information Quality

Low

Med

High

Sampletargetperformance

Aver

age

fore

cast

err

or

Look-ahead time

Model A B C

Winds/4D-TBO - 5 06/24/15

• Analysis focused on impact of wind information accuracy on ability to achieve RTA for range of descent scenarios

4D-TBO Analysis Overview

MeterFix

Time target@ Meter Fix;

Wind updates?

Actualwinds

RTA

AircraftWinds

ATC/Airline

GroundWinds

RTA Compliance Error

Freq

uenc

y RTA 95%confidence

interval

0

Winds/4D-TBO - 6 06/24/15

• Nov/Dec 2011 flight trials of Alaska Airlines B737/GE flights into SEA

• Latest RUC winds uploaded to FMS approx. 70 mins prior to arrival

• RTAs assigned via aircraft/ATC negotiation

Baseline SEA Flight Trials*

σ = 11.4 secs

μ = 9.0 secs

RTA Compliance Performance (secs)Crossing Time – FMS RTA Time

Num

ber o

f Flig

hts

EARLY LATE

* Source: Wynnyk, C., and D. Gouldey, “2011 Seattle Required Time of Arrival (RTA) Flight Trials Analysis Report”, MITRE CAASD, July 2012

Flight trial data indicated importance of wind information for effective 4D-TBO procedure execution in an operational setting

Winds/4D-TBO - 7 06/24/15

• Parametric studies conducted on 3 aircraft/FMS combinations: – B737-700 with GE FMS (B737/GE)

• Full flight phase RTA speed control – B757-200 with Honeywell Pegasus FMS

• Current operational “Black Label” (B757/HW BL) Cruise-only RTA speed control

• Research prototype “Red Label v1” (B757/HW RL) Full flight phase RTA speed control User-adjustable “RTA tolerance setting” Calculation and display of ETA min/max User-adjustable RTA speed limits

Lincoln/MITRE FMS Simulations

Allowed assessment of performance of operational & prototype FMSs in controlled environments

Winds/4D-TBO - 8 06/24/15

Wind Information Analysis Framework

Element Independent Variable Values Tested Number of

Permutations

Wind Scenario Truth field

“Benign” “Severe” with 180° Course Reversal

“Severe Headwind” 3

Forecast error distribution σ = 5 kts RMSE* σ = 25 kts RMSE 2

ATC Scenario

Trajectory Cruise FL290, FL390, Meter fix 12,000 ft 2

RTA assignment distance (from meter fix 150 NM, 250 NM 2

RTA time assigned (Relative to range of

achievable times) Early, Mid-range, Late 3

Waypoint spacing 10 NM, 100 NM 2

Aircraft/Automation Simulation Aircraft/FMS type

B737/GE (±6/±30 secs RTA tol) B757/HW BL (±30 secs RTA tol)

B757/HW RL (±6/±30 secs RTA tol) 4

Totals Total number of Monte Carlo conditions/permutations Total number of runs (25-100 runs per condition)

576 28,800

Lincoln/MITRE FMS Simulation Conditions

𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = �

1𝑁𝑁�(𝑓𝑓𝑛𝑛 − 𝑜𝑜𝑛𝑛)2𝑁𝑁

𝑛𝑛=1

* for N pairs of observations, o and forecasts f

Winds/4D-TBO - 9 06/24/15

Lincoln/MITRE FMS Simulation Results Effects of Aircraft/FMS type, forecast error and RTA tolerance

-50-40-30-20-10

0102030405060708090

100

Wind forecast error (RMSE kts): 5 25 5 25 5 25 5 25

Aircraft/FMS: B737/GE B757/HW RL B757/HW RL B757/HW BL

RTA tolerance setting (secs): 6 6 30 30

RTA

Com

plia

nce

Perf

orm

ance

(sec

s)

Mean RTA TE RTA 95% CI

LATE

EA

RLY

Wind forecast error level and automation capability impact RTA performance

Winds/4D-TBO - 10 06/24/15

Avionics Control and Display Unit

(CDU)

Mode Control Panel

(MCP) Flight Control Computer

(A/P)

Auto Throttle Computer

(A/T)

Air Data Computer

(ADC)

Inertial Reference Computer (ADIRS)

Fuel Quantity/Flow

Computer (FQPU)

GPS

Digital Clock

Thrust Management

Computer (TMC)

FMC Discrete

Pins

FMS

Pilot

Airline ATC

• Can execute 20+ instances in parallel

Lincoln FMSim Implementation

Honeywell Pegasus

FMS

= Lincoln = Honeywell

Lincoln Virtual MCDU

Allows assessment of operational & enhanced FMSs in controlled environments & flexible simulation platform

CDU

Honeywell Pegasus

MCDU

Winds/4D-TBO - 11 06/24/15

• More robust FMS simulation capability enabled validation of Lincoln/MITRE results & expanded scope based on those results

• Key Lincoln/MITRE results and consequent research questions framed as specific hypotheses to be tested using LL FMSim: 1) Increased magnitude wind forecast error causes greater magnitude

RTA time error. 2) Aircraft/FMS types that do not maintain closed-loop control until the

RTA fix location have greater magnitude RTA time error as compared to systems that do.

3) Flights at lower cruise levels are more tolerant of wind forecast errors compared to flights at higher cruise levels.

4) Wind forecast errors closer to the RTA fix cause greater magnitude RTA time error.

5) Increased waypoint (wind forecast point) spacing causes greater magnitude RTA time error.

Lincoln FMSim Implementation

Winds/4D-TBO - 12 06/24/15

• Nominal HRRR field truth wind

• Constant forecast error applied to truth at waypoints 50 nm spacing

• FL350 cruise to 12,000 ft meter fix

• RTA assigned at 150 nm from meter fix

Hypothesis 1 Results Increased magnitude wind forecast error causes greater magnitude RTA time error

-60 -40 -20 0 20 40 60-50

-40

-30

-20

-10

0

10

20

30

40

50

Headwind Forecast Error (kts)

RTA

Erro

r (se

cond

s)

Headwind ScenariosTailwind Scenarios

RTA performance of B757/HW RL FMS relatively tolerant of constant forecast errors within a given range

B757/HW RL FMS

Winds/4D-TBO - 13 06/24/15

• Nominal HRRR field truth wind

• Synthetic wind error field created with peak of 5, 10 & 15 kts RMSE centered at variable location

• FL290 cruise to 12,000 ft meter fix

Hypothesis 4 Results Headwind forecast errors closer to the RTA fix cause greater magnitude RTA TE

Loc. 1 Loc. 2 Loc. 3 Loc. 4-15

-10

-5

0

5

10

15

RTA

Tim

er E

rror (

s)

RMSVE 5 ktsRMSVE 10 ktsRMSVE 15 kts

1 2

3

4

Approx. extent of wind error

field for location 3

Distance (nmi)

Altit

ude

(ft)

-150 150

30000

0

Peak error = 15 kts, h=100 nmi, v=14,000 ft15 kts

10 kts

5 kts

0 kts

Average RTA performance not greatly

impacted by error location relative to meter

fix, but variability is B757/HW RL FMS

Peak Error Location Relative to Meter Fix

Winds/4D-TBO - 14 06/24/15

Scenario Variable Overall Impact on Performance

Worse Performance From…*

Better Performance From…*

FMS RTA capability Major

FMS RTA tolerance Major Wider RTA tolerance Tighter RTA tolerance

Wind forecast error magnitude Major

Wind forecast error location relative to meter fix Medium

Truth wind variability Medium (correlates with error magnitude)

Cruise flight level Medium

Waypoint (wind forecast point) density Minor

Synthesized Findings: RTA Compliance Performance Drivers

Hi forecast error

Lo forecast error

Near forecast error

Far forecast error

Hi var truth wind

Lo var truth wind

High cruise level Low cruise level

Few cruise wind WPs Many cruise wind WPs

* All else being equal

Open-Loop in Descent

Full Closed-Loop Control

Winds/4D-TBO - 15 06/24/15

Integrated RTA Compliance Trade-Space For Aircraft/FMS and Scenarios Studied

Low (5 kts RMS)

Medium (15 kts RMS)

Hi (25 kts RMS)

0

10

20

30

40

50

60

70

80

90

100

A-FMS/Tol 6A-FMS/Tol 30

B-FMS/Tol 30

95%

RTA

Com

plia

nce

(sec

s)

Wind forecasterrorFMS RTA Capability/Tolerance

A-FMS=GE and HW RL FMS with full flight RTA controlB-FMS= HW BL FMS with

cruise RTA control only

“Best” performance for combination

“Worst” performance for combination

Range of expected RTA performance

Allows stakeholders to quantify relationship of RTA performance with wind error magnitude and automation capability for scenarios studied

Winds/4D-TBO - 16 06/24/15

• Need to know forecast performance as f(“look-ahead” time) – No up-to-date, comprehensive reports of forecast model performance

Wind Information Implications Flow Diagram

1. Define scenarios of interest

2. Identify performance trade-space

3. Select target performance level

4. Do feasible combinations of performance drivers meet target level of performance?

Yes (wind error limit)

No

5. Do feasiblecombinations of wind

forecast model and look-ahead time meet error

limit?

Yes

No

Identified need for enhanced

automation and/or wind model

“or”

6. Define procedure/ConOps/supporting technology needs to get required wind info accuracy to aircraft

Perf

Metri

c

Wind Information Quality

Low

Med

High

Sampletargetperformance

Aver

age

fore

cast

erro

r

Look-ahead time

Model ABC

Winds/4D-TBO - 17 06/24/15

Model (Producer) Domain Resolution and

Update Output Forecast Step / Horizon Aviation Users

GFS (NOAA/NCEP)

Global 0-192 hrs: 25 km

204-384 hrs: 80 km Update: 6 hours

3 hrs / 192 hrs 12 hrs / 204-384 hrs

Airlines (flight planning) Commercial vendors

Boundary conditions for RAP model

RAP (NOAA/NCEP)

North America

13 km 50 levels to 10 mb

Update: 1 hour

1 hour 18 hours

NOAA (AWC, Storm Pred. Ctr)

FAA (ATM, CWSUs, ITWS, TMA), CIWS,

CoSPA, Airline dispatchers

Commercial vendors (e.g., WSI, TWC)

Aviation wx research

HRRR (NOAA/NCEP)

CONUS 3 km

50 levels to 20 mb Update: 1 hour

1 hour, 15 min/ 15 hours

AWC, FAA ATCSCC, NCAR, NWP

Selected US Numerical Weather Prediction Models

Winds/4D-TBO - 18 06/24/15

• Assess RMS vector error (RMSVE) performance of each model at relevant forecast look-ahead times – 6+ hours: long range flight planning – 3-6 hours: short range flight planning – 1-2 hours: in-flight wind updates

• Model forecast performance analyzed for 10-month period (Nov 2013 – Aug 2014) and for four regions (SFO, PHX, ORD, EWR)

• Three forms of “truth” data considered – HRRR 0-hr analysis winds – Radiosonde observations (RAOB) – Aircraft met reports (AMDAR)

Wind Model Performance Assessment Approach

RAOB launch

3 RAOB sites

6 RAOB sites

5 RAOB sites

2 launches/site/day

3 RAOB sites

Winds/4D-TBO - 19 06/24/15

• Errors shown are averaged across all times, altitudes and regions

Wind Forecast Model Average Performance

On average, higher resolution forecast models have significantly better accuracy at <4 hr lookahead times, but similar performance to coarser models at >6 hr lookahead times

Winds/4D-TBO - 20 06/24/15

• Errors increase with forecast look-ahead time

• Long-tailed error distributions – Errors at any one time

may be significantly larger than the averages

• Large errors may persist for hours or days (see next slide)

Wind Forecast RMS Error Distributions for Different Look-Ahead Times

All forecast models have significant performance variability over time

Winds/4D-TBO - 21 06/24/15

Temporal Persistence of Large Wind Forecast Errors

6-month means

• Example shows persistent large (> 10 knots) 3-hour forecast errors during Feb 12-13, 2014 – Can we predict

when poor forecast wind model performance is anticipated?

Different forecast models often have similar performance trends, but not always

Winds/4D-TBO - 22 06/24/15

Case Study: Establishing Wind Information Needs and Associated ConOps/Datalink Needs to Support

a Given Level of Required 4D-TBO Performance

95%

RTA

Com

plia

nce

(sec

s)

Wind forecasterrorFMS RTA Capability/Tolerance

A-FMS=GE and HW RL FMS with full flight RTA controlB-FMS= HW BL FMS with

cruise RTA control only

1. Define scenarios of interest

2. Identify performance trade-space

3. Select target performance level

4. Do feasible combinations of performance drivers meet target level of performance?

Yes (wind error limit)

No

5. Do feasiblecombinations of wind

forecast model and look-ahead time meet error

limit?

Yes

No

Identified need for enhanced

automation and/or wind model

“or”

6. Define procedure/ConOps/datalink needs to get required wind info accuracy to aircraft

4D-TBO from cruise at FL290-390 to meter fix at 12,000 ft

±10 secs95% of time

(slice through

trade-spaceat 20 secs)

1) Any combination with 5 kts RMS wind error2) A-FMS/Tol 6 only with 15 kts RMS wind error3) No FMS/operation with 25 kts RMS wind error

Wind information neededto support any studied

aircraft/FMS combination

On average:GFS: cannot support 5 kts RMSE

RAP: <2.1 hrs lookaheadHRRR: <3.0 hrs lookahead

Winds/4D-TBO - 23 06/24/15

• Wind info quality impacts 4D-TBO procedure performance

• Analysis framework developed/exercised to explore relationships – Examples of what wind info & update rate meet target performance to

inform ConOps development and supporting technology needs – Limited set of Wind, ATC Scenarios and Aircraft/FMS types explored

• Findings informing range of stakeholders needs – e.g., RTCA Special Committees

• Next steps – Expanding Wind, ATC Scenarios and Aircraft/FMS types studied – Assessing near-term future FMS enhancements (10 altitude levels) – Predictability of wind errors in operationally-relevant time horizons

Summary & Next Steps