goddard space flight center 6 october 2004 pursuit/evasion of fixed-wing aircraft through...
TRANSCRIPT
Goddard SpaceFlight Center6 October 2004
Pursuit/Evasion of Fixed-wing Aircraft through Model-Predictive ControlThe new album by
Jonathan SprinkleFeaturing, Mike Eklund, Jin Kim
and Shankar Sastry
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 2
Overview
• Motivation• NMPC and PEGs
– Earlier work– Algorithm– Early results
• What’s the idea? (proof of concept)– Architecture– Tools – Simulation results
• Capstone Demonstration– Background– Development and simulation
environment– Experimental test flight results
• Featuring great movies!
• Future Work & Conclusions
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 3
Motivation
• UAVs are very useful for intelligence gathering– Small, low-observable, inexpensive, remote-
piloted (with autonomous tendencies…)
• Time-lag associated with remote control– Acceptable, or annoying, when in “steady-
state” flight• Waypoint flying, high-altitude loitering or
scanning
– Tactically interfering when rapid feedback is required
• Landing, hazardous weather, or stressful maneuvers
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 4
Motivation
• Consider the following:– Intelligence gathering UAV in flight– Alerted (internally, or by an observer) of an
adversary over the horizon• Either piloted, or perhaps fired (missile)
– Large amounts of data recently gathered, but not transmitted
• What to do?– Time lag is too large for remote “escape”– Adversary will most likely locate UAV shortly– Not enough time to transmit back all data
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 5
Motivation:
• Our proposal:– The pilot should be able to turn over
autonomy to the aircraft• will take evasive maneuvers as best it can• not necessarily guaranteed to save the aircraft,
but hopefully will be able to transmit back data
– Aircraft will return autonomy if it escapes– The use of various strategies for control will
enable this capability, and do so in an aircraft independent manner
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 6
Model Predictive Control
• MPC is a method for restricting/encouraging behavior
• A “fortune teller” controller• Restricts input ranges, as well as encourages
some inputs based on safety/stability concerns• Very useful for nonlinear systems, due to the
ability to get good optimizations with non-linear abstractions
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 7
How does it work?
• Basic algorithm:– Examine the mathematical abstraction of the system
(PDE)– Determine value along N time steps into the future– Optimize this value, according to some a priori
specifications (to )J = 0
J = Á(b1N ::M N) +
N ¡ 1X
k=0
L(x;u;b1::M ) = 0 (1)
Á(b1N ::M N) = C
m=MX
m=1
bTmB0m
bm (2)
L(xk;uk;bk1::M ) , C
Ã
xTk X 0xk + uT
k U 0uk +m=MX
m=1
bTmk
B0mbmk
!
(3)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 8
System Model
• In the form of,
• Obviously, very system-dependent• Sometimes an abstraction of the actual
system in order to speed up computation
• Accuracy of the prediction, directly tied to the abstraction
• Eventually, arrive at a snapshot N steps in the future[x1;x2; : : : ;xN ]
_x = f (x;u)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 9
Example: Aircraft Control
Begin
End
L(¢) , xTk X 0xk +uT
k U 0uk + bTm1
B01bm1
(1)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 10
Example: Aircraft Control
Begin
End
Obstacle
L(¢) , xTk X 0xk +uT
k U 0uk + bTm1
B01bm1
(1)
+ bTm2
B02bm2
(3)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 11
Example: Aircraft Control
Begin
End
Obstacle
Boundary
L(¢) , xTk X 0xk +uT
k U 0uk + bTm1
B01bm1
(1)
+ bTm2
B02bm2
(2)
+ bTm3
B03bm3
(3)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 12
Example: Aircraft Control?
Enemy…
Begin
End
Obstacle
Boundary
L(¢) , xTk X 0xk +uT
k U 0uk + bTm1
B01bm1
(1)
+ bTm2
B02bm2
(2)
+ bTm3
B03bm3
(3)
+ bTm?
B0?bm?
(4)
(5)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 13
Example: Aircraft Control
• Now, what do you do?– Hope that you don’t get caught?– First, fight with you left hand, and then
surprise you opponent by not being left-handed
– Encode “getting away” from your opponent into the cost-function
“I admit it, you are better than I am”“Then why are you smiling?”“Because *I* am not left-handed”
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 14
Earlier UC Berkeley Pursuit/Evasion Experiments (David Shim, Jin Kim, et al)
• BErkeley Aerobot Project (BEAR), 1996-– Goal: to build a coordinated, intelligent
network with multiple heterogeneous agents• 11 Rotorcraft-based unmanned aerial vehicles
(UAVs) • 5 Unmanned ground vehicles (UGVs)• Shipdeck simulator (landing platform)
• Stochastic Pursuit-Evasion Games (PEG)– Self-localization– Target detection– Map building– Pursuit policy– Trajectory generation– Control / Action
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 16
Nonlinear Model Predictive Trajectory Control (NMPTC)
• Explicitly addresses nonlinear systems with constraints on operation and performance
• A cost minimization problem in the presence of state and input constraints– Control resulting in the minimum cost is
determined over a model predicted horizon
• Previously demonstrated in rotary wing UAVs[1]
[1] H.J. Kim, D.H. Shim and S. Sastry, ACC, 2002
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 17
NMPTC: Fixed wing application
• Dynamics and constraints are quite different than for rotary wing aircraft– Entirely new aircraft
model required
• Tactics– Function of
constraints on fixed wing aircraft, in particular
• Minimum airspeed • Maximum turn rate
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 18
NMPTC: For Pursuit/Evasion Games
• In this application, the low level control is done independently by the platform– Basically a tactics & trajectory generation
problem
• Same controller is used for pursuit and evasion– Cost function components and gains are
changed• E.g. AOT not used for pursuer
• Implemented as stand-alone application with Matlab interface for testing and controller tuning
• Integrated in OCP for Capstone Demonstration (using a real flight avionics feedback model)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 19
NMPTC: Cost Function Definition
• Cost function is defined by:
1
0
0
, , 1 1
2 2
y x, y,u,p,d,a
where
1 y y y
2 and
1 1 1 x, y,u,d,a y y x x u u
2d d a a
x is the state
N
Nk
TN N N
T T T Tk k k k k k i k i i k
T Tn mk k k k
J L
P
L Q S R p B p
G T
vector u is the control vector
y is the trajectory error d is the pursuer/evader position difference
p is the evader
distance from the boundary
a is the angle off tail (AOT) or other tactical functions
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 20
NMPTC: Cost function illustration
, , 1 1
2 2
1 1 1 x, y,u,d,a y y x x u u
2d d a a
T T T Tk k k k k k i k i i k
T Tn mk k k k
L Q S R p B p
G T
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 21
NMPTC: Cost function minimization
• A gradient decent method is used to minimize the cost function
• Initialization with previous result reduces the number of iterations required– Usually 3-4 iterations are required
• The number of iterations in limited to prevent overruns in real-time– In rapidly changing situation this can result in
suboptimal solution– Sudden changes may take several time steps– However, this is not a problem because the
situation is changing and unpredictable anyway
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 22
NMPTC: PEG game conditions
• In this game the evader tries to cross the playing area while not being targeted by the pursuer
• The pursuer tries to target the evader
Target cone definition (θ=10˚,d=3 nm)Left: F15 not behind UAV, middle: F15 not pointed at
UAV, right: F15 behind AND pointed at UAV
F15
UAV
F15
UAV
F15
UAV
(b)(c)(a)
F15
UAV
F15
UAV
F15
UAV
(b)(c)(a)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 23
NMPTC: Early simulation results
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 24
NMPTC: Early simulation results
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 25
NMPTC: Early simulation results
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 26
Example: Cost functions
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 27
Why are we doing this?
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 28
Software Enabled Control
“The goal of SEC is to develop new controls and software technology that will enable new applications that are impractical or intractable using current approaches.”
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 29
SEC Capstone Demonstration
• Capstone Demonstrations were proposed to highlight and test the technologies developed in the SEC program
• One would be a fixed wing UAV flight test– 6 participant technology developers (TDs)
• Honeywell, Northrop Grumman, U Minnesota, MIT, Stanford, and UCB/U Colorado/CalTech
– System Integrator was Boeing– OCP would be software framework– A T-33 trainer as UAV surrogate– An F-15 as wingman/opponent
• 12-14 month schedule May 03 – June 04
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 30
Demo: Non-Determinism in Experiments
• Customer comment:– “The timing and scope of these (pop-up events, like
faults, targets, threats) should not be totally known a-priori by the TDs.”
• Capstone Demo Approach– For flight test safety, platform owners and flight
crews are more comfortable with constraints on non-determinism
• Temporal non-determinism• Spatial non-determinism
• UCB Pursuit/Evasion Approach:– Two airplanes, no pre-determinism at all– Favorite responses at the last PI meeting:
• “You really don’t know who is going to win?!?!”• “What day is your test flight scheduled? I’ll be there”
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 31
Demo: Dryden Flight Research Center (DFRC)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 32
Demo: DFRC Flight Demonstration Venue
• NASA provides range access, and approves test vehicles and missions
• Boeing provides vehicles and flight plans, and conducts flights
• NASA facilities– Fully instrumented Western Aeronautical Test Range– Mission Control Center – Restricted public access
• Boeing facilities– UCAV Flight Operations Control Center – Boeing only
• Flight test instrumentation, displays, telemetry recording, post-processing
– UCAV Van – DoD only• SEC Workstation control and display, Link 16 Terminal
– TDs were to be provided with a tent• But we got to share a camper van with A/C with the VIPs
some of the time
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 33
Demo: R-2508 Range Complex
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 35
Demo: Mission Area Example in R-2515
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 36
Demo: Mission Guidelines
• Day, Visual Flight Rules – “See and Avoid”• Mission Schedule
– Flight duration: 1:10 in mission area for both aircraft– Two flight per day can be supported
• Airspeed– T-33 alone: 250 kts typical– T-33 and F-15 joint operations: 300 kts typical
• Altitude– Normal operations: 10,000 ft to 20,000 ft MSL– Limited operations: Below 10,000 ft to surface with approval– Edwards-Dryden runways: 2,300 ft MSL– Terrain in R-2515: 2,300 ft to 6,000 ft MSL
• Formation– 1 nmi radial separation– +/- 500 ft
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 37
Demo: TD deliverables
• Software– Design consistent with defined OCP interfaces– Source code
• For emergency fixes and recompilations• For safety of flight reviews
– Binaries for linking– OCP application artifacts
• Simulink mdl file, ComponentInfo files• Test check cases
– Allow us to quickly assess correct functional behavior when hosting in our desktop development system and in platform integration labs (F-15, UCAV)
• Documentation– Describing software– Describing test check cases
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 38
Demo: State data available for control
• All UAV state data available – Limits due to lack of sensors on T-33, e.g.
• alpha, beta measurements
• Different set of F-15 state available– Transmitted by F-15 to the T-33
– Rate information• Data captured from UCAV avionics at 20 Hz• Baseline TD application execution rate is 2 Hz• Freshest 10 state data sets buffered as input at 2 Hz• From F-15 at 1 Hz
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 40
Demo: T-33 Software Architecture
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 41
Demo: Final integration and testing
• Development in desktop environment at UCB
• Sent to Boeing– Testing in UCAV AIC (HWIL) facility.
• Confidence testing of Laptop Demo Applications• Use scripted scenarios with actual mission plans
for Dryden
– Multi-Vehicle configuration testing– Integration of TD Software Components
• Integration into new OCP release B2.6.x
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 42
Demo: UCB PEG Development
• 20 – 60 min. games confirm NMPTC feasibility at real-time– Evader goal: get to final
waypoint or avoid evader– Pursuer goal: ‘target’ evader
• Pursuer and evader restricted to same performance limits
• Planes on the same logical plane, but separated by 6000ft altitude at all times
• Evader and pursuer have a few scenarios– UAV as evader – UAV can become pursuer
OCP Experiment Controller SnapshotT-33: Evader (yellow)F-15: Pursuer (blue)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 43
Demo: Controller Coverage
• Final waypoint is the major component
• Travel to the waypoint can be interrupted by the actions of the pursuer– Relative positions– Angle-off-tail– Distance from each other– Proximity to boundaries of the
testing area– Physical limitations of the
aircraft• Velocity, climb/turn rate
• Use a simpler model for predictive behavior under certain conditions– Still use the Boeing-supplied
(DemoSim) model for behavioral simulations
Approx. same timePursuer goes fortarget cone
Evader turns away(regardless of endpoint)
Endpoint
Target cone definition (θ=10˚,d=3 nm)Left: F15 not behind UAV, middle: F15 not pointed at
UAV, right: F15 behind AND pointed at UAV
F15
UAV
F15
UAV
F15
UAV
(b)(c)(a)
F15
UAV
F15
UAV
F15
UAV
(b)(c)(a)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 44
UCB PEA Experiment #1 Test Plan:UAV as Evader
• UAV attempts to cross Scenario Area (SA) from East to West without being targeted by the F15
• UAV “wins” by:– Reaching the RVPT– Not being targeted for 20 minutes
• F15 “wins” by targeting the UAV
• Note: F15 performance is restricted
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 45
Crew/Pilot Instructions
EventNo.
Event UAV Task F15 Task What to Expect
1 Move to stations Under pilot control, west of SA from INPT between 10,000’ and 12,000’ altitude, heading 90°
In SA, east of W117° 30’ between 18,000’ and 20,000’ altitude
2 Set up UAV Enters SA at INPT under pilot control J-UCAS Mission Plan sets up UAV in hand over
conditions
As above
3 Verification of UAV setup
Monitor Hdg, Alt, Spd commands consistent with handover conditions
As above
4 Start experiment Press CMD-ON button Engage UAV
5 Pursuit-Evasion proceeds
Attempts to reach RVPT while not being targeted by F15
Note: game can be aborted by pressing the “GO EGPT” button
Attempts to target UAV
Interesting maneuvers
6 “Win” condition detected
TD software monitors game conditions and outputs appropriate “win” conditions
As above “WIN: F15 - targeted UAV”“WIN: UAV - reached End Zone!!”“WIN: UAV – time expired”
7 End game or continue playing
Game continues after “win” condition by default to allow experiment to continue
Press GO EGPT to end
As above
8 Return to EGPT Press CMD-OFF button, pilot flies to EGPT Fly to EGPT or remain in SA if further experiments
UAV returning to INPT/EGPT
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 46
UCB PEA Experiment #2 Test Plan:UAV as Evader and Pursuer
• UAV attempts to cross Scenario Area (SA) from East to West without being targeted by the F15, however, UAV will attempt to target F15 if suitable conditions arise
• UAV “wins” by:– Reaching the END ZONE– Not being targeted for 20 minutes– Targeting the F15
• F15 “wins” by targeting the UAV
Note: F15 performance is restricted
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 47
Demo: Experiment #2 Test Plan
EventNo.
Event UAV Task F15 Task What to Expect
5 Pursuit-Evasion proceeds
Attempts to reach RVPT while not being targeted by F15
GIB can select one of three modes:
UAV as evader by pressing “Evade” button
UAV as pursuer by pressing “Pursue” button
UAV determines whether to evade or pursue by pressing the “Ev/Pu” button
Attempts to target UAV
More interesting maneuvers
Test plan is identical to Experiment #1 except for Event No. 5:
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 48
Demo: Experiment #1 Simulation Results
• Using 2nd UAV pretending to be an F15
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 49
Demo: Experiment #2 Simulation Results
• Using Fixed Wing Control Interface with simple way point plan to allow Boeing hardware testing
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 50
Demo: Experiment #2 Simulation Results
• Using 2nd UAV pretending to be an F15 to allow for controller tuning and software testing
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 51
Demo: Flight Test
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 52
Demo: Flight Test Results 1
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 53
Demo: Flight Test Results 1 (faster)
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 54
Demo: Flight Test Results 2
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 55
Demo: Flight Test Feedback
• Very positive responses• Pilot comment:
– UAV behaved as he would have expected a trained pilot
• Results still being analyzed for final report
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 56
Future: Improving implementation
• Cost functions of the NMPTC can be improved– Check against known fighter tactics, try to duplicate
behavior– Using desktop environment
• Enhancing/verifying our predictive model– Boeing’s model is an executable– We are using a mathematical version
• approximations are necessary• results in loss of accuracy, but increased performance
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 57
Conclusions
• MPC can be used to provide interesting behaviors for linear and non-linear control systems
• We hope to reduce the development cycle by at least– Providing a cost-function independent optimizer– Inventing an intuitive interface to generate the cost
function– Developing a method/tool to tune the cost function
for desired behaviors– Experimenting with ways to reverse engineer values
for the matrices, based on desired behaviors under stimuli
• Experience in the SEC program and with OCP has been very positive
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 58
Questions?
“Well HAL, I’m damned if I can find anything wrong with it.”“Yes. It’s puzzling. I don’t think I’ve ever seen anything quite like this before.”
-- 2001: A Space Odyssey
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 59
Future: NMPTC plans
• Currently implementing a new NMPTC problem using different models and designs
• Will be developing the NMPTC interface to provide the behavior for this new application
• Evaluate the new MATLAB MPC toolbox, to see what benefits it offers
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 60
Future: The NMPTC Problem
• Making it work is nice, but– How in the devil did we come up with those
• Equations• Individual components• Matrix values
– Is there a way to derive these from the application constraints?
• Additionally– How hard was it to write a fast optimizer?– Is there a way to make this interface easily usable?
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 62
Future: Toward a solution
• System-dependent– Can be derived for a particular system’s
mathematical definition– In general, quite easy to obtain
• Independent– Software engineering exercise– Once defined, will be reused
• Behavior-dependent– By far the hardest piece of the solution– Not generally derivable, but there are tricks
that should be available for all future implementers, that a parameterized approach can provide
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 66
And of course, NASA...
• How many of you spent at least 24 hours per day for the past 3 weeks working on H&RT Proposals? – Can I get an ‘amen’?
• Boeing-led– Open Robotic Space Control Platform (ORSCP)– for transitioning OCP to space exploration:– Co-Lead is UCB (Shankar Sastry)
• One of two SEC Capstone Demo participants involved
• Berkeley-led– Infrastructure & Architecture for Crew-centered
Operations (IACO)– for deployment of mission-critical decisions to
spacecraft and autonomous robots, rather than mission control
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 67
Demo: Simulation Results 1
10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 68
Demo: Simulation Results 2