ai in decision making capability on board unmanned aerial vehicle

Artificial intelligence methodologies applicable to support the decision-making capability on board Unmanned Aerial Vehicles

Isabella Panella

Thales UK, Aerospace Division, Manor Royal, Crawley, UK [email protected]

Abstract The need for Unmanned Air Vehicles (UAVs) to operate autonomously and to manage their operation with minimal intervention from the ground control station, in order to reduce the datalink utilization and maximize their exploitation in beyond line of sight (BLOS) operations, has been long recognized within industry and research institutes. Many artificial intelligence (AI) techniques try to address the challenge of moving UAV towards full autonomy. However, no single technique has been able to provide the required autonomy for unmanned platforms. This paper presents a Unmanned Air Systems (UAS) architecture within which the different AI methodologies applicable to each subsystem are presented.

1. Introduction

It has long being recognized that the employment of Unmanned Air Systems (UAS) could provide significant advantages in military applications for ‘dirty, dull, and dangerous’ missions as well as for civil and commercial applications, such as search and rescue, border management, pipeline monitoring.

UAV performance can be improved by increasing the autonomy on-board of the platforms in order to reduce the workload placed on the operators and users. The implementation of autonomy on UAV requires the application of innovative and at times new ways of implementing the UAV on board functionalities, specifically the need to implement artificial intelligence methodologies within the functional architecture of UAVs.

This paper suggests a simple and highly modular UAV system architecture for the on board UAV mission system and identifies a relationship between the functionalities the UAV should present and the possible artificial intelligence methodologies that could be applied in order to achieve them.

The aim is to present a possible solution to the problem of autonomy on board of the platform and promote a hybrid approach to the issue of autonomous

systems, as single methodologies cannot address independently the complexity of the problem of machine autonomy.

The architecture specifies the relationship between the various subsystems that are seen to be key enablers to the autonomy of the vehicle combined with legacy systems and the artificial intelligence (AI) techniques that could be used in order to implement them.

The motivation behind this work is to provide a possible UAV functional architecture within which the AI methodologies can be embedded and the need for them to complement each others weaknesses and interrelationship appreciated.

In this paper, unmanned air platforms refer to both the air vehicle and the payloads fitted on the air vehicle in order to provide the required capabilities for the given mission.

The paper is organised as follows. Section 2 provides a definition of automatic versus autonomous. These terms are often confused and misused and it is appropriate to be clear about the difference between them. Section 3 provides an outline of the major functional elements of the platform and the payload is also provided.

Section 4 reports some of the key mission level functionalities and capabilities that are required for the systems to work autonomously. Section 5 identifies the AI fields and techniques that can be applicable to solve some of the identified issues and provides a mapping of those to the identified UAV’s functionalities. Diagrams of the architecture are also included. Finally, section 6 provides the conclusions of this study. 2. Automatic versus Autonomous

The major driver for the adoption of unmanned platforms is to have systems that can continuously provide information and situation awareness without the risk of losing human lives. The need of gathering information is driven by the need to make decisions and to react to the situation that presents itself as soon and effectively as possible.

Current UAV systems are mostly examples of automatic systems, which depend on pre-programmed

Bio-inspired, learning and intelligent systems for security

978-0-7695-3265-3/08 $25.00 © 2008 IEEE

DOI 10.1109/BLISS.2008.14

111

Bio-inspired, Learning and Intelligent Systems for Security

978-0-7695-3265-3/08 $25.00 © 2008 IEEE

DOI 10.1109/BLISS.2008.14

111

flight and require constant monitoring from the human operator (HO). Their use up to now has been described as the use of “goggles in the sky” by a human operator on the ground.

By automatic is meant a repetitive action that does not require external influence or control, but which repeats itself based on some set conditions. A well known automatic system is feedback control system, which automatically adjusts the input in order to obtain the desired outputs

The major difference between automatic and autonomous systems is that autonomous systems can change their behaviour in response to unanticipated events, whereas automatic systems would produce the same outputs regardless of any changes experienced by the system or its surroundings. In robotic applications, automatic actions allow the machines to be operationally autonomous, but they do not allow them to have decisional autonomy.

An autonomous action or event is defined as an action or event that is ‘independent in mind or judgment’, a self-directed, self-governing entity, not controlled by others or by outside forces.

Autonomous UAVs are herein defined as air vehicle systems with embedded autonomous functionalities.

In this research the focus is on identifying methodologies to allow the level of autonomy for UAVs to increase and the implementation of a dynamic, responsive behaviour on the platform itself to reduce the operator workload and provide independence from pre-programmed flight. 3. UAV Mission System

The main gaps in current UAV systems are represented by the reliance on datalink, the ‘dumbness’ of the control and payload systems, which means that they are not reactive to the environment, and the latency of information to the ground control station. In order to understand where the identified technological gaps can present a significant drawback within the UAV systems, the general UAV functionalities are captured.

A UAV mission system is seen to provide the translation of mission objectives into quantifiable, scientific descriptions that provide a measure to judge the performance of the platform, i.e. the system in which the mission objectives are transformed into system parameters. According to the AGARD report on Integrated Vehicle Management Systems [Ref. 1], the mission management functions of modern aerospace vehicle can be split into two different functional elements: the payload functions and the vehicle management function.

The payload functions are represented by the mission functional system, i.e. those systems which are mission specific, and they are defined in [Ref. 1] (chp1, pp1), as those functions that include the set of all functions that directly relate to the mission of a given vehicle

The payload of each platform is mission specific and it will vary with the specific mission requirement and it will need to be fitted on the air vehicle and its control defined and implemented. Examples of possible sensors that could be fitted on a UAV are: • Electro-Optical including Optical and IR sensors for

obstacle detection and target detection • Ground Moving Target Indicator (GMTI) radar and

Synthetic Aperture Radar (SAR) • GPS/INS navigation system.

The vehicle management system (VMS), on the other hand, is defined in [1] as the collection of functions that are required for the vehicle to understand, plan, control, and monitor the air vehicle operations. They usually represent the safety critical functionalities required for the safe employment of the platform; hence they include all the flight-critical and safety-related functions.

The mission system functional capabilities can be classified according to the level of autonomy that each of them would require if the HO was not included in the loop.

Three major levels of autonomy have been identified: • Low Level - The low level of autonomy can be

considered the reactive side of the UAV, where, given an event, the UAV will automatically react according to pre-defined limits and following given dynamic models. This level is characterised by the group of functionalities that would be required in order to fly the platform remotely. These include, but are not limited, to the flight control system, the actuator function, the engine or propulsion control, and the aircraft flight mechanics and air data acquisition.

• Medium Level – The medium level of autonomy captures the ‘reflective’ capability that should be embedded on board of the platform in order to enable it to reason about its internal and external state. That is the ability of the platform to detect internal faults or malfunctions, to self-regulate when subjected to unexpected events while carrying out the mission. The functionalities of the mission systems that have been identified to belong to this class are: o The flight path command and performance

envelope protection, such as the waypoint following system and the guidance and navigation functions;

o The health manager and fault tolerant control, in order to detect and react to possible system failures and malfunctions;

112112

o The power management system, included optimising the power consumption and maximising the mission time.

• High Level – The high level represents the most sophisticated layer of autonomy that it is desirable to provide on board the air vehicle. It provides the platform with decision making capabilities and with the ability of interoperate with other platforms and /or systems in order to gather and analyse information, as well as the ability to reason and act upon the conclusions it draws. The functional systems which have been identified to require a higher level of autonomy are: o the fault detection and identification (ID), i.e. the

ability for the platform to detect the malfunction, reason about it and take an appropriate action without any HO intervention;

o the situation awareness manager, responsible to maintain and update the state of the world (i.e. the representation of the environment in which the UAV is operating) and communicate detected changes to the mission manager.

A high level of autonomy is also required to make decisions and to provide interoperability with other systems. Therefore, the mission goal manager and sensor manager, as well as the integrated signal and data processing managers fall into this layer. The mission goal manager is responsible for redefining the mission objectives and goals and assessing their validity while cooperating with other air platforms or systems. The sensor manager needs to monitor, select, and allocate the sensor when the target is detected and decide both the required accuracy and precision of information for the given task, and also the data that need to be recorded in order to successfully support the mission execution. Data may come from the own sensor platforms mounted on the UAV or from other air vehicles or systems. The integrated signal and data processing manager has the task of processing the multiple source information, including the operator commands and reasoning about their soundness and applicability within the mission context, and also to process them into appropriate information useful by the other subsystems. Figure 1 captures the envisaged mission system architecture and the three major autonomy levels as described above.

It is important to observe that the application of artificial intelligence technique is envisaged to be used mainly in the high-level autonomy layer.

4. Identified mission level functionalities for UAVs

In this section the key mission-level capabilities for a UAV are discussed and linked to the above functional

systems. By developing those on the platform, the air vehicle is

envisaged to achieve the required level of autonomy for the drone. The key capabilities and the associated system where it can be built in pointed out are reported in Table 1. A possible application of these fields is provided in the implementation of the systems reported in Table 1.

��

��

5. Artificial intelligence techniques applicable to UAVs

Conventional control systems and deterministic optimisation techniques provide a means to deal with uncertainty within certain boundaries. However, the multitude of case scenarios faced by the UAV even before it starts its mission requires a real time adaptive system capable of reacting to unforeseen events to the best of its possibilities. Machines are now required to ‘think’ and ‘learn’ from the environment, as it has been pointed out in the previous paragraphs.

UAV capabilities Functional subsystem React to changes in environment and be capable of re-planning

Mission Planner Mission Goal Manager Sensor Manager

Navigation through complex terrain, possibly at high speed

Flight Path

Dynamic allocation of on board resources, i.e. fuel, power, sensors.

Power Manager Sensor Manager

Interoperability with other systems and platforms

Sensor and Integrated Signal and Data Processing Manager

Constant maintenance of situational awareness

Sensor Manager, Integrated Signal and Data Processing Manager Situational awareness Manager

Perform autonomous manoeuvre at the limits of the flight envelope to maximise performance

Mission Planner Flight path envelope protection

Provide on board data and information processing capability

Integrated Signal and Data Processing Manager

Operate outside communication link limits

Mission Goal Manager Mission Planner

Sense and avoid threats, collision avoidance

Sensor Manager Mission Planner

Autonomous reconfiguration of systems in case of fault

Fault Tolerant/Reconfigurable control Fault Detection and ID Manager

113113

��

Therefore, the introduction of artificial intelligence methodology in order to implement, enhance, and improve the UAV’s autonomous functionalities seems to be a natural evolution of more conventional feedback control theory.

The following branches of AI seem to be the most significant for potentially providing autonomous capabilities for the air vehicle. They were adapted partially from [2], pp. 248, and they are as follows: • Interpretation – consists of data analysis coupled with

domain knowledge forming high-level conclusions; • Prediction – projecting probable consequences of

given situations; • Diagnostic – Determining the cause of malfunctions

in complex situations based on observable symptoms. Identify abnormalities in the observed states of the UAV’s systems and possibly suggest remedies to mitigate fault;

• Design – finding a configuration of system components, which can meet the required performance while satisfying the constraints;

• Planning/Scheduling – devise a series of actions to achieve certain goal and co-ordinate them sequentially, for instance sensor pointing time or route planning;

• Decision Support/Decision Making – advise the HO, both on the ground or on a fighter aircraft acting as mission manager, to aid them in difficult cognitive tasks;

• Monitoring – identify changes in a system’s observed state;

• Risk Analysis – identify issues within a given course of action and plan/provide mitigations, such as alternative routes in case of detected obstacles;

• Data Analysis/Processing – for instance data mining, identification of trends, extrapolation of data;

• Optimisation – streamlining a system or object, such as the route plan or fuel consumption for given manoeuvre, to achieve the best performance, resource allocation;

• Classification – Assignment of a category to an object, for instance threat identification;

• Control of Systems – Governing the behaviour of complex systems. Manipulation of a system’s interaction with the world adjusting (actuating) the control surfaces, for instance to maintain a flight path

The applicability of those fields to the functional subsystems previously outlined is shown in the following table and it is based on the author’s engineering judgment.

�� !��"� ��

AI Field Functional Subsystem Interpretation • Mission Planner

• Situation Awareness Manager • Fault Detection and ID • Mission Goal Manager • Sensor Manager

Prediction • Mission Planner • Situation Awareness Manager • Fault Detection and ID

�� - ��

�� - ��

�� - !�� - �� "�� #��$�� - ��%��

�%�� %�� %�� %�� • �� • �� • �� • ��

Vehicle Management Functions

Payload Functions Mission Design Integration

UAV Mission System

Mission Objectives

Mission requirements

Surveillance/IFF

Target Detection and Tracking

Battle Damage Assessment/Reconnaissance/ Search and Rescue

�� &��

Mission Communication

Sensor Control/Sensor Fusion

Aerial Refuelling/Rearming

Deployment of Humanitarian Aids

!�� !�� !�� !�� • �� '� �� '�• ��• � �� %��'��'��'��'��((((��)��$�� )��$�� )��$�� )��$�� • ��*��• �� • �� '� ��

��

114114

• Mission Goal Manager • Sensor Manager

Diagnostic • Fault Detection and ID • Fault

Tolerant/Reconfigurable Control

• Health Manager Planning and Scheduling

• Mission Planner • Mission Goal Manager • Sensor Manager

Decision Support/Making

• Mission Planner • Situation Awareness Manager • Fault Detection and ID • Mission Goal Manager • Sensor Manager

Monitoring • Mission Planner • Situation Awareness Manager • Fault Detection and ID • Mission Goal Manager • Sensor Manager • Fault

Tolerant/Reconfigurable Control

• Health Manager Risk Analysis • Mission Planner

• Situation Awareness Manager • Fault Detection and ID • Mission Goal Manager • Sensor Manager

Data Analysis & Processing

• Sensor Manager • Integrated Signal and Data

Processing Manager Optimisation • Sensor Manager

• Power Manager • Situation Awareness Manager

Classification • Mission Planner • Situation Awareness Manager • Fault Detection and ID • Mission Goal Manager • Sensor Manager

Control of Systems • Health Manager • Flight Protection

Commands/Envelope protection

• Multifunction Integrated Navigation System

• Fault tolerant/Reconfigurable Control

It is important to observe that each subsystem requires

multiple AI fields in order to satisfy the required level of autonomy and that even though control theory in itself is

no longer sufficient to provide the additional capabilities of the UAV, it is still crucial to the implementation of adaptive and reconfigurable control subsystems. Also, optimisation is more and more being adopted for subsystems requiring resource allocation, planning and scheduling. In fact, heuristic searches and genetic algorithms are found to be well suited providing a quick and optimal solution when faced with multiple variable scenarios and incomplete information.

Over the years, several AI techniques have been developed, each claiming to provide significant advantages over the others. No individual technique has proved to be the answer to the problem of creating machine autonomy. Therefore, it is necessary to blend the different methodologies and provide a new level of integration in order to create hybrid systems. Only by combining different methodologies and matching them to the systems requirements it will be possible to move autonomy forward.

In this research, the focus has been on identifying methodologies that could allow an increased capability for a dynamic, responsive behaviour and its implementation on the platform itself so to reduce the operator workload and provide independence from pre-programmed flight. Therefore, this paper has focused on techniques, which are believed to be matured enough or most appropriate for the application to UAV.

The AI methodologies selected to provide autonomy to the UAV were:

• Artificial Neural Networks (ANN or NN) ([7], [2],[11]);

• Fuzzy Logic (Fuzzy) ([16],); • Genetic Algorithms (GA) (16], [11], [2]); • Reinforcement Learning (RL) ([4], [8], [13]); • Temporal Logic (TL) ([15], [2]); • Knowledge Based Systems (KBS); • Rule Base Systems (RBS) [14], [10], [11]); • Case Based Reasoning (CBR) ([9], [11], [2]); • Constrain Satisfaction Problem (CSP) ([3], [11]); • Model Based Reasoning (MBR) ([2], [11]). In order to understand what techniques could be

applied to the given AI fields in order to enable more autonomous functionalities on board of the platform, the selected techniques were tabulated against the identified AI fields and they were rated according to their applicability. The following table (Table 3) shows how each technique could contribute to the set tasks which need to be performed.

In order to provide interpretation and diagnostic capabilities to the UAV systems, it is important to introduce data fusion techniques and data mining procedures to quickly process and analyse the data. Information and data fusion become particularly important in the sensor management system, where the sensor information are collected and processed in order to

115115

provide the current world state situation awareness and update the belief functions on board of the UAV.

The following is a qualitative and judgmental analysis. The analysis carried out in the previous paragraphs led

to the definition of a functional UAV mission system architecture with possible AI techniques associated with those subsystems. The result is shown in Figure 2 and Figure 3. Figure 2 captures the high level decision making functionalities, usually performed by the pilot or the HO. Figure two represents the subsystems such as the guidance system that require a medium level of autonomy and the most inner part of the control loop, which can be fully automatic. The architecture has been subdivided into three components each of which groups together the functional subsystems according to their required autonomy level and to their functional flow of information.

The AI techniques selected for each subsystem are drawn from Table 3.

��#�$��!��%�� !��

NN Fuzzy GA RL TL KBS RBS CBR CSP MBS

Scheduling/ Planning Decision Support/ Decision Making

Diagnostic Risk analysis Data Analysis/ Processing

Monitor Optimisation Interpretation Classification Control System, Prediction Design

Legend:

7. Future work There are several issues that have not been addressed

in this paper. Future research should investigate the

advantages and drawbacks of implementing the individual systems with the suggested techniques and analyse the issues associated with their integration. Moreover, by implementing specific systems, different techniques may become more appealing compared to the one suggested.

Therefore, more research should be carried out on overall UAV functional system design and to their integration. The key difference to current systems is the inclusion of artificial intelligence techniques within the functional domain.

7. References [1] AGARD Advisory Report, ‘Integrated Vehicle Management

System’, AR 343, NATO, April 1996. [2] Luger, G. F., ‘Artificial Intelligence: Structures and

Strategies for Complex Problem Solving’, Addison Wesley, IV Edition, 2002

[3] Barták, R., ‘Constraint Processing’, IJCAI_07 Tutorial [4] Berenji, H. R., et alt.,‘Co-evolutionary Perception-based

Reinforcement Learning For Sensor Allocation in Autonomous Vehicles’, IEEE, 2003

[5] Dufrene, W. R., ‘Approach for Autonomous Control of Unmanned Aerial Vehicle Using Intelligent Agents for Knowledge Creation’, IEEE, 2004

[6] Grelle, C., Ippolito, L., Loia, V., and Siano, P., ‘Agent –based architecture for designing hybrid control systems’, Information Sciences, Vol. 176, pp. 1103-1130, 2006.

[7] Haykin, S., ‘Neural Networks: A comprehensive Foundation’, Prentice Hall, II Edition

[8] Harmon, M. E., and Harmon, S., ‘Reinforcement Learning: A Tutorial’.

[9] Lees, B., ‘9th UK Workshop on Case-Based Reasoning: Proceeding’, SGAI, December 2004

[10] Nilsson, N. J., ‘Introduction to Machine Learning: An early draft of a proposed textbook’, Robotics Laboratory, Dep. Of Computer Science, Stanford University, Dec 1996

[11] Russell, S. and Norvig, P., ‘Artificial Intelligence: A Modern Approach’, Prentice Hall, 2003

[12] Shirazi, M. A., and Soroor, J., ‘An intelligent agent-based architecture for strategic information system application’. Knowledge Based Systems, 2006.

[13] Ten Hagen, S. and Kröse, B., ‘A Short Introduction to Reinforcement Learning’, 7th Belgian-Dutch Conference on Machine Learning, pp 7-12, 1997

[14] Tunstel, E., et al., ‘Rule-based reasoning and neural networks for safe off-road robot mobility’, Expert Systems, Vol.19, No 4, September 2002

[15] Vila, L., ‘A Survey on Temporal Reasoning in Artificial Intelligence’, AI Communications, Vol. 7, No. 1, March 1994

[16] Johnson, J. and Picton, P., ‘Concepts in Artificial Intelligence: designing Intelligent Machines, Volume 2’, Butterworth-Heinemann editions in association with the Open University, 2001

Highly applicable

Potentially applicable

Not applicable

116116

��

��

�!� �

�%��

��

��

��

��

��

��

��

��

��

��%�%

��&�

��

� ��

��

Fau

lt D

etec

tion

&

ID

CB

R &

CSP

PA

YL

OA

D

Rad

io

Alt

SA

R/

GM

TI

INS

/G

PS

EO

Inte

grat

ed S

igna

l and

Dat

a P

roce

ssin

g

Inte

grat

ed

Dat

a F

usio

n P

roce

ssin

g

KB

S

AN

N, G

A,

RL

Inte

grat

ed

Sign

al

Pro

cess

ing

OU

TP

UT

S:

• R

ange

•

Ran

ge R

ate

• A

ngle

•

Pot

entia

l ID

, Si

ze, S

hape

Mis

sion

O

bjec

tive

s M

issi

on

Req

uire

men

ts

Mis

sion

Goa

l Man

ager

G

OA

L M

anag

er:

• U

pdat

e G

oals

; •

Elim

inat

e th

e on

ere

ache

d/un

achi

evab

le;

• G

ener

ate

new

goa

ls

CB

R, C

SP, R

L, G

A

Mis

sion

Pla

nner

C

SP,

RL

, GA

Wor

ld K

BS

Dat

abas

e M

BS

on A

ir

vehi

cle

Hig

h L

evel

Con

trol

On-

Boa

rd P

C

Situ

atio

n A

war

enes

s M

anag

er

K

BS

, GA

, RL

Sens

or M

ange

r

TL

, RL

, CB

R, G

A

MB

S on

Sen

sor

Fau

lt

Det

ecti

on &

ID

C

BR

, GA

, NN

Flig

ht P

ath

Com

man

ds

117117

��

��#�

�!� �

�%��

��

��

��

��

��

��

��

��

��

��

��

��&

��

��

��

DATALINK/INTERFACE

Flig

ht

mec

hani

cs/P

ropu

lsi

on/F

CS/

Act

uato

rs

H

ealt

h M

anag

er

CB

S, M

BS,

TL

Flig

ht P

ath

Com

man

ds

T

raje

ctor

y G

ener

ator

R

L,C

SP,T

L

Mul

tifu

ncti

on

inte

grat

ed n

avig

atio

n A

dapt

ive

Mod

e T

rans

itio

n K

BS,

RL

, TL

, AN

N

MB

S f

or

FCS

MB

S fo

r A

ctua

tors

M

BS

for

Eng

ine

Flig

ht C

ontr

ol/

Loa

ds/P

ropu

lsio

n/F

light

M

echa

nics

Env

elop

e P

rote

ctio

n R

L, C

SP

Flig

ht

Mec

hani

cs/P

ropu

lsio

n/

FC

S/E

lect

rica

l/Air

Dat

a

Fau

lt T

oler

ant

Con

trol

R

econ

figu

rati

on

MB

S &

CB

S, T

L, A

NN

, F

uzzy

Mid

-Lev

el C

omm

and

UA

V S

tate

s

UA

V S

tate

s

UA

VSt

ates

Low

Lev

el F

light

C

ontr

ol

U

AV

6- D

OF

Mod

el

FCS

NN

, G

A,

Fuz

zy

Eng

ine

Mod

el

Act

uato

r M

odel

Sen

sor

Inpu

ts

Ter

rain

dat

abas

e

Com

man

ds f

rom

M

issi

on M

ana g

er

118118

ai in decision making capability on board unmanned aerial vehicle

Documents