Universal Mission Controller with Run-Time Ethics

Checking for Autonomous Unmanned VehiclesAUV 2012 Southampton UKIEEE Oceanic Engineering Society

25 September 2012

Don Brutzman, Bob McGhee, Duane DavisModeling, Virtual Environments, Simulation (MOVES)

Naval Postgraduate School (NPS), Monterey California USA

TopicsMotivation: human-taskable underwater

robots• Expressive power, semantics, syntax for a full

solutionRational Behavior Model (RBM) 3-layer

architecture• Strategic plans, Tactical operations, Execution

controlMission Planning and Execution• Declarative mission exemplar is helpful, general• What does it mean to define, apply ethical

constraints?Future Work• Broader integration, tasking with ethical

constraints

Motivating problem: ethicsHow can robotic systems behave

ethically?• Important for military operations abroad• Civil and scientific operations also coexist

dailyNot necessarily about religion or morality• Law of armed conflict internationally

recognized; often captured as Rules of Engagement (ROE) guidance for specific national or coalition forces

• Military readiness must be prepared for combat

Legal basis for civil activity is often well defined• Also measurable: are you vulnerable to

lawsuit?!

Motivation: human-taskable underwater robots

We’re looking to near-term future when UUVs can offload work and augment human activity

Various excellent robot solutions must remain compatible with human concepts and tasking• Necessary for mission planning and

justification• Otherwise the robots are simply not

autonomousExtensive/exhaustive prelaunch testing is

critical• For mission confidence and in-water reliability

This work is building an extendable architecture… • … for continued efforts bridging human, robot

logic

Our approachDefine missions that include ethical

constraints without relying on artificial intelligence (AI) or obscure abstractions for appropriate behavior• No embedded homunculus or ethicist engine

Design robot missions in way that can be adapted to a variety of disparate robot paradigms• Generally adaptable to tasking of diverse

systemsBuild on patterns that work well for human

groups cooperating on difficult, dangerous tasks• Otherwise still need human at end of comms

tether

Rational Behavior Model (RBM)

3-layer robot architectureStrategic• Declarative planning of goals while avoiding

obstacles and observing constraintsTactical• Operational control of navigation, tactical

and mission tasks. Sensor employment.Execution• Low-level control tasks. Open-loop, closed-

loop commands for propulsors and effectors.

Twenty+ years of well-documented, progressive work

Turing machineTuring machine (TM) • consists of a finite state machine (FSM) • augmented by an external agent in the form

of a potentially infinite memory • realized as tape of an “incremental tape

recorder”• Often referenced but seldom used• Extensive theoretical development has shown

that Universal TM has greatest computational power

• Clumsy to program, infrequently used• Nevertheless, appealing basis for theoretical

design since it maps to general theory of any computation

Prior work: two technical reports on Mission Execution

Automaton (MEA)We adapted Universal Turing Machine (UTM)

architecture to create a universal multiphase human-interactive MEA• Express fundamental logic of mission planning• Corresponds to RBM Strategic level

implementation• Goal: demonstrate computational generality

Implemented using Prolog language for easy backtracking during planning• Prolog dialect: Allegro Common Lisp• Interactive, text based • Human provides external-world sensing

responses

Goal-based Mission Example

• Simple yet general mission goals, decision logic

• Common approach, adaptable to other vehicles

• Extendable and refinable mission tasking

Goal 1. Proceed to Area A and search the area. If the search is successful execute Goal 2. If the search is unsuccessful, execute Goal 3.

Goal 2. Obtain environment sample from Area A. If the sample is obtained, execute Goal 3. If the sample cannot be obtained, proceed to recovery position to complete the mission.

Goal 3. Proceed to Area B and search the area. Upon search success or failure, execute Goal 4.

Goal 4. Proceed to Area C, rendezvous with UUV-2. Upon rendezvous success or failure, transit to recovery position to complete the mission.

Example Goal-based Mission Definition

Phase 1: Search Area A

Phase 2:

Sample Environment in Area APhase 3: Search Area B

Phase 4: Rendezvous with

UUV-2 in Area C

SucceedFail

FailFail Succeed Succeed

MEA Start

Sample Environment Phase Controller

Area Definition

Sampling Requirements

Operational and Ethical Constraints Checking

Queries

Responses: Succeed or Fail

Tactical Level

System State and Sensor Data

• Controller Parameters

• Sensor Settings, Responses

Phase 5: Transit Return to Base

Succeed

Fail

Mission Abort

Mission Complete

SucceedFail

Execution Level• handles real-time control and response

Tactical-Level modules with shared vehicle state

and telemetry history

VehicleCommands

Goal-based Mission ExampleStrategic Level

Real-time orders

Tactics module sequencer

Surface Recovery position

Mission execution rule set: simply loop through phases

Executable mission rule set: loop through phases

(<-- (execute_mission)(initialize_mission)(repeat) (execute_current_phase)

(done) !)(<-- (initialize_mission)

(abolish current_phase 1)(asserta ((current_phase 1))))

(<-- (execute_current_phase) (current_phase ?x) (execute_phase ?x) !)

(<-- (done) (current_phase 'mission_complete)) (<- (done) (current_phase 'mission_abort))

Executable mission phases for this example added to Prolog rule set

Adding ethical constraints to mission requirements

Following the leader:how do human teams accomplish tasks

ethically?

The same rules need to apply to unmanned systems.

Example ethical constraints, civil

Safe navigation, follow pertinent rules of road

Satisfactory navigational accuracy (GPS etc.)

Have received timely clearance to enter a specific geographic area for given time period• Also vertical clearance for underwater

depth zone or airborne altitude zoneSufficient vehicle health, power, safety statusMeet communication requirements for tasking• Identity beacon, transponder, AIS tracking,

etc.• Recording and reporting on situational

data, etc.

Example ethical constraints, military

Meet all relevant, international civil requirements

Identification friend foe (IFF), blue-force tracking• friendly/hostile/neutral/unknown

Prior determination of hostile intent• Robot option to warn without fear of self

protectionROE use of deadly force, weapons releasability• Brevity codes: weapons safe, hold, tight, free• Confirmation and permission requirements

After-action reporting, damage assessment

Et cetera, et cetera…

Key insight

Ethical behaviors don’t define the mission plan.

rather

Ethical constraints inform the mission plan.

Phase 1: Search Area A

Phase 2:

Sample Environment in Area APhase 3: Search Area B

Phase 4: Rendezvous with

UUV-2 in Area C

SucceedFail

FailFail Succeed Succeed

MEA Start

Sample Environment Phase Controller

Area Definition

Sampling Requirements

Operational and Ethical Constraints Checking

Queries

Responses: Succeed or Fail

Tactical Level

System State and Sensor Data

• Controller Parameters

• Sensor Settings, Responses

Phase 5: Transit Return to Base

Succeed

Fail

Mission Abort

Mission Complete

SucceedFail

Execution Level• handles real-time control and response

Tactical-Level modules with shared vehicle state

and telemetry history

VehicleCommands

Goal-based Mission Example, with constraints

Strategic Level

Real-time orders

Tactics module sequencer

Surface Recovery position

Constraint

ConstraintConstraint

ConstraintConstraint

Running mission example #1CG-USER(1): (?-

(execute_mission))Search Area A!Search successful? yesSample environment!Sample obtained? yesSearch Area B!Search successful? yesRendezvous UUV2!Rendezvous successful? yesReturn to base!At base? yesMission succeeded.Yes

User enters responses to mission execution automaton (MEA) queries, thereby providing environmental responses and testing mission logicThese correspond to ethics checks measured by robot, or supervised by a human

Running mission examples # 2, 3

CG-USER(2): (?- (execute_mission))

Search Area A!Search successful? yesSample environment!Sample obtained? noReturn to base!At base? yesMission succeeded.Yes

CG-USER(3): (?- (execute_mission))

Search Area A!Search successful? noSearch Area B!Search successful? noRendezvous UUV2!Rendezvous successful? yesReturn to base!At base? noMission failed.Yes

Human operators can logically test various mission alternatives to gain confidenceSimulation can also run through every

option exhaustively to further confirm correctness

Theory versus practice

In theory: theory and practice are the same.

In practice: they’re not.

- Yogi Berra

Challenge: broad implementation

Can we• Define mission goals readable by humans +

robots• Produce actionable tasking for different

AUVs• Produce software examples that can run

properly in simulation and in robots

Can we also• Define goal constraints ethically and

measurably

Yes

Appears possible

AUV WorkbenchAutonomous Unmanned Vehicle

Workbench supports underwater, surface and air vehicles• physics-based mission rehearsal• real-time task-level control of robot

missions, and• replay of recorded results• Industry-friendly open-source license,

Sourceforge• Basis: RBM 3-level architecture, AVCL

commandsUsed to rehearse strategic-level

MEAMission.xml• https://savage.nps.edu/AuvWorkbench

Tutorial on

Thursday

Autonomous Vehicle Command Language (AVCL)

AVCL is a command and control language for autonomous unmanned vehicles• Close correspondence to human naval

terminology• Common XML representations for mission

scripts, agenda plans, and post-mission recorded telemetry

Operators can utilize a single archivable and validatable format for robot tasking, results • directly convertible to and from a wide

variety of different robot command languages

https://savage.nps.edu/Savage/AuvWorkbench/AVCL/AVCL.html

AVCL vocabulary: AllCommandsUuv.xml

List, unit test all execution, tactical AVCL commands• Simple unit testing of workbench software• Confirm AUV Workbench interface, XML

validation• Mission unit tests for comprehensive coverage

likely to followAlso for other robotics platforms• AllCommandsUsv.xml AllCommandsUav.xml

AllCommandsUgv.xml• Each provides a vocabulary list showing the

current tactical repertoire (rather than runnable missions)

Example mission, as pseudo-code XML

<?xml version="1.0" encoding="UTF-8"?><UUVMission> <GoalSet> <Goal area=”A” id=”goal1”> <Search nextOnSucceed=”goal2” nextOnFail=”goal3”/> </Goal> <Goal area=”A” id=”goal2”> <SampleEnvironment nextOnSucceed=”goal3” nextOnFail=”recover”/> </Goal> <Goal area=”B” id=”goal3”> <Search nextOnSucceed=”goal4” nextOnFail=”goal4”/> </Goal> <Goal area=”C” id=”goal4”> <Rendezvous nextOnSucceed=”recover” nextOnFail=”recover”/> </Goal> <Goal area=”recoveryPosition” id=”recover”> <Transit nextOnSucceed=”missionComplete” nextOnFail=”missionAbort”/> </Goal> </GoalSet></UUVMission> Readable either by human or

robotCaptures logic of canonical

mission

Corresponding example:

MEAMission.xmlusing AVCL xml

AUV Workbench MEA mission

MeaMission.xml results

Search planningproduces

waypoints

Search complete, return

Launch Recovery

Generality to other robotsHigh-level mission planners tend to be

dissimilar• Much of AI can be posed as search algorithms• Various RBM strategic-level implementations

possibleStrong match between RBM tactical,

execution AVCL vocabularies and many other robots• We’ve written many AVCL importers/exporters

Generality principle still holds, however: • Outputs of higher-level mission planners can

be expressed as AVCL tactical commands!• Perhaps unsurprising since we’ve designed

AVCL command sets using human procedures at sea

Looking ahead

Thoughts on autonomous ethicsEverything old is new again• Largely unexplored or unconsidered

territory?• Significant history of experience: mine

warfareAssume success… then what?• Even if perfectly executable, proper robot

logic is not useful unless it is a directly extension of warfighter logic

• Rules of Engagement, Concepts of Operations, Doctrine, Tactics, etc. must be expressible in equivalent terms in order to be effective, usable

• Civilian equivalents: Rules of Road, safety, etc.

Multi-layer robot architectures… onward and

upward!3 layers defined in Rational Behavior Model

(RBM) robot architecture are fairly common in robotics• Strategic: planning, goal evaluation, strategy• Tactical: navigation, operations, mission conduct• Execution: low-level control of systems

Autonomous Vehicle Command Language (AVCL) • Continue mappings to different robot dialects

Possible 4th layer above: Rules of Engagement?• provide ethical constraints and boundary

conditions on robot strategic planning and tactics

• Must work cooperatively, satisfactorily with humans

NPS wargames and workshops include robot tasking, ethics

issuesFuture Unmanned Naval Systems (FUNS)

wargame explored long-term autonomy issues for robot groups sharing reconnaissance tasks

CETMONS workshop explored robot ethics• Sponsored by USNA Stockdale Leadership

Center• Can robots ever be given lethal weapons?• Both humans, robotic systems make errors in

war• If robots make fewer errors than humans, then

likely it becomes immoral to avoid using robots!

• Slippery slope but includes potential benefits also

Next steps, wish listKeep building realistic models for AUV

Workbench• Have added our first ROV model + joystick

control• Mission reports, KML, X3D graphics

visualization• Control and dynamics coefficients, sensor

models• Emphasis on education and undergraduate

studentsDevelop larger repertoire of missions• XSLT stylesheet to convert AVCL strategic

mission into usable source code (Prolog, Java, C++, etc.)

• Let virtual world provide sensor responsesIntegrate standard Prolog within AUV

Workbench• Gnu open-source Java-based Gnu Prolog

Acknowledgements and Discussion

This work would not be possible without many years of work and guidance by Bob

McGhee.

Collaboration, contributions and dialogare always welcome

Don Brutzman, Ph.D.brutzman@nps.navy.mil

brutzman@nps.navy.smil.mil http://faculty.nps.edu/brutzman

Code USW/Br, Naval Postgraduate SchoolMonterey California 93943-5000 USA

1.831.656.2149 work1.831.402.4809 cell

Contact

Bob McGhee, Ph.D.robertbmcghee@gmail.com

MOVES InstituteNaval Postgraduate School

Monterey California 93943-5000 USA

Contact

CDR Duane Davis USN (Ret.) Ph.D.dtdavi1@nps.edu

Code CS/DaNaval Postgraduate School

Monterey California 93943-5000 USA1.831.656.7980 work

Contact