AI on the Battlefield: an Experimental Exploration

Alexander Kott 

BBN Technologies

Robert Rasch

US Army

Battle Command Battle Lab

Views expressed in this paper are those of the authors and do not necessarily reflect those of the U. S. Army or any agency of the U.S. government.

Kenneth Forbus 

Northwestern University

Outline

Motivation for the experiment The experimental rig Experimental procedure Findings A surprising challenge uncovered

The Role of BCBL-L

Exploration of new techniques and tool for Army C2 – a key focus of BCBL-LApparent emergence and maturing of multiple technologies for MDMPWhat is the right way to apply such technologies? Value? Drawbacks?BCBL-L proposed and executed the Concept Experimentation Program (CEP) - Integrated Course of Action Critiquing and Elaboration System (ICCES)

Room for Controversy

Some call for “…fast new planning processes… between man and machine… decision aids…” Extensive training and specialization requirements?Detract from intuitive, adaptive, art-like aspects of military command?Undue dependence on vulnerable technology? Make the plans and actions more predictable to the enemy?The experiment was designed to address such concerns

The Experimental RigCOA Creator, by the Qualitative Reasoning Group at Northwestern University - allows a user to sketch a COAThe COA statement tool, by Alphatech, allows the user to enter the COA statement Fusion engine, by Teknowledge, fuses the COA sketch and statementCADET, by Carnegie Group & BBN – elaborates the fused sketch-and-statement into a detailed plan and estimates

Input:Mission and Intelligence

Analysis

CADETTool

FusionTool

COAStatement

Tool

COACreator

Tool

Output:Detailed

Synchron.Matrix

The COA Entry Bottleneck

The key bottleneck in MDMP digitization: Time / effort / distraction Training requirements Downstream representation language

Our approach – COA Creator, based on nuSketch Sketching = interactive drawing plus linguistic I/O Rich conceptual understanding of the domain Speech often not preferred in mix of modalities Include “speechless” multimodal interface (buttons

plus gestures) Expressible in the underlying knowledge representation

Terrain features and characterization

Units and control lines

Objective and engagement areas

Friendly tasks are defined

The Experimental Procedure Comparison with the conventional process Exploratory vs. statistical rigor

Training

Interviews,Products Review

Team 1, Case 2Team 2, Case 2

Team 2, Case 1Team1, Case1

ConventionalManualProcess

ICCES-Based

Process

Key Findings Low training requirements

Largely due to “naturalness” of sketching Simple, frugal CONOPS

No impact on creative aspects of the process Largely driven by human-generated sketch-and-

statement Opportunity to explore more options

Dramatic time savings (3-5 times faster) Mainly in downstream processing (e.g., planning)

Comparable quality of products Few edits of ICCES-built products Comparable quantitative measures (e.g., friendly losses)

Parallel Experiments – Quality of Plans

0

0.05

0.1

0.15

0.2

0.25

0 2 4 6 8 10 12

CADET

Human

Rigorous experimental comparison: computer-assisted vs. conventionalMultiple cases, subject, judgesConclusions: indistinguishable quality of products, dramatically faster

Products of 5 past exercises

Grade by 9“Blind” Judges

Generate w/ CADET

Give “computer look”

inputsoutputs

Surprise: Plan Presentation is a Key Concern

Conventional output presentation paradigms, i.e., sync. matrix is ineffective Larger number of

elements Inadequate spatial aspect Difficult to detect errors

Alternatives: Animation? Cartoon sketches?

Conclusions

For Army professionals:Technologies like ICCES have near-term deployment potentialNo impact on creativity, predictabilityDramatic acceleration, comparable qualityChallenges in inspecting, comprehending the new MDMP products

For AI R&D community:Dominant role of HMI challenges calls for new mechanismsValue of natural sketch-based interfacesSimple, straightforward, all-in-one CONOPS for usersNo substitute for comparative experiments, from both practical and research perspectives

BACKUP SLIDES